2024-09-19T04:46:14.509 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T04:46:14.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:46:14.635 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911652' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5bCq8p3N9V3LLYXcDpAm2KzVJXg5mREFBjXQ2pVJce3k8TspHcHsb9w2jcwyaVaOKvZI8Ea7b8lVTaKToNd1E= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2NbXQHvL1Fanx4Q9j9JN0Re4h5aPvFJ/3pkllyOYwp8P9otdPYGeORsoEU4cTsiFoNgLObu2mXf8GWVqFiiZw= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBND3JdlPwchgJJ+0iDaJWQWQYEtLi5oIG4OBG0ZlHzGvkzDtPU1gV+Un6DGKDPizEqqp3TJsH2sfwp2TI0faAhg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T04:46:14.635 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T04:46:14.636 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T04:46:14.636 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T04:46:14.638 INFO:teuthology.task.internal:Checking packages... 2024-09-19T04:46:14.660 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T04:46:14.660 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T04:46:14.660 INFO:teuthology.packaging:ref: None 2024-09-19T04:46:14.660 INFO:teuthology.packaging:tag: None 2024-09-19T04:46:14.660 INFO:teuthology.packaging:branch: squid 2024-09-19T04:46:14.660 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:46:14.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T04:46:14.901 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T04:46:14.903 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T04:46:14.912 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T04:46:14.912 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T04:46:14.952 INFO:teuthology.task.internal:Saving configuration 2024-09-19T04:46:14.972 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T04:46:14.988 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T04:46:15.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:40:58.609741', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5bCq8p3N9V3LLYXcDpAm2KzVJXg5mREFBjXQ2pVJce3k8TspHcHsb9w2jcwyaVaOKvZI8Ea7b8lVTaKToNd1E='} 2024-09-19T04:46:15.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:40:58.607288', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2NbXQHvL1Fanx4Q9j9JN0Re4h5aPvFJ/3pkllyOYwp8P9otdPYGeORsoEU4cTsiFoNgLObu2mXf8GWVqFiiZw='} 2024-09-19T04:46:15.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:40:58.608612', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBND3JdlPwchgJJ+0iDaJWQWQYEtLi5oIG4OBG0ZlHzGvkzDtPU1gV+Un6DGKDPizEqqp3TJsH2sfwp2TI0faAhg='} 2024-09-19T04:46:15.048 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T04:46:15.062 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T04:46:15.062 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T04:46:15.062 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T04:46:15.062 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T04:46:15.200 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f42d562b520>, signals=[15]) 2024-09-19T04:46:15.200 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T04:46:15.211 INFO:teuthology.task.internal:Opening connections... 2024-09-19T04:46:15.211 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T04:46:15.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:46:15.289 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T04:46:15.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:46:15.368 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-09-19T04:46:15.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:46:15.443 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T04:46:15.454 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-09-19T04:46:15.472 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-09-19T04:46:15.472 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-09-19T04:46:15.530 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:46:15.531 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:46:15.532 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:46:15.532 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-09-19T04:46:15.552 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-19T04:46:15.571 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-19T04:46:15.571 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:46:15.629 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:46:15.630 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:46:15.630 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-19T04:46:15.653 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-09-19T04:46:15.671 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-09-19T04:46:15.672 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-09-19T04:46:15.729 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:46:15.730 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-09-19T04:46:15.731 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:46:15.731 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:46:15.731 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:46:15.731 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:46:15.731 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:46:15.731 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:46:15.731 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-09-19T04:46:15.753 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T04:46:15.765 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T04:46:15.839 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T04:46:15.839 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:46:15.841 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:46:15.844 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:46:15.861 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T04:46:15.877 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T04:46:15.877 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:46:15.898 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:46:15.901 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:46:15.931 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T04:46:16.055 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T04:46:16.122 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T04:46:16.122 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T04:46:16.122 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:46:16.123 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:46:16.123 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:46:16.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:16.141 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-19T04:46:16.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:16.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:16.144 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-09-19T04:46:16.145 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-19T04:46:16.199 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-19T04:46:16.199 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-503.el9.x86_64 2024-09-19T04:46:16.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-09-19T04:46:16.202 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-503.el9.x86_64 2024-09-19T04:46:16.202 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.14.0-503.el9.x86_64 2024-09-19T04:46:16.202 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2024-09-19T04:46:16.203 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-503.el9.x86_64 2024-09-19T04:46:16.203 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-503.el9.x86_64 2024-09-19T04:46:16.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-09-19T04:46:18.719 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-19T04:46:18.773 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-19T04:46:20.533 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-19T04:46:20.987 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.3 MB 00:03 2024-09-19T04:46:22.680 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-19T04:46:23.420 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-09-19T04:46:25.109 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-19T04:46:26.826 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-19T04:46:27.121 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-19T04:46:27.938 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-19T04:46:28.554 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-19T04:46:29.397 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-19T04:46:30.146 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-19T04:46:30.451 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-19T04:46:31.217 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-19T04:46:33.628 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T04:46:35.576 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T04:46:35.576 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T04:46:35.672 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-19T04:46:35.680 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:46:35.681 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 79 M 2024-09-19T04:46:35.682 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 126 M 2024-09-19T04:46:35.682 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-19T04:46:35.815 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-19T04:46:35.912 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 332 kB/s | 36 kB 00:00 2024-09-19T04:46:36.565 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-19T04:46:36.621 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-19T04:46:37.080 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-19T04:46:37.438 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-19T04:46:37.440 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:46:37.440 INFO:teuthology.orchestra.run.smithi064.stdout:Total 45 MB/s | 79 MB 00:01 2024-09-19T04:46:37.672 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T04:46:37.672 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T04:46:37.769 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T04:46:37.777 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T04:46:37.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:46:37.778 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-19T04:46:37.778 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-09-19T04:46:37.778 INFO:teuthology.orchestra.run.smithi174.stdout:Install 4 Packages 2024-09-19T04:46:37.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:46:37.778 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 79 M 2024-09-19T04:46:37.779 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 126 M 2024-09-19T04:46:37.779 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-19T04:46:38.147 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-19T04:46:38.450 INFO:teuthology.orchestra.run.smithi174.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 63 kB/s | 36 kB 00:00 2024-09-19T04:46:38.513 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T04:46:38.513 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T04:46:38.549 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-19T04:46:38.549 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-19T04:46:38.595 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-19T04:46:38.607 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout:Install 4 Packages 2024-09-19T04:46:38.608 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:46:38.609 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 79 M 2024-09-19T04:46:38.609 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 126 M 2024-09-19T04:46:38.609 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-19T04:46:38.825 INFO:teuthology.orchestra.run.smithi103.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 302 kB/s | 36 kB 00:00 2024-09-19T04:46:39.267 INFO:teuthology.orchestra.run.smithi174.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T04:46:39.554 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-19T04:46:39.554 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-19T04:46:39.909 INFO:teuthology.orchestra.run.smithi174.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-19T04:46:40.725 INFO:teuthology.orchestra.run.smithi174.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-19T04:46:40.726 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:46:40.726 INFO:teuthology.orchestra.run.smithi174.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-19T04:46:41.258 INFO:teuthology.orchestra.run.smithi103.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 6.1 MB/s | 16 MB 00:02 2024-09-19T04:46:41.401 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-19T04:46:41.744 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-19T04:46:41.744 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-19T04:46:42.102 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-19T04:46:42.726 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-19T04:46:42.726 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-19T04:46:42.791 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T04:46:42.888 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:46:43.858 INFO:teuthology.orchestra.run.smithi103.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.7 MB/s | 29 MB 00:05 2024-09-19T04:46:44.183 INFO:teuthology.orchestra.run.smithi103.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 6.3 MB/s | 35 MB 00:05 2024-09-19T04:46:44.184 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:46:44.184 INFO:teuthology.orchestra.run.smithi103.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-19T04:46:44.869 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-19T04:46:45.172 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-19T04:46:45.244 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-19T04:46:45.245 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-19T04:46:45.630 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:46:45.747 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:46:45.875 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T04:46:45.960 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:46:46.216 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-19T04:46:46.216 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-19T04:46:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-19T04:46:48.718 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:46:48.852 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:46:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T04:46:49.536 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:46:50.877 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:46:50.923 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:46:52.299 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:46:52.437 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:46:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:46:53.727 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:46:56.097 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:46:57.298 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:46:57.422 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:46:58.599 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:02.300 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:34.955 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:34.997 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:35.292 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:35.292 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T04:47:35.292 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:47:35.292 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:47:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:47:36.096 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-19T04:47:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T04:47:36.096 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T04:47:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T04:47:36.097 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:47:36.097 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-19T04:47:36.447 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T04:47:36.536 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:36.536 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T04:47:36.536 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T04:47:36.536 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T04:47:36.536 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T04:47:36.537 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T04:47:36.537 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T04:47:36.537 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T04:47:36.559 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:36.595 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:36.595 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T04:47:36.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-09-19T04:47:36.599 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:36.635 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 82.el9 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8274670 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T04:47:36.636 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:47:36.637 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T04:47:36.638 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T04:47:36.638 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-508.el9.x86_64 2024-09-19T04:47:36.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T04:47:36.968 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:36.968 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T04:47:36.968 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:47:36.968 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:47:37.286 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:47:37.853 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-19T04:47:38.223 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T04:47:38.316 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:38.316 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T04:47:38.316 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T04:47:38.316 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T04:47:38.317 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T04:47:38.317 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-19T04:47:38.317 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T04:47:38.317 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T04:47:38.375 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:38.376 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T04:47:38.376 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2-tools 2024-09-19T04:47:38.413 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2-tools 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.06 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 82.el9 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Group : Unspecified 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 8274670 2024-09-19T04:47:38.414 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2024-09-19T04:47:38.415 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T04:47:38.415 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T04:47:38.415 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T04:47:38.415 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T04:47:38.415 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : builder@centos.org 2024-09-19T04:47:38.416 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2024-09-19T04:47:38.416 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T04:47:38.416 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Support tools for GRUB. 2024-09-19T04:47:38.416 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2024-09-19T04:47:38.416 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:47:38.416 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T04:47:38.417 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T04:47:38.417 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T04:47:38.417 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2024-09-19T04:47:38.417 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:47:38.417 INFO:teuthology.orchestra.run.smithi174.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T04:47:38.418 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T04:47:38.418 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 5.14.0-508.el9.x86_64 2024-09-19T04:47:38.418 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T04:47:38.569 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T04:47:38.601 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-09-19T04:47:38.603 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-09-19T04:47:38.618 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.U0iOxNArAQ 2024-09-19T04:47:38.618 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.U0iOxNArAQ 2024-09-19T04:47:38.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.U0iOxNArAQ 2024-09-19T04:47:38.817 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.U0iOxNArAQ is 6KB 2024-09-19T04:47:38.866 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.U0iOxNArAQ 2024-09-19T04:47:38.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T04:47:38.944 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T04:47:38.944 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T04:47:38.944 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T04:47:38.945 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T04:47:38.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T04:47:39.062 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2024-09-19T04:47:39.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-09-19T04:47:40.267 INFO:teuthology.orchestra.run.smithi174.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T04:47:40.301 INFO:teuthology.orchestra.run.smithi174.stderr:done 2024-09-19T04:47:40.303 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-09-19T04:47:40.318 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.SgMp6LtYxm 2024-09-19T04:47:40.319 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SgMp6LtYxm 2024-09-19T04:47:40.390 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.SgMp6LtYxm 2024-09-19T04:47:40.532 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.SgMp6LtYxm is 6KB 2024-09-19T04:47:40.580 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.SgMp6LtYxm 2024-09-19T04:47:40.595 DEBUG:teuthology.orchestra.run.smithi174:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T04:47:40.662 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T04:47:40.662 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T04:47:40.662 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T04:47:40.663 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T04:47:40.664 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T04:47:40.763 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:40.802 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:40.825 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2024-09-19T04:47:41.165 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:41.165 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T04:47:41.165 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T04:47:41.165 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T04:47:42.027 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:47:42.028 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-19T04:47:42.353 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T04:47:42.456 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:42.457 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T04:47:42.457 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T04:47:42.457 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T04:47:42.457 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T04:47:42.458 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T04:47:42.458 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T04:47:42.458 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T04:47:42.516 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T04:47:42.517 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T04:47:42.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-19T04:47:42.556 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T04:47:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:47:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T04:47:42.559 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T04:47:42.559 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 5.14.0-508.el9.x86_64 2024-09-19T04:47:42.559 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T04:47:43.206 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-19T04:47:44.454 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T04:47:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-19T04:47:44.488 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-19T04:47:44.510 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.zUjYHIN48I 2024-09-19T04:47:44.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zUjYHIN48I 2024-09-19T04:47:44.581 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.zUjYHIN48I 2024-09-19T04:47:44.718 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.zUjYHIN48I is 6KB 2024-09-19T04:47:44.768 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.zUjYHIN48I 2024-09-19T04:47:44.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T04:47:44.846 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T04:47:44.846 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T04:47:44.846 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T04:47:44.846 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T04:47:44.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T04:47:44.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-19T04:48:09.134 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T04:48:09.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T04:48:09.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:48:10.829 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-19T04:48:10.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T04:48:10.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:48:14.993 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T04:48:14.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T04:48:14.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:48:43.585 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-09-19T04:48:49.473 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T04:48:52.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T04:48:52.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:48:58.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T04:48:58.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:49:01.533 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T04:49:10.841 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T04:49:10.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-09-19T04:49:13.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T04:49:13.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:49:19.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T04:49:19.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:49:20.360 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-19T04:49:20.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T04:49:20.928 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-508.el9.x86_64"... 2024-09-19T04:49:20.928 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-09-19T04:49:20.943 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-508.el9.x86_64 2024-09-19T04:49:20.944 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T04:49:20.944 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:49:20.944 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T04:49:21.945 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-19T04:49:21.946 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T04:49:22.016 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2024-09-19T04:49:22.044 DEBUG:teuthology.parallel:result is None 2024-09-19T04:49:22.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T04:49:22.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:49:23.341 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-19T04:49:23.741 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T04:49:23.741 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-19T04:49:23.741 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-19T04:49:23.756 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-508.el9.x86_64 2024-09-19T04:49:23.757 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T04:49:23.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:49:23.757 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T04:49:24.758 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T04:49:24.758 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T04:49:24.897 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-09-19T04:49:24.932 DEBUG:teuthology.parallel:result is None 2024-09-19T04:49:45.658 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-19T04:49:45.974 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T04:49:45.974 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-19T04:49:45.974 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-19T04:49:45.989 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-508.el9.x86_64 2024-09-19T04:49:45.989 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T04:49:45.989 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:49:45.989 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T04:49:46.990 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T04:49:46.990 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T04:49:47.064 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-09-19T04:49:47.091 DEBUG:teuthology.parallel:result is None 2024-09-19T04:49:47.091 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T04:49:47.102 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T04:49:47.102 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:49:47.106 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:49:47.108 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:49:47.128 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T04:49:47.162 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T04:49:47.225 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T04:49:47.226 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:49:47.228 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:49:47.231 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:49:47.289 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T04:49:47.353 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T04:49:47.353 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:49:47.356 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:49:47.358 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:49:47.394 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:49:47.395 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:49:47.396 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:49:47.406 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:49:47.406 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:49:47.408 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:49:47.409 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T04:49:47.473 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T04:49:47.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:49:47.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:49:47.480 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:49:47.509 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T04:49:47.556 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T04:49:47.556 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:49:47.559 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:49:47.561 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:49:47.578 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:49:47.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:49:47.710 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:49:47.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:49:47.838 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:49:47.838 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:49:47.905 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:49:47.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:49:48.013 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:49:48.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:49:48.100 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:49:48.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:49:48.168 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:49:48.207 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:49:48.270 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:49:48.345 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:49:48.396 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:49:48.396 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:49:48.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-09-19T04:49:48.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-19T04:49:48.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-09-19T04:49:48.516 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:49:48.520 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:49:48.554 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:49:49.027 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T04:49:49.034 INFO:teuthology.task.internal:Starting timer... 2024-09-19T04:49:49.035 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T04:49:49.137 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T04:49:49.187 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T04:49:49.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-09-19T04:49:49.255 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-09-19T04:49:49.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-19T04:49:49.319 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-19T04:49:49.321 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-09-19T04:49:49.384 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-09-19T04:49:49.385 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T04:49:49.386 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-09-19T04:49:49.409 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-09-19T04:49:49.410 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-19T04:49:49.470 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-19T04:49:49.471 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-09-19T04:49:49.494 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-09-19T04:49:49.495 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-09-19T04:49:49.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:49:49.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:49:49.528 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:49:49.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:49:49.556 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:49:49.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:49:49.584 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T04:49:49.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:49:49.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:49:49.653 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:49:49.685 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T04:49:49.695 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T04:49:49.698 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T04:49:49.811 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T04:49:49.824 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T04:49:49.825 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi103.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T04:53:35.765 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-09-19T04:53:35.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T04:53:35.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:53:35.864 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-19T04:53:35.949 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-19T04:53:35.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T04:53:35.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:53:36.068 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-19T04:53:36.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T04:53:36.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T04:53:36.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:53:36.229 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-19T04:53:36.305 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T04:53:36.306 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T04:53:36.317 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T04:53:36.317 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:53:36.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:53:36.320 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:53:36.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:53:36.323 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:53:36.324 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:53:36.350 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:53:36.351 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:53:36.366 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:53:36.367 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:53:36.376 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:53:36.390 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-19T04:53:36.391 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:53:36.392 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-09-19T04:53:36.401 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-19T04:53:36.403 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-09-19T04:53:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:53:36.417 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-09-19T04:53:36.418 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:53:36.428 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-09-19T04:53:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:53:36.432 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:53:36.443 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:53:36.458 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:53:36.491 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:53:36.492 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:53:36.494 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:53:36.495 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-19T04:53:36.496 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.496 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.496 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.496 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.517 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:53:36.520 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:53:36.521 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-19T04:53:36.521 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.521 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.521 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.521 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:53:36.522 INFO:teuthology.run_tasks:Running task install... 2024-09-19T04:53:36.530 DEBUG:teuthology.task.install:project ceph 2024-09-19T04:53:36.530 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T04:53:36.530 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T04:53:36.530 INFO:teuthology.task.install:Using flavor: default 2024-09-19T04:53:36.536 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T04:53:36.536 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T04:53:36.537 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:53:36.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:36.539 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:53:36.539 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:36.540 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:53:36.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:36.683 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:53:36.684 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:53:36.687 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:53:36.687 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:53:36.697 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:53:36.697 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:53:36.799 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:53:36.799 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:53:36.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:53:36.802 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:53:36.802 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:53:36.802 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:53:36.810 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:53:36.811 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:53:36.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:53:36.827 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:53:36.827 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:53:36.829 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:53:36.830 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:53:36.836 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:53:36.836 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:53:36.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:53:36.896 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:53:36.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:53:36.971 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:53:36.972 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:53:36.977 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:53:37.036 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-09-19T04:53:37.037 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-09-19T04:53:37.038 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-09-19T04:53:37.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-09-19T04:53:37.042 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-19T04:53:37.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-19T04:53:37.263 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-09-19T04:53:37.276 INFO:teuthology.orchestra.run.smithi064.stdout:45 files removed 2024-09-19T04:53:37.285 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-19T04:53:37.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:53:37.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:53:37.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:53:37.863 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 322 kB/s | 84 kB 00:00 2024-09-19T04:53:37.864 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 294 kB/s | 84 kB 00:00 2024-09-19T04:53:37.864 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 325 kB/s | 84 kB 00:00 2024-09-19T04:53:38.139 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-19T04:53:38.147 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T04:53:38.198 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-09-19T04:53:38.415 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T04:53:38.422 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T04:53:38.482 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T04:53:38.924 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-19T04:53:40.083 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.3 MB 00:01 2024-09-19T04:53:40.140 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.3 MB 00:01 2024-09-19T04:53:40.825 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-19T04:53:41.867 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-19T04:53:42.166 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-19T04:53:46.292 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-19T04:53:46.625 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-19T04:53:47.469 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-19T04:53:47.767 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-19T04:53:48.626 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 19 kB 00:00 2024-09-19T04:53:48.863 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-19T04:53:49.120 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-19T04:53:49.813 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T04:53:50.215 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-19T04:53:54.219 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-19T04:53:55.428 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T04:53:55.696 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T04:53:56.379 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:53:56.379 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:53:56.390 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:53:56.391 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:53:56.392 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:53:56.393 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:53:56.394 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:53:56.397 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:53:56.397 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:53:56.459 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:53:56.479 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:53:56.480 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:53:56.481 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:53:56.482 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:53:56.483 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:53:56.484 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:53:56.485 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:53:56.486 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:53:56.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:53:56.488 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:53:56.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:53:56.490 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:53:56.491 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:53:56.492 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 8 Packages 2024-09-19T04:53:56.493 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:53:56.494 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 226 M 2024-09-19T04:53:56.494 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-19T04:53:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-19T04:53:57.637 INFO:teuthology.orchestra.run.smithi174.stdout:(2/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-19T04:53:57.696 INFO:teuthology.orchestra.run.smithi174.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.5 MB/s | 145 kB 00:00 2024-09-19T04:53:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:53:57.734 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:53:57.748 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:53:57.749 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:53:57.750 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:53:57.751 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:53:57.753 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:53:57.756 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:53:57.757 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:53:57.763 INFO:teuthology.orchestra.run.smithi174.stdout:(4/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.6 MB/s | 896 kB 00:00 2024-09-19T04:53:57.825 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-19T04:53:57.849 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-19T04:53:57.849 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-19T04:53:57.849 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-19T04:53:57.850 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-19T04:53:57.850 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:53:57.850 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:53:57.850 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:53:57.850 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:53:57.850 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:53:57.851 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:53:57.852 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:53:57.853 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:53:57.854 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:53:57.855 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:53:57.856 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:53:57.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:53:57.858 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:53:57.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:53:57.860 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:53:57.861 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:53:57.862 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-09-19T04:53:57.863 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:53:57.864 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 226 M 2024-09-19T04:53:57.864 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-19T04:53:57.888 INFO:teuthology.orchestra.run.smithi174.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-19T04:53:57.900 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:53:57.901 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:53:57.915 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:53:57.916 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:53:57.917 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:53:57.918 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:53:57.920 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:53:57.922 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:53:57.923 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:53:57.987 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:53:58.003 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:53:58.004 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:53:58.005 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:53:58.006 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:53:58.007 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:53:58.008 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:53:58.009 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:53:58.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:53:58.011 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:53:58.012 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:53:58.013 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:53:58.014 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:53:58.015 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 8 Packages 2024-09-19T04:53:58.016 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:53:58.017 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 226 M 2024-09-19T04:53:58.017 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-19T04:53:58.180 INFO:teuthology.orchestra.run.smithi174.stdout:(6/156): ceph-common-19.1.1-361.g2f599a46.el9.x 20 MB/s | 21 MB 00:01 2024-09-19T04:53:58.222 INFO:teuthology.orchestra.run.smithi174.stdout:(7/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-09-19T04:53:58.297 INFO:teuthology.orchestra.run.smithi174.stdout:(8/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-19T04:53:58.347 INFO:teuthology.orchestra.run.smithi174.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 507 kB/s | 25 kB 00:00 2024-09-19T04:53:58.781 INFO:teuthology.orchestra.run.smithi103.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 21 kB/s | 6.6 kB 00:00 2024-09-19T04:53:58.798 INFO:teuthology.orchestra.run.smithi064.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 24 kB/s | 6.6 kB 00:00 2024-09-19T04:53:58.872 INFO:teuthology.orchestra.run.smithi174.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 24 MB/s | 17 MB 00:00 2024-09-19T04:53:58.923 INFO:teuthology.orchestra.run.smithi174.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T04:53:59.123 INFO:teuthology.orchestra.run.smithi103.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.6 MB/s | 896 kB 00:00 2024-09-19T04:53:59.148 INFO:teuthology.orchestra.run.smithi064.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.5 MB/s | 896 kB 00:00 2024-09-19T04:53:59.198 INFO:teuthology.orchestra.run.smithi103.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.9 MB/s | 145 kB 00:00 2024-09-19T04:53:59.223 INFO:teuthology.orchestra.run.smithi064.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.9 MB/s | 145 kB 00:00 2024-09-19T04:53:59.248 INFO:teuthology.orchestra.run.smithi174.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 83 kB/s | 27 kB 00:00 2024-09-19T04:53:59.331 INFO:teuthology.orchestra.run.smithi174.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 381 kB/s | 32 kB 00:00 2024-09-19T04:53:59.432 INFO:teuthology.orchestra.run.smithi174.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 7.3 MB/s | 751 kB 00:00 2024-09-19T04:53:59.490 INFO:teuthology.orchestra.run.smithi064.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.6 MB/s | 5.4 MB 00:00 2024-09-19T04:53:59.490 INFO:teuthology.orchestra.run.smithi174.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T04:53:59.491 INFO:teuthology.orchestra.run.smithi103.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-09-19T04:53:59.574 INFO:teuthology.orchestra.run.smithi174.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.5 MB/s | 127 kB 00:00 2024-09-19T04:53:59.674 INFO:teuthology.orchestra.run.smithi174.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.0 MB/s | 506 kB 00:00 2024-09-19T04:53:59.724 INFO:teuthology.orchestra.run.smithi103.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-19T04:53:59.749 INFO:teuthology.orchestra.run.smithi064.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.9 MB/s | 1.5 MB 00:00 2024-09-19T04:53:59.791 INFO:teuthology.orchestra.run.smithi174.stdout:(18/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 6.8 MB/s | 11 MB 00:01 2024-09-19T04:53:59.900 INFO:teuthology.orchestra.run.smithi103.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-09-19T04:54:00.032 INFO:teuthology.orchestra.run.smithi064.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-09-19T04:54:00.050 INFO:teuthology.orchestra.run.smithi174.stdout:(19/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-19T04:54:00.142 INFO:teuthology.orchestra.run.smithi174.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T04:54:00.192 INFO:teuthology.orchestra.run.smithi174.stdout:(21/156): python3-ceph-argparse-19.1.1-361.g2f5 113 kB/s | 45 kB 00:00 2024-09-19T04:54:00.217 INFO:teuthology.orchestra.run.smithi174.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.1 MB/s | 161 kB 00:00 2024-09-19T04:54:00.259 INFO:teuthology.orchestra.run.smithi174.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T04:54:00.266 INFO:teuthology.orchestra.run.smithi064.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 9.1 MB/s | 4.7 MB 00:00 2024-09-19T04:54:00.275 INFO:teuthology.orchestra.run.smithi103.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 8.6 MB/s | 4.7 MB 00:00 2024-09-19T04:54:00.293 INFO:teuthology.orchestra.run.smithi174.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T04:54:00.318 INFO:teuthology.orchestra.run.smithi174.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T04:54:00.360 INFO:teuthology.orchestra.run.smithi174.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-19T04:54:00.443 INFO:teuthology.orchestra.run.smithi174.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T04:54:00.527 INFO:teuthology.orchestra.run.smithi174.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 382 kB/s | 32 kB 00:00 2024-09-19T04:54:00.627 INFO:teuthology.orchestra.run.smithi174.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.4 MB/s | 146 kB 00:00 2024-09-19T04:54:00.744 INFO:teuthology.orchestra.run.smithi174.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 7.3 MB/s | 3.1 MB 00:00 2024-09-19T04:54:00.911 INFO:teuthology.orchestra.run.smithi174.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T04:54:00.995 INFO:teuthology.orchestra.run.smithi174.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.9 MB/s | 249 kB 00:00 2024-09-19T04:54:01.070 INFO:teuthology.orchestra.run.smithi174.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 658 kB/s | 49 kB 00:00 2024-09-19T04:54:01.154 INFO:teuthology.orchestra.run.smithi174.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 197 kB/s | 16 kB 00:00 2024-09-19T04:54:01.237 INFO:teuthology.orchestra.run.smithi174.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.3 MB/s | 285 kB 00:00 2024-09-19T04:54:01.337 INFO:teuthology.orchestra.run.smithi174.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-19T04:54:01.438 INFO:teuthology.orchestra.run.smithi174.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 182 kB/s | 18 kB 00:00 2024-09-19T04:54:01.591 INFO:teuthology.orchestra.run.smithi064.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 8.0 MB/s | 11 MB 00:01 2024-09-19T04:54:01.638 INFO:teuthology.orchestra.run.smithi174.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.3 MB/s | 7.4 MB 00:00 2024-09-19T04:54:01.641 INFO:teuthology.orchestra.run.smithi064.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 508 kB/s | 25 kB 00:00 2024-09-19T04:54:01.655 INFO:teuthology.orchestra.run.smithi174.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-19T04:54:01.733 INFO:teuthology.orchestra.run.smithi103.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 7.3 MB/s | 11 MB 00:01 2024-09-19T04:54:01.780 INFO:teuthology.orchestra.run.smithi174.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 5.5 MB/s | 801 kB 00:00 2024-09-19T04:54:01.792 INFO:teuthology.orchestra.run.smithi103.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 433 kB/s | 25 kB 00:00 2024-09-19T04:54:01.822 INFO:teuthology.orchestra.run.smithi174.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-19T04:54:01.847 INFO:teuthology.orchestra.run.smithi174.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-19T04:54:01.898 INFO:teuthology.orchestra.run.smithi174.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T04:54:01.973 INFO:teuthology.orchestra.run.smithi174.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-09-19T04:54:01.998 INFO:teuthology.orchestra.run.smithi174.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-19T04:54:02.023 INFO:teuthology.orchestra.run.smithi174.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T04:54:02.057 INFO:teuthology.orchestra.run.smithi174.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T04:54:02.176 INFO:teuthology.orchestra.run.smithi103.stdout:(10/156): ceph-common-19.1.1-361.g2f599a46.el9. 5.7 MB/s | 21 MB 00:03 2024-09-19T04:54:02.226 INFO:teuthology.orchestra.run.smithi103.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T04:54:02.276 INFO:teuthology.orchestra.run.smithi103.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 542 kB/s | 27 kB 00:00 2024-09-19T04:54:02.326 INFO:teuthology.orchestra.run.smithi103.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 636 kB/s | 32 kB 00:00 2024-09-19T04:54:02.477 INFO:teuthology.orchestra.run.smithi103.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.9 MB/s | 751 kB 00:00 2024-09-19T04:54:02.491 INFO:teuthology.orchestra.run.smithi174.stdout:(48/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 12 MB/s | 49 MB 00:04 2024-09-19T04:54:02.527 INFO:teuthology.orchestra.run.smithi103.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T04:54:02.586 INFO:teuthology.orchestra.run.smithi103.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:54:02.591 INFO:teuthology.orchestra.run.smithi174.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-09-19T04:54:02.616 INFO:teuthology.orchestra.run.smithi174.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-09-19T04:54:02.641 INFO:teuthology.orchestra.run.smithi174.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-19T04:54:02.675 INFO:teuthology.orchestra.run.smithi174.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-09-19T04:54:02.677 INFO:teuthology.orchestra.run.smithi103.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T04:54:02.700 INFO:teuthology.orchestra.run.smithi174.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-09-19T04:54:02.734 INFO:teuthology.orchestra.run.smithi174.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T04:54:02.759 INFO:teuthology.orchestra.run.smithi174.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-19T04:54:02.792 INFO:teuthology.orchestra.run.smithi174.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-19T04:54:02.818 INFO:teuthology.orchestra.run.smithi174.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T04:54:02.835 INFO:teuthology.orchestra.run.smithi174.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T04:54:02.877 INFO:teuthology.orchestra.run.smithi174.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-19T04:54:02.947 INFO:teuthology.orchestra.run.smithi174.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-19T04:54:02.969 INFO:teuthology.orchestra.run.smithi174.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 267 kB/s | 24 kB 00:00 2024-09-19T04:54:03.003 INFO:teuthology.orchestra.run.smithi174.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-19T04:54:03.029 INFO:teuthology.orchestra.run.smithi174.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 588 kB/s | 47 kB 00:00 2024-09-19T04:54:03.053 INFO:teuthology.orchestra.run.smithi174.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T04:54:03.078 INFO:teuthology.orchestra.run.smithi174.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-19T04:54:03.103 INFO:teuthology.orchestra.run.smithi174.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-19T04:54:03.221 INFO:teuthology.orchestra.run.smithi174.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-09-19T04:54:03.254 INFO:teuthology.orchestra.run.smithi174.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T04:54:03.269 INFO:teuthology.orchestra.run.smithi103.stdout:(18/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 5.0 MB/s | 17 MB 00:03 2024-09-19T04:54:03.288 INFO:teuthology.orchestra.run.smithi174.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-19T04:54:03.304 INFO:teuthology.orchestra.run.smithi174.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-19T04:54:03.361 INFO:teuthology.orchestra.run.smithi103.stdout:(19/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 7.8 MB/s | 5.3 MB 00:00 2024-09-19T04:54:03.380 INFO:teuthology.orchestra.run.smithi174.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-19T04:54:03.387 INFO:teuthology.orchestra.run.smithi103.stdout:(20/156): python3-ceph-argparse-19.1.1-361.g2f5 386 kB/s | 45 kB 00:00 2024-09-19T04:54:03.412 INFO:teuthology.orchestra.run.smithi103.stdout:(21/156): python3-ceph-common-19.1.1-361.g2f599 2.6 MB/s | 136 kB 00:00 2024-09-19T04:54:03.422 INFO:teuthology.orchestra.run.smithi174.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-19T04:54:03.446 INFO:teuthology.orchestra.run.smithi103.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.7 MB/s | 161 kB 00:00 2024-09-19T04:54:03.447 INFO:teuthology.orchestra.run.smithi174.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T04:54:03.471 INFO:teuthology.orchestra.run.smithi103.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 5.3 MB/s | 320 kB 00:00 2024-09-19T04:54:03.522 INFO:teuthology.orchestra.run.smithi103.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T04:54:03.530 INFO:teuthology.orchestra.run.smithi174.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-19T04:54:03.547 INFO:teuthology.orchestra.run.smithi103.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.3 MB/s | 99 kB 00:00 2024-09-19T04:54:03.556 INFO:teuthology.orchestra.run.smithi174.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 563 kB/s | 61 kB 00:00 2024-09-19T04:54:03.572 INFO:teuthology.orchestra.run.smithi103.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:54:03.581 INFO:teuthology.orchestra.run.smithi174.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-19T04:54:03.606 INFO:teuthology.orchestra.run.smithi174.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T04:54:03.616 INFO:teuthology.orchestra.run.smithi064.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 4.7 MB/s | 17 MB 00:03 2024-09-19T04:54:03.631 INFO:teuthology.orchestra.run.smithi103.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T04:54:03.631 INFO:teuthology.orchestra.run.smithi174.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-19T04:54:03.657 INFO:teuthology.orchestra.run.smithi174.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-19T04:54:03.673 INFO:teuthology.orchestra.run.smithi103.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 765 kB/s | 32 kB 00:00 2024-09-19T04:54:03.682 INFO:teuthology.orchestra.run.smithi174.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T04:54:03.707 INFO:teuthology.orchestra.run.smithi174.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-19T04:54:03.723 INFO:teuthology.orchestra.run.smithi103.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T04:54:03.800 INFO:teuthology.orchestra.run.smithi174.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-19T04:54:03.808 INFO:teuthology.orchestra.run.smithi064.stdout:(11/156): ceph-common-19.1.1-361.g2f599a46.el9. 4.0 MB/s | 21 MB 00:05 2024-09-19T04:54:03.824 INFO:teuthology.orchestra.run.smithi174.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-19T04:54:03.833 INFO:teuthology.orchestra.run.smithi064.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.0 MB/s | 228 kB 00:00 2024-09-19T04:54:03.848 INFO:teuthology.orchestra.run.smithi103.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-19T04:54:03.849 INFO:teuthology.orchestra.run.smithi174.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T04:54:03.850 INFO:teuthology.orchestra.run.smithi064.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 646 kB/s | 27 kB 00:00 2024-09-19T04:54:03.875 INFO:teuthology.orchestra.run.smithi174.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:54:03.876 INFO:teuthology.orchestra.run.smithi064.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 760 kB/s | 32 kB 00:00 2024-09-19T04:54:03.892 INFO:teuthology.orchestra.run.smithi174.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-19T04:54:03.917 INFO:teuthology.orchestra.run.smithi174.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-19T04:54:03.934 INFO:teuthology.orchestra.run.smithi064.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T04:54:03.942 INFO:teuthology.orchestra.run.smithi174.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T04:54:03.968 INFO:teuthology.orchestra.run.smithi174.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T04:54:03.993 INFO:teuthology.orchestra.run.smithi174.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T04:54:03.993 INFO:teuthology.orchestra.run.smithi064.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T04:54:04.018 INFO:teuthology.orchestra.run.smithi174.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:54:04.027 INFO:teuthology.orchestra.run.smithi064.stdout:(17/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.2 MB/s | 751 kB 00:00 2024-09-19T04:54:04.085 INFO:teuthology.orchestra.run.smithi174.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-19T04:54:04.110 INFO:teuthology.orchestra.run.smithi174.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-19T04:54:04.127 INFO:teuthology.orchestra.run.smithi064.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 3.7 MB/s | 506 kB 00:00 2024-09-19T04:54:04.136 INFO:teuthology.orchestra.run.smithi174.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-09-19T04:54:04.161 INFO:teuthology.orchestra.run.smithi174.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T04:54:04.185 INFO:teuthology.orchestra.run.smithi064.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 774 kB/s | 45 kB 00:00 2024-09-19T04:54:04.186 INFO:teuthology.orchestra.run.smithi174.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-19T04:54:04.211 INFO:teuthology.orchestra.run.smithi174.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-19T04:54:04.237 INFO:teuthology.orchestra.run.smithi174.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T04:54:04.244 INFO:teuthology.orchestra.run.smithi064.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.3 MB/s | 136 kB 00:00 2024-09-19T04:54:04.257 INFO:teuthology.orchestra.run.smithi103.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.7 MB/s | 3.6 MB 00:00 2024-09-19T04:54:04.262 INFO:teuthology.orchestra.run.smithi174.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T04:54:04.287 INFO:teuthology.orchestra.run.smithi174.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T04:54:04.302 INFO:teuthology.orchestra.run.smithi064.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.7 MB/s | 161 kB 00:00 2024-09-19T04:54:04.313 INFO:teuthology.orchestra.run.smithi174.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:54:04.316 INFO:teuthology.orchestra.run.smithi103.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.2 MB/s | 249 kB 00:00 2024-09-19T04:54:04.338 INFO:teuthology.orchestra.run.smithi174.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-19T04:54:04.375 INFO:teuthology.orchestra.run.smithi103.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 848 kB/s | 49 kB 00:00 2024-09-19T04:54:04.379 INFO:teuthology.orchestra.run.smithi064.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T04:54:04.426 INFO:teuthology.orchestra.run.smithi103.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 330 kB/s | 16 kB 00:00 2024-09-19T04:54:04.438 INFO:teuthology.orchestra.run.smithi174.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-19T04:54:04.470 INFO:teuthology.orchestra.run.smithi064.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.2 MB/s | 300 kB 00:00 2024-09-19T04:54:04.488 INFO:teuthology.orchestra.run.smithi174.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T04:54:04.529 INFO:teuthology.orchestra.run.smithi064.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T04:54:04.529 INFO:teuthology.orchestra.run.smithi103.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.0 MB/s | 285 kB 00:00 2024-09-19T04:54:04.579 INFO:teuthology.orchestra.run.smithi064.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T04:54:04.609 INFO:teuthology.orchestra.run.smithi103.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 9.7 MB/s | 7.4 MB 00:00 2024-09-19T04:54:04.647 INFO:teuthology.orchestra.run.smithi174.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-19T04:54:04.676 INFO:teuthology.orchestra.run.smithi103.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 274 kB/s | 18 kB 00:00 2024-09-19T04:54:04.718 INFO:teuthology.orchestra.run.smithi103.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 3.7 MB/s | 765 kB 00:00 2024-09-19T04:54:04.764 INFO:teuthology.orchestra.run.smithi174.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.1 MB/s | 308 kB 00:00 2024-09-19T04:54:04.797 INFO:teuthology.orchestra.run.smithi174.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-19T04:54:04.829 INFO:teuthology.orchestra.run.smithi064.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 6.7 MB/s | 5.3 MB 00:00 2024-09-19T04:54:04.831 INFO:teuthology.orchestra.run.smithi174.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 109 kB/s | 19 kB 00:00 2024-09-19T04:54:04.843 INFO:teuthology.orchestra.run.smithi103.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-19T04:54:04.856 INFO:teuthology.orchestra.run.smithi174.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 458 kB/s | 26 kB 00:00 2024-09-19T04:54:04.874 INFO:teuthology.orchestra.run.smithi174.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-09-19T04:54:04.889 INFO:teuthology.orchestra.run.smithi064.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T04:54:04.898 INFO:teuthology.orchestra.run.smithi174.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-19T04:54:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-19T04:54:04.923 INFO:teuthology.orchestra.run.smithi174.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-19T04:54:04.938 INFO:teuthology.orchestra.run.smithi064.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 637 kB/s | 32 kB 00:00 2024-09-19T04:54:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-19T04:54:04.982 INFO:teuthology.orchestra.run.smithi174.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-19T04:54:04.986 INFO:teuthology.orchestra.run.smithi103.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-19T04:54:04.988 INFO:teuthology.orchestra.run.smithi064.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T04:54:04.999 INFO:teuthology.orchestra.run.smithi174.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T04:54:05.016 INFO:teuthology.orchestra.run.smithi174.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T04:54:05.036 INFO:teuthology.orchestra.run.smithi103.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-19T04:54:05.039 INFO:teuthology.orchestra.run.smithi064.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.7 MB/s | 3.1 MB 00:00 2024-09-19T04:54:05.061 INFO:teuthology.orchestra.run.smithi103.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T04:54:05.083 INFO:teuthology.orchestra.run.smithi174.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T04:54:05.087 INFO:teuthology.orchestra.run.smithi103.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T04:54:05.108 INFO:teuthology.orchestra.run.smithi174.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 154 kB/s | 14 kB 00:00 2024-09-19T04:54:05.112 INFO:teuthology.orchestra.run.smithi103.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T04:54:05.137 INFO:teuthology.orchestra.run.smithi103.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-19T04:54:05.142 INFO:teuthology.orchestra.run.smithi174.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-09-19T04:54:05.167 INFO:teuthology.orchestra.run.smithi174.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-19T04:54:05.192 INFO:teuthology.orchestra.run.smithi174.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-19T04:54:05.217 INFO:teuthology.orchestra.run.smithi174.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T04:54:05.251 INFO:teuthology.orchestra.run.smithi174.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-19T04:54:05.276 INFO:teuthology.orchestra.run.smithi174.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-19T04:54:05.301 INFO:teuthology.orchestra.run.smithi174.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-19T04:54:05.327 INFO:teuthology.orchestra.run.smithi174.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-19T04:54:05.352 INFO:teuthology.orchestra.run.smithi174.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-19T04:54:05.377 INFO:teuthology.orchestra.run.smithi174.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-19T04:54:05.403 INFO:teuthology.orchestra.run.smithi174.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 388 kB/s | 20 kB 00:00 2024-09-19T04:54:05.436 INFO:teuthology.orchestra.run.smithi174.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T04:54:05.462 INFO:teuthology.orchestra.run.smithi174.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:54:05.487 INFO:teuthology.orchestra.run.smithi174.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T04:54:05.520 INFO:teuthology.orchestra.run.smithi174.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-19T04:54:05.546 INFO:teuthology.orchestra.run.smithi174.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-19T04:54:05.548 INFO:teuthology.orchestra.run.smithi064.stdout:(31/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 12 MB/s | 49 MB 00:03 2024-09-19T04:54:05.571 INFO:teuthology.orchestra.run.smithi174.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-19T04:54:05.621 INFO:teuthology.orchestra.run.smithi174.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-19T04:54:05.638 INFO:teuthology.orchestra.run.smithi103.stdout:(48/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 13 MB/s | 49 MB 00:03 2024-09-19T04:54:05.646 INFO:teuthology.orchestra.run.smithi174.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-09-19T04:54:05.672 INFO:teuthology.orchestra.run.smithi174.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-09-19T04:54:05.697 INFO:teuthology.orchestra.run.smithi174.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T04:54:05.722 INFO:teuthology.orchestra.run.smithi174.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T04:54:05.747 INFO:teuthology.orchestra.run.smithi174.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T04:54:05.773 INFO:teuthology.orchestra.run.smithi174.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-19T04:54:05.780 INFO:teuthology.orchestra.run.smithi103.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-19T04:54:05.798 INFO:teuthology.orchestra.run.smithi174.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-19T04:54:05.822 INFO:teuthology.orchestra.run.smithi103.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-19T04:54:05.823 INFO:teuthology.orchestra.run.smithi174.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:54:05.847 INFO:teuthology.orchestra.run.smithi103.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 512 kB/s | 107 kB 00:00 2024-09-19T04:54:05.848 INFO:teuthology.orchestra.run.smithi174.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 795 kB/s | 20 kB 00:00 2024-09-19T04:54:05.856 INFO:teuthology.orchestra.run.smithi064.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 810 kB/s | 249 kB 00:00 2024-09-19T04:54:05.872 INFO:teuthology.orchestra.run.smithi103.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 235 kB/s | 22 kB 00:00 2024-09-19T04:54:05.874 INFO:teuthology.orchestra.run.smithi174.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T04:54:05.899 INFO:teuthology.orchestra.run.smithi174.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:54:05.906 INFO:teuthology.orchestra.run.smithi103.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-19T04:54:05.923 INFO:teuthology.orchestra.run.smithi103.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-19T04:54:05.924 INFO:teuthology.orchestra.run.smithi174.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T04:54:05.940 INFO:teuthology.orchestra.run.smithi064.stdout:(33/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 3.7 MB/s | 3.6 MB 00:00 2024-09-19T04:54:05.948 INFO:teuthology.orchestra.run.smithi103.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 433 kB/s | 18 kB 00:00 2024-09-19T04:54:05.965 INFO:teuthology.orchestra.run.smithi064.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 456 kB/s | 49 kB 00:00 2024-09-19T04:54:05.973 INFO:teuthology.orchestra.run.smithi103.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T04:54:05.974 INFO:teuthology.orchestra.run.smithi174.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T04:54:05.990 INFO:teuthology.orchestra.run.smithi064.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 328 kB/s | 16 kB 00:00 2024-09-19T04:54:06.007 INFO:teuthology.orchestra.run.smithi103.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T04:54:06.016 INFO:teuthology.orchestra.run.smithi064.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T04:54:06.032 INFO:teuthology.orchestra.run.smithi103.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-09-19T04:54:06.099 INFO:teuthology.orchestra.run.smithi064.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 219 kB/s | 18 kB 00:00 2024-09-19T04:54:06.191 INFO:teuthology.orchestra.run.smithi064.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 3.7 MB/s | 765 kB 00:00 2024-09-19T04:54:06.233 INFO:teuthology.orchestra.run.smithi103.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-19T04:54:06.258 INFO:teuthology.orchestra.run.smithi103.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-09-19T04:54:06.267 INFO:teuthology.orchestra.run.smithi064.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-19T04:54:06.275 INFO:teuthology.orchestra.run.smithi174.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 9.1 MB/s | 3.4 MB 00:00 2024-09-19T04:54:06.317 INFO:teuthology.orchestra.run.smithi103.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-09-19T04:54:06.333 INFO:teuthology.orchestra.run.smithi174.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.7 MB/s | 3.2 MB 00:00 2024-09-19T04:54:06.342 INFO:teuthology.orchestra.run.smithi103.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-19T04:54:06.358 INFO:teuthology.orchestra.run.smithi064.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-19T04:54:06.367 INFO:teuthology.orchestra.run.smithi103.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-09-19T04:54:06.375 INFO:teuthology.orchestra.run.smithi174.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 5.8 MB/s | 2.3 MB 00:00 2024-09-19T04:54:06.401 INFO:teuthology.orchestra.run.smithi103.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-19T04:54:06.417 INFO:teuthology.orchestra.run.smithi064.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-09-19T04:54:06.428 INFO:teuthology.orchestra.run.smithi174.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-19T04:54:06.429 INFO:teuthology.orchestra.run.smithi103.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 656 kB/s | 38 kB 00:00 2024-09-19T04:54:06.452 INFO:teuthology.orchestra.run.smithi103.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-19T04:54:06.484 INFO:teuthology.orchestra.run.smithi174.stdout:(153/156): NetworkManager-team-1.51.0-1.el9.x86 272 kB/s | 41 kB 00:00 2024-09-19T04:54:06.501 INFO:teuthology.orchestra.run.smithi064.stdout:(42/156): ceph-mgr-diskprediction-local-19.1.1- 5.1 MB/s | 7.4 MB 00:01 2024-09-19T04:54:06.526 INFO:teuthology.orchestra.run.smithi064.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-19T04:54:06.567 INFO:teuthology.orchestra.run.smithi174.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-19T04:54:06.584 INFO:teuthology.orchestra.run.smithi064.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-19T04:54:06.594 INFO:teuthology.orchestra.run.smithi103.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T04:54:06.609 INFO:teuthology.orchestra.run.smithi174.stdout:(155/156): NetworkManager-libnm-1.51.0-1.el9.x8 5.4 MB/s | 1.8 MB 00:00 2024-09-19T04:54:06.618 INFO:teuthology.orchestra.run.smithi064.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-09-19T04:54:06.643 INFO:teuthology.orchestra.run.smithi064.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T04:54:06.694 INFO:teuthology.orchestra.run.smithi103.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-19T04:54:06.742 INFO:teuthology.orchestra.run.smithi064.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-19T04:54:06.769 INFO:teuthology.orchestra.run.smithi064.stdout:(48/156): python3-ply-3.11-14.el9.noarch.rpm 397 kB/s | 106 kB 00:00 2024-09-19T04:54:06.769 INFO:teuthology.orchestra.run.smithi103.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-19T04:54:06.818 INFO:teuthology.orchestra.run.smithi174.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-09-19T04:54:06.823 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:54:06.823 INFO:teuthology.orchestra.run.smithi174.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-19T04:54:06.845 INFO:teuthology.orchestra.run.smithi103.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 11 MB/s | 4.9 MB 00:00 2024-09-19T04:54:06.878 INFO:teuthology.orchestra.run.smithi064.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-09-19T04:54:06.912 INFO:teuthology.orchestra.run.smithi064.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-19T04:54:06.912 INFO:teuthology.orchestra.run.smithi103.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-19T04:54:06.936 INFO:teuthology.orchestra.run.smithi064.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-19T04:54:06.970 INFO:teuthology.orchestra.run.smithi103.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T04:54:06.978 INFO:teuthology.orchestra.run.smithi064.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 786 kB/s | 33 kB 00:00 2024-09-19T04:54:07.004 INFO:teuthology.orchestra.run.smithi103.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-09-19T04:54:07.012 INFO:teuthology.orchestra.run.smithi064.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-19T04:54:07.040 INFO:teuthology.orchestra.run.smithi064.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-19T04:54:07.062 INFO:teuthology.orchestra.run.smithi103.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-19T04:54:07.063 INFO:teuthology.orchestra.run.smithi064.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-19T04:54:07.088 INFO:teuthology.orchestra.run.smithi064.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T04:54:07.121 INFO:teuthology.orchestra.run.smithi103.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-19T04:54:07.130 INFO:teuthology.orchestra.run.smithi064.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T04:54:07.155 INFO:teuthology.orchestra.run.smithi064.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-19T04:54:07.206 INFO:teuthology.orchestra.run.smithi064.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-19T04:54:07.222 INFO:teuthology.orchestra.run.smithi103.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-09-19T04:54:07.247 INFO:teuthology.orchestra.run.smithi103.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-09-19T04:54:07.247 INFO:teuthology.orchestra.run.smithi064.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 593 kB/s | 24 kB 00:00 2024-09-19T04:54:07.272 INFO:teuthology.orchestra.run.smithi103.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-19T04:54:07.297 INFO:teuthology.orchestra.run.smithi103.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-19T04:54:07.298 INFO:teuthology.orchestra.run.smithi064.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-19T04:54:07.323 INFO:teuthology.orchestra.run.smithi064.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.3 MB/s | 662 kB 00:00 2024-09-19T04:54:07.331 INFO:teuthology.orchestra.run.smithi103.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T04:54:07.356 INFO:teuthology.orchestra.run.smithi103.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-19T04:54:07.390 INFO:teuthology.orchestra.run.smithi064.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-09-19T04:54:07.415 INFO:teuthology.orchestra.run.smithi064.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-19T04:54:07.432 INFO:teuthology.orchestra.run.smithi103.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-19T04:54:07.441 INFO:teuthology.orchestra.run.smithi064.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-19T04:54:07.457 INFO:teuthology.orchestra.run.smithi103.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T04:54:07.491 INFO:teuthology.orchestra.run.smithi103.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T04:54:07.516 INFO:teuthology.orchestra.run.smithi103.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T04:54:07.542 INFO:teuthology.orchestra.run.smithi103.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-19T04:54:07.575 INFO:teuthology.orchestra.run.smithi103.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T04:54:07.609 INFO:teuthology.orchestra.run.smithi103.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T04:54:07.634 INFO:teuthology.orchestra.run.smithi103.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-19T04:54:07.658 INFO:teuthology.orchestra.run.smithi064.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.4 MB/s | 3.0 MB 00:00 2024-09-19T04:54:07.701 INFO:teuthology.orchestra.run.smithi103.stdout:(90/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-19T04:54:07.726 INFO:teuthology.orchestra.run.smithi103.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-19T04:54:07.752 INFO:teuthology.orchestra.run.smithi103.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-09-19T04:54:07.810 INFO:teuthology.orchestra.run.smithi103.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 761 kB/s | 44 kB 00:00 2024-09-19T04:54:07.836 INFO:teuthology.orchestra.run.smithi103.stdout:(94/156): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-09-19T04:54:07.869 INFO:teuthology.orchestra.run.smithi103.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T04:54:07.894 INFO:teuthology.orchestra.run.smithi103.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-09-19T04:54:07.920 INFO:teuthology.orchestra.run.smithi103.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-19T04:54:07.970 INFO:teuthology.orchestra.run.smithi103.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.9 MB/s | 304 kB 00:00 2024-09-19T04:54:07.983 INFO:teuthology.orchestra.run.smithi064.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-19T04:54:07.996 INFO:teuthology.orchestra.run.smithi103.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 766 kB/s | 58 kB 00:00 2024-09-19T04:54:08.004 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-19T04:54:08.038 INFO:teuthology.orchestra.run.smithi103.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-19T04:54:08.058 INFO:teuthology.orchestra.run.smithi064.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-19T04:54:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-09-19T04:54:08.105 INFO:teuthology.orchestra.run.smithi103.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-09-19T04:54:08.125 INFO:teuthology.orchestra.run.smithi064.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 7.0 MB/s | 4.9 MB 00:00 2024-09-19T04:54:08.150 INFO:teuthology.orchestra.run.smithi064.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-19T04:54:08.154 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-19T04:54:08.154 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-19T04:54:08.172 INFO:teuthology.orchestra.run.smithi103.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-19T04:54:08.222 INFO:teuthology.orchestra.run.smithi103.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T04:54:08.234 INFO:teuthology.orchestra.run.smithi064.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-19T04:54:08.256 INFO:teuthology.orchestra.run.smithi103.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T04:54:08.281 INFO:teuthology.orchestra.run.smithi103.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-19T04:54:08.284 INFO:teuthology.orchestra.run.smithi064.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-19T04:54:08.448 INFO:teuthology.orchestra.run.smithi103.stdout:(107/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-19T04:54:08.451 INFO:teuthology.orchestra.run.smithi064.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-09-19T04:54:08.499 INFO:teuthology.orchestra.run.smithi103.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-09-19T04:54:08.526 INFO:teuthology.orchestra.run.smithi064.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-09-19T04:54:08.532 INFO:teuthology.orchestra.run.smithi103.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T04:54:08.543 INFO:teuthology.orchestra.run.smithi064.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 668 kB/s | 61 kB 00:00 2024-09-19T04:54:08.569 INFO:teuthology.orchestra.run.smithi064.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 5.7 MB/s | 245 kB 00:00 2024-09-19T04:54:08.574 INFO:teuthology.orchestra.run.smithi103.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-19T04:54:08.594 INFO:teuthology.orchestra.run.smithi064.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T04:54:08.608 INFO:teuthology.orchestra.run.smithi103.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-19T04:54:08.611 INFO:teuthology.orchestra.run.smithi064.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-09-19T04:54:08.628 INFO:teuthology.orchestra.run.smithi064.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 567 kB/s | 19 kB 00:00 2024-09-19T04:54:08.642 INFO:teuthology.orchestra.run.smithi103.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-19T04:54:08.686 INFO:teuthology.orchestra.run.smithi064.stdout:(80/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-19T04:54:08.712 INFO:teuthology.orchestra.run.smithi064.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-19T04:54:08.725 INFO:teuthology.orchestra.run.smithi103.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 4.4 MB 00:00 2024-09-19T04:54:08.737 INFO:teuthology.orchestra.run.smithi064.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-19T04:54:08.751 INFO:teuthology.orchestra.run.smithi103.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T04:54:08.779 INFO:teuthology.orchestra.run.smithi064.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-09-19T04:54:08.784 INFO:teuthology.orchestra.run.smithi103.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-19T04:54:08.809 INFO:teuthology.orchestra.run.smithi103.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-09-19T04:54:08.835 INFO:teuthology.orchestra.run.smithi103.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-19T04:54:08.854 INFO:teuthology.orchestra.run.smithi064.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-09-19T04:54:08.860 INFO:teuthology.orchestra.run.smithi103.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-09-19T04:54:08.879 INFO:teuthology.orchestra.run.smithi064.stdout:(85/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-09-19T04:54:08.885 INFO:teuthology.orchestra.run.smithi103.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-19T04:54:08.905 INFO:teuthology.orchestra.run.smithi064.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 614 kB/s | 77 kB 00:00 2024-09-19T04:54:08.911 INFO:teuthology.orchestra.run.smithi103.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-19T04:54:08.939 INFO:teuthology.orchestra.run.smithi064.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 503 kB/s | 42 kB 00:00 2024-09-19T04:54:08.944 INFO:teuthology.orchestra.run.smithi103.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-19T04:54:08.964 INFO:teuthology.orchestra.run.smithi064.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-19T04:54:08.986 INFO:teuthology.orchestra.run.smithi103.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.5 MB/s | 144 kB 00:00 2024-09-19T04:54:08.989 INFO:teuthology.orchestra.run.smithi064.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-19T04:54:09.012 INFO:teuthology.orchestra.run.smithi103.stdout:(123/156): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-19T04:54:09.014 INFO:teuthology.orchestra.run.smithi064.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T04:54:09.028 INFO:teuthology.orchestra.run.smithi103.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-09-19T04:54:09.040 INFO:teuthology.orchestra.run.smithi064.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:54:09.045 INFO:teuthology.orchestra.run.smithi103.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 528 kB/s | 18 kB 00:00 2024-09-19T04:54:09.062 INFO:teuthology.orchestra.run.smithi103.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2024-09-19T04:54:09.065 INFO:teuthology.orchestra.run.smithi064.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-19T04:54:09.080 INFO:teuthology.orchestra.run.smithi103.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 575 kB/s | 19 kB 00:00 2024-09-19T04:54:09.090 INFO:teuthology.orchestra.run.smithi064.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-19T04:54:09.097 INFO:teuthology.orchestra.run.smithi103.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 583 kB/s | 20 kB 00:00 2024-09-19T04:54:09.115 INFO:teuthology.orchestra.run.smithi064.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-19T04:54:09.138 INFO:teuthology.orchestra.run.smithi103.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-19T04:54:09.158 INFO:teuthology.orchestra.run.smithi064.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 859 kB/s | 36 kB 00:00 2024-09-19T04:54:09.164 INFO:teuthology.orchestra.run.smithi103.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:54:09.183 INFO:teuthology.orchestra.run.smithi064.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-19T04:54:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-19T04:54:09.214 INFO:teuthology.orchestra.run.smithi103.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-19T04:54:09.239 INFO:teuthology.orchestra.run.smithi103.stdout:(132/156): python3-logutils-0.3.5-21.el9.noarch 325 kB/s | 46 kB 00:00 2024-09-19T04:54:09.250 INFO:teuthology.orchestra.run.smithi064.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T04:54:09.265 INFO:teuthology.orchestra.run.smithi103.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 581 kB/s | 58 kB 00:00 2024-09-19T04:54:09.290 INFO:teuthology.orchestra.run.smithi103.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-19T04:54:09.300 INFO:teuthology.orchestra.run.smithi064.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-19T04:54:09.315 INFO:teuthology.orchestra.run.smithi103.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T04:54:09.325 INFO:teuthology.orchestra.run.smithi064.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-19T04:54:09.341 INFO:teuthology.orchestra.run.smithi103.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-19T04:54:09.351 INFO:teuthology.orchestra.run.smithi064.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:54:09.366 INFO:teuthology.orchestra.run.smithi103.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 607 kB/s | 31 kB 00:00 2024-09-19T04:54:09.376 INFO:teuthology.orchestra.run.smithi064.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-09-19T04:54:09.391 INFO:teuthology.orchestra.run.smithi103.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-19T04:54:09.417 INFO:teuthology.orchestra.run.smithi103.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T04:54:09.442 INFO:teuthology.orchestra.run.smithi103.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-19T04:54:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-19T04:54:09.493 INFO:teuthology.orchestra.run.smithi103.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-19T04:54:09.521 INFO:teuthology.orchestra.run.smithi103.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:54:09.543 INFO:teuthology.orchestra.run.smithi103.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T04:54:09.552 INFO:teuthology.orchestra.run.smithi064.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-19T04:54:09.569 INFO:teuthology.orchestra.run.smithi103.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-19T04:54:09.594 INFO:teuthology.orchestra.run.smithi103.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:54:09.610 INFO:teuthology.orchestra.run.smithi064.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-09-19T04:54:09.628 INFO:teuthology.orchestra.run.smithi103.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-19T04:54:09.635 INFO:teuthology.orchestra.run.smithi064.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-19T04:54:09.694 INFO:teuthology.orchestra.run.smithi064.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.6 MB/s | 308 kB 00:00 2024-09-19T04:54:09.703 INFO:teuthology.orchestra.run.smithi103.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T04:54:09.736 INFO:teuthology.orchestra.run.smithi064.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-19T04:54:09.795 INFO:teuthology.orchestra.run.smithi064.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 452 kB/s | 26 kB 00:00 2024-09-19T04:54:09.820 INFO:teuthology.orchestra.run.smithi064.stdout:(109/156): grpc-data-1.46.7-10.el9.noarch.rpm 110 kB/s | 19 kB 00:00 2024-09-19T04:54:09.845 INFO:teuthology.orchestra.run.smithi064.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-19T04:54:09.871 INFO:teuthology.orchestra.run.smithi064.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T04:54:09.886 INFO:teuthology.orchestra.run.smithi103.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-19T04:54:09.896 INFO:teuthology.orchestra.run.smithi064.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-19T04:54:09.955 INFO:teuthology.orchestra.run.smithi064.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-19T04:54:09.962 INFO:teuthology.orchestra.run.smithi103.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-19T04:54:09.988 INFO:teuthology.orchestra.run.smithi064.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-19T04:54:10.009 INFO:teuthology.orchestra.run.smithi103.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-09-19T04:54:10.013 INFO:teuthology.orchestra.run.smithi064.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-09-19T04:54:10.034 INFO:teuthology.orchestra.run.smithi103.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 565 kB/s | 41 kB 00:00 2024-09-19T04:54:10.047 INFO:teuthology.orchestra.run.smithi064.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 547 kB/s | 32 kB 00:00 2024-09-19T04:54:10.060 INFO:teuthology.orchestra.run.smithi103.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.8 MB/s | 249 kB 00:00 2024-09-19T04:54:10.072 INFO:teuthology.orchestra.run.smithi064.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-19T04:54:10.101 INFO:teuthology.orchestra.run.smithi064.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T04:54:10.123 INFO:teuthology.orchestra.run.smithi064.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-19T04:54:10.135 INFO:teuthology.orchestra.run.smithi103.stdout:(154/156): librados2-19.1.1-361.g2f599a46.el9.x 6.3 MB/s | 3.4 MB 00:00 2024-09-19T04:54:10.182 INFO:teuthology.orchestra.run.smithi064.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-19T04:54:10.185 INFO:teuthology.orchestra.run.smithi103.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-19T04:54:10.207 INFO:teuthology.orchestra.run.smithi064.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-19T04:54:10.241 INFO:teuthology.orchestra.run.smithi064.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-19T04:54:10.244 INFO:teuthology.orchestra.run.smithi103.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-19T04:54:10.248 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:54:10.248 INFO:teuthology.orchestra.run.smithi103.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-19T04:54:10.266 INFO:teuthology.orchestra.run.smithi064.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-19T04:54:10.291 INFO:teuthology.orchestra.run.smithi064.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-19T04:54:10.321 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-19T04:54:10.322 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-19T04:54:10.325 INFO:teuthology.orchestra.run.smithi064.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-19T04:54:10.350 INFO:teuthology.orchestra.run.smithi064.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 835 kB/s | 19 kB 00:00 2024-09-19T04:54:10.375 INFO:teuthology.orchestra.run.smithi064.stdout:(127/156): python3-jaraco-8.2.1-3.el9.noarch.rp 79 kB/s | 11 kB 00:00 2024-09-19T04:54:10.401 INFO:teuthology.orchestra.run.smithi064.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-19T04:54:10.426 INFO:teuthology.orchestra.run.smithi064.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:54:10.451 INFO:teuthology.orchestra.run.smithi064.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T04:54:10.502 INFO:teuthology.orchestra.run.smithi064.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-09-19T04:54:10.527 INFO:teuthology.orchestra.run.smithi064.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-09-19T04:54:10.552 INFO:teuthology.orchestra.run.smithi064.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-19T04:54:10.577 INFO:teuthology.orchestra.run.smithi064.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T04:54:10.603 INFO:teuthology.orchestra.run.smithi064.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T04:54:10.628 INFO:teuthology.orchestra.run.smithi064.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-09-19T04:54:10.653 INFO:teuthology.orchestra.run.smithi064.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T04:54:10.687 INFO:teuthology.orchestra.run.smithi064.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-09-19T04:54:10.714 INFO:teuthology.orchestra.run.smithi064.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T04:54:10.737 INFO:teuthology.orchestra.run.smithi064.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-19T04:54:10.763 INFO:teuthology.orchestra.run.smithi064.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T04:54:10.788 INFO:teuthology.orchestra.run.smithi064.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 984 kB/s | 74 kB 00:00 2024-09-19T04:54:10.814 INFO:teuthology.orchestra.run.smithi064.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:54:10.840 INFO:teuthology.orchestra.run.smithi064.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T04:54:10.865 INFO:teuthology.orchestra.run.smithi064.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 388 kB/s | 20 kB 00:00 2024-09-19T04:54:10.899 INFO:teuthology.orchestra.run.smithi064.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T04:54:10.925 INFO:teuthology.orchestra.run.smithi064.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-19T04:54:11.000 INFO:teuthology.orchestra.run.smithi064.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T04:54:11.325 INFO:teuthology.orchestra.run.smithi064.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 8.0 MB/s | 3.4 MB 00:00 2024-09-19T04:54:11.367 INFO:teuthology.orchestra.run.smithi064.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-09-19T04:54:11.417 INFO:teuthology.orchestra.run.smithi064.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 6.4 MB/s | 3.2 MB 00:00 2024-09-19T04:54:11.442 INFO:teuthology.orchestra.run.smithi064.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 542 kB/s | 41 kB 00:00 2024-09-19T04:54:11.482 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-19T04:54:11.593 INFO:teuthology.orchestra.run.smithi064.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-19T04:54:11.635 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-19T04:54:11.635 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-19T04:54:11.651 INFO:teuthology.orchestra.run.smithi064.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 5.5 MB/s | 1.8 MB 00:00 2024-09-19T04:54:11.676 INFO:teuthology.orchestra.run.smithi064.stdout:(155/156): NetworkManager-tui-1.51.0-1.el9.x86_ 960 kB/s | 249 kB 00:00 2024-09-19T04:54:11.793 INFO:teuthology.orchestra.run.smithi064.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-19T04:54:11.799 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:54:11.799 INFO:teuthology.orchestra.run.smithi064.stdout:Total 16 MB/s | 226 MB 00:13 2024-09-19T04:54:13.047 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-19T04:54:13.215 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-19T04:54:13.215 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-19T04:54:13.857 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-19T04:54:13.857 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-19T04:54:15.550 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-19T04:54:15.551 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-19T04:54:16.709 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-19T04:54:16.812 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:54:16.850 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:54:16.912 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:54:16.979 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:54:17.063 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:54:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:54:17.460 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:54:17.555 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:54:17.677 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:54:17.706 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:54:17.797 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:54:17.827 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:54:17.996 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:54:18.057 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:54:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:54:18.188 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:54:18.211 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:54:18.276 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:54:18.295 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:54:18.318 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:54:18.505 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:54:18.672 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:54:18.812 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:54:18.923 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:19.003 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:19.238 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:54:19.454 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:54:19.482 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:54:19.517 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:54:19.582 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:54:19.626 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:54:19.658 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:54:19.702 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:54:19.735 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:54:19.762 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:54:19.831 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:54:19.884 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:54:19.916 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:54:19.950 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:54:20.025 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:54:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:54:20.104 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:54:20.132 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:54:20.146 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-19T04:54:20.201 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:54:20.247 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:54:20.290 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:54:20.320 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:54:20.328 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:54:20.382 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:54:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:54:20.474 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:54:20.516 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:54:20.650 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:54:20.716 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:54:21.032 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:54:21.051 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:54:21.118 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:54:21.149 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:54:21.168 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:54:21.192 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:54:21.230 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:54:21.273 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:54:21.290 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:54:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:54:21.429 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:54:21.458 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:54:21.482 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:54:21.536 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:54:21.610 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:54:21.650 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:54:21.658 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:54:21.686 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:54:21.705 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:54:21.732 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:54:21.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:54:21.769 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:54:21.811 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:54:21.844 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:54:21.854 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:54:21.875 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:54:21.921 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:54:21.951 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:54:21.973 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:54:21.988 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:54:22.030 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:54:22.107 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:54:22.171 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:54:22.227 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-19T04:54:22.340 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:54:22.356 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:54:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:54:22.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:54:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:54:22.458 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:54:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:54:22.532 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:54:22.570 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:22.596 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:54:22.668 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:22.761 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:54:22.838 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:54:22.871 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:54:22.896 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:54:22.902 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:54:23.010 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:54:23.021 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:54:23.058 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:54:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:54:23.150 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:54:23.157 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:54:23.182 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:54:23.185 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:54:23.200 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:54:23.209 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:54:23.222 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:54:23.259 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:54:23.266 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:54:23.298 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:54:23.301 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:54:23.309 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:54:23.329 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:54:23.342 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:54:23.350 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:54:23.360 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:54:23.396 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:54:23.397 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:54:23.428 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:54:23.428 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:54:23.463 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:54:23.468 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:54:23.503 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:54:23.518 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:54:23.531 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:54:23.586 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:54:23.590 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:54:23.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:54:23.613 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:54:23.624 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:54:23.633 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:54:23.664 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:54:23.667 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:54:23.719 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:54:23.743 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:54:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:54:23.767 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:54:23.778 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:54:23.810 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:54:23.815 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:54:23.821 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:54:23.832 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:54:23.850 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:54:23.854 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:54:23.921 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:54:23.973 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:54:24.047 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:54:24.053 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:54:24.180 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:54:24.228 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:54:24.235 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:54:24.375 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:54:24.494 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:24.532 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:54:24.565 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:24.579 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:54:24.613 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:54:24.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:54:24.756 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:54:24.767 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:54:24.792 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:54:24.812 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:54:24.838 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:54:24.867 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:54:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:54:24.899 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:54:24.902 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:54:24.931 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:54:24.940 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:54:24.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:54:24.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:54:25.042 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:54:25.089 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:54:25.125 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:54:25.154 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:54:25.164 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:54:25.196 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:54:25.216 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:54:25.232 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:54:25.256 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:54:25.293 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:54:25.299 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:54:25.332 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:54:25.348 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:54:25.359 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:54:25.421 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:54:25.430 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:54:25.453 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:54:25.489 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:54:25.516 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:54:25.536 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:54:25.554 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:54:25.572 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:54:25.611 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:54:25.615 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:54:25.665 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:54:25.693 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:54:25.733 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:54:25.777 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:54:25.800 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:54:25.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:54:25.852 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:54:25.878 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:54:25.892 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:54:25.893 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:54:25.929 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:54:25.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:54:26.002 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:54:26.006 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:54:26.067 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:54:26.160 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:54:26.215 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:54:26.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:54:26.401 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:54:26.425 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:54:26.532 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:54:26.682 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:54:26.719 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:54:26.748 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:54:26.779 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:54:26.782 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:54:26.830 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:54:26.869 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:54:26.879 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:54:26.900 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:54:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:54:26.934 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:54:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:54:26.967 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:54:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:54:27.009 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:54:27.043 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:54:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:54:27.213 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:54:27.233 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:54:27.256 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:54:27.298 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:54:27.324 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:54:27.351 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:54:27.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:54:27.384 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:54:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:54:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:54:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:54:27.516 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:54:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:54:27.555 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:54:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:54:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:54:27.683 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:54:27.686 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:54:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:54:27.731 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:54:27.740 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:54:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:54:27.852 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:54:27.889 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:54:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:54:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:54:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:54:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:54:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:54:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:54:28.372 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:54:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:54:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:54:28.408 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:54:28.460 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:54:28.480 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:54:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:54:28.510 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:54:28.512 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:54:28.546 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:54:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:54:28.651 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:54:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:54:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:54:28.709 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:54:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:54:28.761 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:54:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:54:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:54:28.848 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:54:28.852 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:54:28.887 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:54:28.889 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:54:28.928 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:54:28.979 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:54:29.021 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:54:29.094 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:54:29.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:54:29.120 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:54:29.160 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:54:29.191 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:54:29.207 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:54:29.307 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:54:29.360 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:54:29.875 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:54:29.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:54:30.130 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:54:30.183 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:54:30.216 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:54:30.319 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:54:30.372 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:54:30.417 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:54:30.508 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:54:30.552 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:54:30.743 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:54:30.783 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:54:30.846 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:30.863 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:54:30.881 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:31.253 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:54:31.343 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:54:31.370 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:54:31.404 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:54:31.427 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:31.468 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:54:31.559 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:54:31.559 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:54:31.559 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:54:31.559 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:31.561 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:54:31.602 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:54:31.677 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:54:31.713 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:54:31.767 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:54:31.858 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:54:31.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:54:31.941 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:54:32.421 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:54:32.453 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:54:32.525 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:54:32.555 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:54:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:54:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:54:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:54:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:54:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:54:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:54:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:54:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:54:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:54:34.217 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:54:34.238 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:54:34.310 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:54:34.340 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:54:34.908 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:34.944 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:54:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:54:35.622 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:54:35.622 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:54:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:54:36.858 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:36.887 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:37.465 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:54:37.507 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:54:37.623 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:54:37.624 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:54:37.624 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:54:37.624 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:37.658 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:54:42.983 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-09-19T04:54:42.984 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:43.169 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:54:43.185 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:54:43.185 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:43.185 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:54:43.186 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:44.673 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:54:44.699 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:54:44.699 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:44.699 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:54:44.699 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:54:44.699 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:54:44.700 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:44.924 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:54:44.946 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:54:44.947 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:44.947 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:54:44.947 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:54:44.947 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:54:44.947 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:45.383 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:54:45.406 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:54:45.406 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:45.406 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:54:45.406 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:54:45.406 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:54:45.407 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:45.504 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:54:45.560 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:54:45.615 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:54:45.800 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:54:45.840 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:54:45.909 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:54:45.999 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:54:46.161 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:54:46.203 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:54:46.239 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:54:46.267 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:54:46.737 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:54:46.765 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:54:46.797 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:54:46.859 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:54:46.896 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:54:47.284 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-19T04:54:47.285 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:47.456 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:54:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:54:47.472 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:47.472 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:54:47.472 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:47.938 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:54:47.973 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:54:48.934 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:54:48.965 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:49.067 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:54:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:54:49.274 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:54:49.274 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:49.274 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:54:49.274 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:54:49.274 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:54:49.274 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:49.318 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:54:49.330 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:54:49.427 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:54:49.480 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:54:49.695 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:54:49.719 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:54:49.719 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:49.719 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:54:49.719 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:54:49.720 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:54:49.720 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:49.825 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:54:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-09-19T04:54:49.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:49.929 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:54:50.104 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:54:50.113 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:54:50.121 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:54:50.121 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:50.121 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:54:50.122 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:50.163 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:54:50.249 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:54:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:54:50.396 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:54:50.428 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:54:50.452 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:54:50.453 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:50.453 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:54:50.453 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:54:50.453 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:54:50.454 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:50.477 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:54:50.495 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:54:50.518 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:54:50.519 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:50.519 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:54:50.519 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:50.525 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:54:50.571 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:54:50.579 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:54:50.598 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:54:50.602 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:54:50.602 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:50.602 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:54:50.602 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:54:50.602 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:54:50.603 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:50.905 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:54:50.928 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:54:50.928 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:50.929 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:54:50.929 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:54:50.929 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:54:50.929 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:54:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:54:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:54:51.079 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:54:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:54:51.170 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:54:51.658 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:54:51.686 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:54:51.686 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:51.686 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:54:51.686 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:54:51.686 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:54:51.686 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:51.926 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:54:51.950 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:54:51.951 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:51.951 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:54:51.951 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:54:51.951 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:54:51.951 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:52.219 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:54:52.264 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:54:52.410 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:54:52.435 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:54:52.435 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:52.435 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:54:52.435 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:54:52.435 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:54:52.436 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:52.541 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:54:52.599 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:54:52.663 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:54:52.866 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:54:52.920 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:54:53.001 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:54:53.099 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:54:53.226 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:54:53.249 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:54:53.256 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:54:53.294 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:54:53.327 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:54:53.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:54:53.359 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:54:53.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:54:53.558 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:54:53.582 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:54:53.582 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:53.582 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:54:53.582 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:54:53.583 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:54:53.583 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:53.626 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:54:53.637 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:54:53.746 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:54:53.801 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:54:53.801 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:54:53.829 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:54:53.860 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:54:53.924 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:54:53.951 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:54:54.721 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:54:54.754 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:54:54.778 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:54:54.778 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:54.778 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:54:54.778 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:54:54.778 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:54:54.779 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:54.821 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:54:54.844 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:54:54.844 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:54.844 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:54:54.844 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:54.915 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:54:54.938 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:54:54.939 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:54.939 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:54:54.939 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:54:54.939 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:54:54.939 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:55.055 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:54:55.095 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:54:55.248 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:54:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:54:55.271 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:55.271 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:54:55.271 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:54:55.271 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:54:55.271 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:54:55.483 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:54:55.525 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:54:55.565 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:54:55.599 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:54:55.616 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:54:55.808 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:54:55.854 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:54:55.885 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:54:55.920 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:54:55.969 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:54:56.105 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:54:56.145 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:54:56.254 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:54:56.423 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:54:56.447 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:54:56.473 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:54:56.473 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:56.473 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:54:56.473 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:54:56.473 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:54:56.473 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:56.523 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:54:56.523 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:54:56.535 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:54:56.566 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:54:56.586 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:54:56.655 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:54:56.714 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:54:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:54:57.487 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:54:57.537 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:54:57.640 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:54:57.673 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:54:57.699 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:54:57.713 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:54:57.732 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:54:57.733 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:54:57.742 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:54:57.767 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:54:57.767 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:57.767 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:54:57.767 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:54:57.767 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:54:57.767 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:57.809 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:54:57.817 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:54:57.817 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:54:57.833 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:54:57.833 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:57.833 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:54:57.833 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:57.894 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:54:57.900 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:54:57.918 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:54:57.918 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:57.918 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:54:57.918 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:54:57.919 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:54:57.919 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:57.925 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:54:57.930 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:54:57.930 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:54:58.110 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:54:58.111 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:54:58.194 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:54:58.244 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:54:58.260 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:54:58.269 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:54:58.269 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:54:58.269 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:54:58.269 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:54:58.269 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:54:58.270 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:54:59.827 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:54:59.868 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:54:59.908 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:54:59.941 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:54:59.968 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:55:00.168 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:55:00.224 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:55:00.254 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:55:00.284 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:55:00.330 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:55:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:55:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:55:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:55:01.869 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:55:01.960 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:55:02.009 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:55:02.104 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:55:02.138 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:55:02.172 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:55:02.206 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:55:02.206 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:55:02.291 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:55:02.291 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:55:02.375 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:55:02.400 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:55:02.405 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:55:02.405 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:55:02.585 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:55:02.585 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:55:02.678 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:55:02.744 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:55:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:55:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:55:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:55:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:55:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:55:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:55:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:55:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:55:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:55:03.818 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:55:03.819 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:55:03.820 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:55:03.821 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:55:03.821 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:55:03.821 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:55:03.834 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:55:03.835 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:55:03.836 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:55:03.837 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:55:03.838 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:55:03.839 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:55:03.840 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:55:03.841 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:55:03.842 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:55:03.843 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:55:03.844 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:55:03.845 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:55:04.310 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:55:04.348 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:55:04.370 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:55:05.274 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:55:05.362 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:55:05.412 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:55:05.515 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:55:05.548 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:55:05.582 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:55:05.625 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:55:05.625 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:55:05.717 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:55:05.717 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:55:05.801 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:55:05.828 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:55:05.834 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:55:05.834 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:55:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:55:06.029 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:55:06.113 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:55:06.182 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.616 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.617 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:55:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:55:06.622 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:55:06.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:55:06.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:55:06.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:55:06.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:55:06.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:55:06.628 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:55:06.629 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:55:06.630 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:55:06.630 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:55:06.630 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:55:06.630 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:55:06.630 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:55:06.630 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-19T04:55:07.066 DEBUG:teuthology.parallel:result is None 2024-09-19T04:55:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:55:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:55:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:55:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:55:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:55:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:55:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:55:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:55:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:55:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:55:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:55:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:55:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:55:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:55:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:55:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:55:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:55:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:55:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:55:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:55:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:55:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:55:11.202 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:11.202 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:11.203 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.204 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.205 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:55:11.206 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:55:11.207 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:55:11.208 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:55:11.209 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:55:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:55:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:55:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:55:11.213 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:55:11.214 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:55:11.215 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:55:11.216 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:55:11.216 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:55:11.216 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:55:11.216 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:55:11.217 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:55:11.217 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:55:11.217 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-19T04:55:11.683 DEBUG:teuthology.parallel:result is None 2024-09-19T04:55:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:55:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:55:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:55:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:55:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:55:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:55:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:55:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:55:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:55:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:55:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:55:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:55:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:55:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:55:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:55:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:55:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:55:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:55:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:55:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:55:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:55:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:55:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:55:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:55:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:55:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:55:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:55:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:55:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:55:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:55:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:55:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:55:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:55:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:55:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:55:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:55:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:55:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:55:14.374 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-19T04:55:14.776 DEBUG:teuthology.parallel:result is None 2024-09-19T04:55:14.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:55:14.919 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:55:14.942 INFO:teuthology.orchestra.run.smithi064.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:55:14.942 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:55:14.942 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:55:14.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:55:15.083 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:55:15.109 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:55:15.110 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:55:15.110 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:55:15.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:55:15.244 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:55:15.272 INFO:teuthology.orchestra.run.smithi174.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:55:15.272 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:55:15.272 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:55:15.274 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T04:55:15.275 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:55:15.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:55:15.305 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:55:15.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:55:15.338 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:55:15.339 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:55:15.370 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T04:55:15.371 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:55:15.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:55:15.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:55:15.466 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:55:15.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:55:15.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:55:15.565 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:55:15.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:55:15.594 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:55:15.662 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T04:55:15.662 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:55:15.662 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:55:15.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:55:15.757 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:55:15.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:55:15.787 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:55:15.857 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:55:15.858 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:55:15.885 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:55:15.951 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T04:55:15.951 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:55:15.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:55:15.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:55:16.049 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:55:16.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:55:16.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:55:16.148 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:55:16.149 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:55:16.176 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:55:16.242 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T04:55:16.358 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T04:55:16.358 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:55:16.359 INFO:tasks.cephadm:Cluster fsid is 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:16.359 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T04:55:16.359 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.b': '172.21.15.103', 'mon.c': '172.21.15.174'} 2024-09-19T04:55:16.360 INFO:tasks.cephadm:First mon is mon.a on smithi064 2024-09-19T04:55:16.360 INFO:tasks.cephadm:First mgr is x 2024-09-19T04:55:16.360 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T04:55:16.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-09-19T04:55:16.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-19T04:55:16.423 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-09-19T04:55:16.451 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T04:55:16.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:55:16.600 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T04:55:16.739 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:55:16.741 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:55:16.741 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:55:16.741 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.121 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:55 /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.122 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.483 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:55 /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.483 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.876 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:55 /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.876 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.896 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.916 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:55:17.950 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T04:55:17.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:55:17.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:55:17.959 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:55:19.549 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:55:19.614 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:55:19.626 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:55:47.870 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T04:55:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:55:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:55:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-19T04:55:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:55:47.871 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T04:55:47.871 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T04:55:55.432 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T04:55:55.433 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:55:55.433 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:55:55.433 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-09-19T04:55:55.433 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:55:55.433 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-09-19T04:55:55.433 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T04:56:14.601 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T04:56:14.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-09-19T04:56:14.666 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-19T04:56:14.699 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-09-19T04:56:14.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-09-19T04:56:14.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-19T04:56:14.788 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-09-19T04:56:14.818 INFO:tasks.cephadm:Writing seed config... 2024-09-19T04:56:14.819 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T04:56:14.819 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T04:56:14.819 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T04:56:14.819 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T04:56:14.819 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T04:56:14.819 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T04:56:14.820 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T04:56:14.821 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-19T04:56:14.822 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T04:56:14.823 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T04:56:14.824 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:56:14.824 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T04:56:14.840 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5ccce474-7643-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T04:56:14.841 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T04:56:14.883 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T04:56:14.926 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T04:56:14.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:56:15.135 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:56:15.135 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '5ccce474-7643-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-09-19T04:56:15.136 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T04:56:15.136 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2024-09-19T04:56:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:56:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2024-09-19T04:56:15.161 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2024-09-19T04:56:15.169 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:56:15.169 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:56:15.175 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:56:15.175 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-19T04:56:15.182 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-09-19T04:56:15.189 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-09-19T04:56:15.189 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:56:15.189 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2024-09-19T04:56:15.212 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:56:15.212 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T04:56:15.212 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2024-09-19T04:56:15.212 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2024-09-19T04:56:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:56:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:56:15.225 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:56:15.226 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-19T04:56:15.233 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-09-19T04:56:15.239 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-09-19T04:56:15.239 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:56:15.240 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2024-09-19T04:56:15.240 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:15.240 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 140065969992704 on /run/cephadm/5ccce474-7643-11ef-bceb-c7b262605968.lock 2024-09-19T04:56:15.240 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 140065969992704 acquired on /run/cephadm/5ccce474-7643-11ef-bceb-c7b262605968.lock 2024-09-19T04:56:15.241 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2024-09-19T04:56:15.241 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2024-09-19T04:56:15.241 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-19T04:56:15.244 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2024-09-19T04:56:15.244 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-09-19T04:56:15.247 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T04:56:15.247 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T04:56:15.247 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T04:56:15.249 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T04:56:15.250 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T04:56:15.250 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:56:15.250 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T04:56:15.250 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-19T04:56:15.250 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:56:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-09-19T04:56:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-09-19T04:56:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T04:56:15.251 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T04:56:15.252 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:56:15.775 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:56:15.775 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:56:15.775 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T04:56:15.776 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T04:56:15.776 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T04:56:15.776 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:56:15.776 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T04:56:16.688 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:56:16.688 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:56:16.688 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T04:56:17.644 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2024-09-19T04:56:17.644 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2024-09-19T04:56:18.666 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDxrutmOyJKORAAO/RR9tC7D9N2eFc56Ib7vA== 2024-09-19T04:56:19.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDyrutm+0IdOhAADWVAxFJdcgnKNU5hRHV3RQ== 2024-09-19T04:56:20.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQDzrutmRPH6MhAA/Gh11kkSrJ+D9WNAi+P8Vg== 2024-09-19T04:56:20.486 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = quincy 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:56:21.417 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:56:21.418 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2024-09-19T04:56:22.331 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2024-09-19T04:56:22.550 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T04:56:22.703 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T04:56:22.904 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5ccce474-7643-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-5ccce474-7643-11ef-bceb-c7b262605968.target. 2024-09-19T04:56:22.904 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5ccce474-7643-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-5ccce474-7643-11ef-bceb-c7b262605968.target. 2024-09-19T04:56:23.139 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:56:23.139 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T04:56:23.323 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5ccce474-7643-11ef-bceb-c7b262605968.target.wants/ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5ccce474-7643-11ef-bceb-c7b262605968@.service. 2024-09-19T04:56:23.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 podman[30583]: 2024-09-19 04:56:23.469990086 +0000 UTC m=+0.016358385 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:56:23.855 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:56:23.855 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-19T04:56:23.862 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:56:23.862 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-19T04:56:23.862 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-19T04:56:23.862 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:56:23.862 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2024-09-19T04:56:23.863 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2024-09-19T04:56:23.986 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 podman[30583]: 2024-09-19 04:56:23.756658048 +0000 UTC m=+0.303026353 container init b0f50a1fa5116cbf178f2aa86f3b11d2139b8b38030464ee8b8345df83fe214b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T04:56:23.986 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 podman[30583]: 2024-09-19 04:56:23.759861747 +0000 UTC m=+0.306230055 container start b0f50a1fa5116cbf178f2aa86f3b11d2139b8b38030464ee8b8345df83fe214b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:56:23.986 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 ceph-mon[30602]: mkfs 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:23.986 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 ceph-mon[30602]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:56:23.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 bash[30583]: b0f50a1fa5116cbf178f2aa86f3b11d2139b8b38030464ee8b8345df83fe214b 2024-09-19T04:56:23.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:23 smithi064 systemd[1]: Started Ceph mon.a for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: monmap epoch 1 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: last_changed 2024-09-19T04:56:20.817470+0000 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: min_mon_release 19 (squid) 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: election_strategy: 1 2024-09-19T04:56:24.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:56:24.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: fsmap 2024-09-19T04:56:24.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:56:24.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: mgrmap e1: no daemons active 2024-09-19T04:56:24.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:24 smithi064 ceph-mon[30602]: from='client.? 172.21.15.64:0/3294726919' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.572951s) 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T04:56:24.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.296+0000 7f0708cc8640 1 Processor -- start 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.296+0000 7f0708cc8640 1 -- start start 2024-09-19T04:56:24.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.296+0000 7f0708cc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070410a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.296+0000 7f0708cc8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f070407be10 con 0x7f0704108130 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.297+0000 7f070259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070410a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.297+0000 7f070259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070410a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54740/0 (socket says 172.21.15.64:54740) 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.297+0000 7f070259c640 1 -- 172.21.15.64:0/721939253 learned_addr learned my addr 172.21.15.64:0/721939253 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.297+0000 7f070259c640 1 -- 172.21.15.64:0/721939253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070410aa60 con 0x7f0704108130 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.299+0000 7f070259c640 1 --2- 172.21.15.64:0/721939253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070410a520 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009820 tx=0x7f06f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829ef74d1a8234c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.299+0000 7f070159a640 1 -- 172.21.15.64:0/721939253 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f4037070 con 0x7f0704108130 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.299+0000 7f070159a640 1 -- 172.21.15.64:0/721939253 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f4032440 con 0x7f0704108130 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.299+0000 7f070159a640 1 -- 172.21.15.64:0/721939253 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f4032740 con 0x7f0704108130 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.299+0000 7f0708cc8640 1 -- 172.21.15.64:0/721939253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 msgr2=0x7f070410a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:24.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.299+0000 7f0708cc8640 1 --2- 172.21.15.64:0/721939253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070410a520 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009820 tx=0x7f06f402a270 comp rx=0 tx=0).stop 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 -- 172.21.15.64:0/721939253 shutdown_connections 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 --2- 172.21.15.64:0/721939253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070410a520 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 -- 172.21.15.64:0/721939253 >> 172.21.15.64:0/721939253 conn(0x7f0704101d90 msgr2=0x7f07041041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 -- 172.21.15.64:0/721939253 shutdown_connections 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 -- 172.21.15.64:0/721939253 wait complete. 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 Processor -- start 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.300+0000 7f0708cc8640 1 -- start start 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f0708cc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f0708cc8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f070410aed0 con 0x7f0704108130 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f070259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f070259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54748/0 (socket says 172.21.15.64:54748) 2024-09-19T04:56:24.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f070259c640 1 -- 172.21.15.64:0/3294726919 learned_addr learned my addr 172.21.15.64:0/3294726919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:24.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f070259c640 1 -- 172.21.15.64:0/3294726919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070419b340 con 0x7f0704108130 2024-09-19T04:56:24.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f070259c640 1 --2- 172.21.15.64:0/3294726919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070419ae00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f06f40090b0 tx=0x7f06f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:24.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f4041070 con 0x7f0704108130 2024-09-19T04:56:24.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f403c7a0 con 0x7f0704108130 2024-09-19T04:56:24.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.301+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070419b5d0 con 0x7f0704108130 2024-09-19T04:56:24.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.302+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070419e2c0 con 0x7f0704108130 2024-09-19T04:56:24.953 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.302+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f4037040 con 0x7f0704108130 2024-09-19T04:56:24.953 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.302+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f06f4044070 con 0x7f0704108130 2024-09-19T04:56:24.953 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.302+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06f403d3f0 con 0x7f0704108130 2024-09-19T04:56:24.953 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.302+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e40052c0 con 0x7f0704108130 2024-09-19T04:56:24.953 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.305+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f06f4046450 con 0x7f0704108130 2024-09-19T04:56:24.953 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.367+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f06e4005880 con 0x7f0704108130 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.367+0000 7f06f37fe640 1 -- 172.21.15.64:0/3294726919 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f06f4046630 con 0x7f0704108130 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 msgr2=0x7f070419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 --2- 172.21.15.64:0/3294726919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070419ae00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f06f40090b0 tx=0x7f06f40046c0 comp rx=0 tx=0).stop 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 shutdown_connections 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 --2- 172.21.15.64:0/3294726919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0704108130 0x7f070419ae00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 >> 172.21.15.64:0/3294726919 conn(0x7f0704101d90 msgr2=0x7f07041034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 shutdown_connections 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:24.368+0000 7f0708cc8640 1 -- 172.21.15.64:0/3294726919 wait complete. 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2024-09-19T04:56:24.954 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T04:56:26.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:25 smithi064 ceph-mon[30602]: from='client.? 172.21.15.64:0/309978606' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:56:26.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:25 smithi064 ceph-mon[30602]: from='client.? 172.21.15.64:0/309978606' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:56:26.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.345+0000 7f9a9420a640 1 Processor -- start 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.345+0000 7f9a9420a640 1 -- start start 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.345+0000 7f9a9420a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.345+0000 7f9a9420a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c106f40 con 0x7f9a8c106570 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.346+0000 7f9a91fa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.346+0000 7f9a91fa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54762/0 (socket says 172.21.15.64:54762) 2024-09-19T04:56:26.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.346+0000 7f9a91fa6640 1 -- 172.21.15.64:0/3851928217 learned_addr learned my addr 172.21.15.64:0/3851928217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.346+0000 7f9a91fa6640 1 -- 172.21.15.64:0/3851928217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c107720 con 0x7f9a8c106570 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.346+0000 7f9a91fa6640 1 --2- 172.21.15.64:0/3851928217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c106970 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009e80 tx=0x7f9a8402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394f09d402e29b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a90fa4640 1 -- 172.21.15.64:0/3851928217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a84036070 con 0x7f9a8c106570 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a90fa4640 1 -- 172.21.15.64:0/3851928217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a84031400 con 0x7f9a8c106570 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a90fa4640 1 -- 172.21.15.64:0/3851928217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a84031700 con 0x7f9a8c106570 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 -- 172.21.15.64:0/3851928217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 msgr2=0x7f9a8c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 --2- 172.21.15.64:0/3851928217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c106970 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009e80 tx=0x7f9a8402a7a0 comp rx=0 tx=0).stop 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 -- 172.21.15.64:0/3851928217 shutdown_connections 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 --2- 172.21.15.64:0/3851928217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c106970 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 -- 172.21.15.64:0/3851928217 >> 172.21.15.64:0/3851928217 conn(0x7f9a8c101d80 msgr2=0x7f9a8c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 -- 172.21.15.64:0/3851928217 shutdown_connections 2024-09-19T04:56:26.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.347+0000 7f9a9420a640 1 -- 172.21.15.64:0/3851928217 wait complete. 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a9420a640 1 Processor -- start 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a9420a640 1 -- start start 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a9420a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a9420a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c107cc0 con 0x7f9a8c106570 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a91fa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a91fa6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54764/0 (socket says 172.21.15.64:54764) 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.348+0000 7f9a91fa6640 1 -- 172.21.15.64:0/309978606 learned_addr learned my addr 172.21.15.64:0/309978606 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a91fa6640 1 -- 172.21.15.64:0/309978606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c19b340 con 0x7f9a8c106570 2024-09-19T04:56:26.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a91fa6640 1 --2- 172.21.15.64:0/309978606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c19ae00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9a84034040 tx=0x7f9a84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a84041070 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8403c650 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a84036040 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c19b5d0 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c19e2c0 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a84044070 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.349+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9a84046c10 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.350+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c108310 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.352+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9a8403c950 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.411+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9a8c19f9e0 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.414+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f9a8403cb30 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.415+0000 7f9a8affd640 1 -- 172.21.15.64:0/309978606 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9a84031400 con 0x7f9a8c106570 2024-09-19T04:56:26.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 msgr2=0x7f9a8c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 --2- 172.21.15.64:0/309978606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c19ae00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9a84034040 tx=0x7f9a84004a40 comp rx=0 tx=0).stop 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 shutdown_connections 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 --2- 172.21.15.64:0/309978606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a8c106570 0x7f9a8c19ae00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 >> 172.21.15.64:0/309978606 conn(0x7f9a8c101d80 msgr2=0x7f9a8c07b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 shutdown_connections 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:25.416+0000 7f9a9420a640 1 -- 172.21.15.64:0/309978606 wait complete. 2024-09-19T04:56:26.076 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2024-09-19T04:56:27.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:26 smithi064 ceph-mon[30602]: from='client.? 172.21.15.64:0/2831227292' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:28.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.470+0000 7fa5933a2640 1 Processor -- start 2024-09-19T04:56:28.096 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.470+0000 7fa5933a2640 1 -- start start 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.471+0000 7fa5933a2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.471+0000 7fa5933a2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c106d40 con 0x7fa58c106370 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.471+0000 7fa5923a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.471+0000 7fa5923a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54778/0 (socket says 172.21.15.64:54778) 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.471+0000 7fa5923a0640 1 -- 172.21.15.64:0/648189828 learned_addr learned my addr 172.21.15.64:0/648189828 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.471+0000 7fa5923a0640 1 -- 172.21.15.64:0/648189828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c107520 con 0x7fa58c106370 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.472+0000 7fa5923a0640 1 --2- 172.21.15.64:0/648189828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c106770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa580009820 tx=0x7fa58002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89e901b0196435e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.472+0000 7fa59139e640 1 -- 172.21.15.64:0/648189828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa580037070 con 0x7fa58c106370 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.472+0000 7fa59139e640 1 -- 172.21.15.64:0/648189828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa5800328d0 con 0x7fa58c106370 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.472+0000 7fa59139e640 1 -- 172.21.15.64:0/648189828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5800307d0 con 0x7fa58c106370 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.472+0000 7fa5933a2640 1 -- 172.21.15.64:0/648189828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 msgr2=0x7fa58c106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:28.097 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.472+0000 7fa5933a2640 1 --2- 172.21.15.64:0/648189828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c106770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa580009820 tx=0x7fa58002a270 comp rx=0 tx=0).stop 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 -- 172.21.15.64:0/648189828 shutdown_connections 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 --2- 172.21.15.64:0/648189828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c106770 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 -- 172.21.15.64:0/648189828 >> 172.21.15.64:0/648189828 conn(0x7fa58c101b80 msgr2=0x7fa58c103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 -- 172.21.15.64:0/648189828 shutdown_connections 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 -- 172.21.15.64:0/648189828 wait complete. 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 Processor -- start 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.473+0000 7fa5933a2640 1 -- start start 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5933a2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5933a2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c10a8d0 con 0x7fa58c106370 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5923a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:28.098 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5923a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54780/0 (socket says 172.21.15.64:54780) 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5923a0640 1 -- 172.21.15.64:0/2831227292 learned_addr learned my addr 172.21.15.64:0/2831227292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5923a0640 1 -- 172.21.15.64:0/2831227292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c19b950 con 0x7fa58c106370 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5923a0640 1 --2- 172.21.15.64:0/2831227292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c19b410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa580009940 tx=0x7fa580032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa580042070 con 0x7fa58c106370 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa580030c80 con 0x7fa58c106370 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c19bbe0 con 0x7fa58c106370 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa580037040 con 0x7fa58c106370 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.474+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c19e8d0 con 0x7fa58c106370 2024-09-19T04:56:28.099 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.475+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa580045070 con 0x7fa58c106370 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.475+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa580047a40 con 0x7fa58c106370 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.475+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58c10aa50 con 0x7fa58c106370 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.478+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa58003c700 con 0x7fa58c106370 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.537+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa58c1953f0 con 0x7fa58c106370 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.537+0000 7fa57f7fe640 1 -- 172.21.15.64:0/2831227292 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa58003c8e0 con 0x7fa58c106370 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 msgr2=0x7fa58c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 --2- 172.21.15.64:0/2831227292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c19b410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa580009940 tx=0x7fa580032e10 comp rx=0 tx=0).stop 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 shutdown_connections 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 --2- 172.21.15.64:0/2831227292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa58c106370 0x7fa58c19b410 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 >> 172.21.15.64:0/2831227292 conn(0x7fa58c101b80 msgr2=0x7fa58c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 shutdown_connections 2024-09-19T04:56:28.100 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:26.538+0000 7fa5933a2640 1 -- 172.21.15.64:0/2831227292 wait complete. 2024-09-19T04:56:28.101 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2024-09-19T04:56:28.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:28 smithi064 systemd[1]: Stopping Ceph mon.a for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T04:56:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a[30598]: 2024-09-19T04:56:29.083+0000 7fc044b25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:56:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a[30598]: 2024-09-19T04:56:29.083+0000 7fc044b25640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T04:56:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:29 smithi064 podman[30841]: 2024-09-19 04:56:29.289109181 +0000 UTC m=+1.022334061 container died b0f50a1fa5116cbf178f2aa86f3b11d2139b8b38030464ee8b8345df83fe214b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T04:56:33.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:32 smithi064 podman[30841]: 2024-09-19 04:56:32.783994996 +0000 UTC m=+4.517219864 container cleanup b0f50a1fa5116cbf178f2aa86f3b11d2139b8b38030464ee8b8345df83fe214b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:56:33.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:32 smithi064 bash[30841]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:56:35.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:34 smithi064 podman[30853]: 2024-09-19 04:56:34.915284188 +0000 UTC m=+5.623477909 container remove b0f50a1fa5116cbf178f2aa86f3b11d2139b8b38030464ee8b8345df83fe214b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T04:56:35.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:35 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T04:56:35.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:35 smithi064 systemd[1]: Stopped Ceph mon.a for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:56:35.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:35 smithi064 systemd[1]: Starting Ceph mon.a for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T04:56:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:35 smithi064 podman[30941]: 2024-09-19 04:56:35.161854397 +0000 UTC m=+0.016745806 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:56:36.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 podman[30941]: 2024-09-19 04:56:36.112830701 +0000 UTC m=+0.967722092 container create 32af6c18b4dd9417c414d715f79d9f990376f0ac82e0c17d63fccf098299ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:56:36.588 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 podman[30941]: 2024-09-19 04:56:36.508482761 +0000 UTC m=+1.363374163 container init 32af6c18b4dd9417c414d715f79d9f990376f0ac82e0c17d63fccf098299ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 podman[30941]: 2024-09-19 04:56:36.512039781 +0000 UTC m=+1.366931174 container start 32af6c18b4dd9417c414d715f79d9f990376f0ac82e0c17d63fccf098299ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: pidfile_write: ignore empty --pid-file 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: load: jerasure load: lrc 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:56:36.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Git sha 0 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: DB SUMMARY 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: DB Session ID: ZMGW9F6WTZ2CGO2RURZZ 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106092 ; 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:56:36.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:56:36.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:56:36.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.env: 0x55ece48da940 2024-09-19T04:56:36.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:56:36.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.info_log: 0x55ece5bfb6e0 2024-09-19T04:56:36.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:56:36.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.statistics: (nil) 2024-09-19T04:56:36.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.use_fsync: 0 2024-09-19T04:56:36.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:56:36.705 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.db_log_dir: 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.wal_dir: 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:56:36.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.write_buffer_manager: 0x55ece5c0adc0 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:56:36.707 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.unordered_write: 0 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.row_cache: None 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.wal_filter: None 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:56:36.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.wal_compression: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:56:36.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_open_files: -1 2024-09-19T04:56:36.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Compression algorithms supported: 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kZSTD supported: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:56:36.711 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.merge_operator: 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_filter: None 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ece5bfb320) 2024-09-19T04:56:36.712 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_type: 0 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_index_type: 0 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_shortening: 1 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: checksum: 4 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: no_block_cache: 0 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache: 0x55ece5bef1f0 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_options: 2024-09-19T04:56:36.713 INFO:journalctl@ceph.mon.a.smithi064.stdout: capacity : 536870912 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_shard_bits : 4 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: strict_capacity_limit : 0 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_compressed: (nil) 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: persistent_cache: (nil) 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size: 4096 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size_deviation: 10 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_restart_interval: 16 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_block_restart_interval: 1 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: metadata_block_size: 4096 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: partition_filters: 0 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: use_delta_encoding: 1 2024-09-19T04:56:36.714 INFO:journalctl@ceph.mon.a.smithi064.stdout: filter_policy: bloomfilter 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: whole_key_filtering: 1 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: verify_compression: 0 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: format_version: 5 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_index_compression: 1 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_align: 0 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: max_auto_readahead_size: 262144 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: prepopulate_block_cache: 0 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression: NoCompression 2024-09-19T04:56:36.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.num_levels: 7 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:56:36.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:56:36.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:56:36.718 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:56:36.719 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:56:36.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.ttl: 2592000 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:56:36.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2cf3bf25-5ad0-49c5-95b0-58f20879d1aa 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721796540191, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721796541383, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 98961, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14514, "raw_average_key_size": 49, "raw_value_size": 91113, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726721796, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2cf3bf25-5ad0-49c5-95b0-58f20879d1aa", "db_session_id": "ZMGW9F6WTZ2CGO2RURZZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721796541488, "job": 1, "event": "recovery_finished"} 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:56:36.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ece5c1ce00 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: DB pointer 0x55ece5d18000 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** DB Stats ** 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-19T04:56:36.723 INFO:journalctl@ceph.mon.a.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: L0 2/0 100.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: Sum 2/0 100.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:56:36.724 INFO:journalctl@ceph.mon.a.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative compaction: 0.00 GB write, 27.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval compaction: 0.00 GB write, 27.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache BinnedLRUCache@0x55ece5bef1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???) e1 preinit fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:36.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).mds e1 new map 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).mds e1 print_map 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: e1 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: btime 2024-09-19T04:56:23:797082+0000 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: legacy client fscid: -1 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout: No filesystems configured 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T04:56:36.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 bash[30941]: 32af6c18b4dd9417c414d715f79d9f990376f0ac82e0c17d63fccf098299ebb7 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 systemd[1]: Started Ceph mon.a for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: monmap epoch 1 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: last_changed 2024-09-19T04:56:20.817470+0000 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: min_mon_release 19 (squid) 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: election_strategy: 1 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: fsmap 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:56:36.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:36 smithi064 ceph-mon[30960]: mgrmap e1: no daemons active 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.045+0000 7f40d27a1640 1 Processor -- start 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40d27a1640 1 -- start start 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40d27a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40d27a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc106f40 con 0x7f40cc106570 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60234/0 (socket says 172.21.15.64:60234) 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40cbfff640 1 -- 172.21.15.64:0/1835945648 learned_addr learned my addr 172.21.15.64:0/1835945648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:37.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.046+0000 7f40cbfff640 1 -- 172.21.15.64:0/1835945648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc107720 con 0x7f40cc106570 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.048+0000 7f40cbfff640 1 --2- 172.21.15.64:0/1835945648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc106970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009820 tx=0x7f40bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26afa1e10ed0eff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.048+0000 7f40caffd640 1 -- 172.21.15.64:0/1835945648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc037070 con 0x7f40cc106570 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40caffd640 1 -- 172.21.15.64:0/1835945648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f40bc0328d0 con 0x7f40cc106570 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40caffd640 1 -- 172.21.15.64:0/1835945648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc0307d0 con 0x7f40cc106570 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 -- 172.21.15.64:0/1835945648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 msgr2=0x7f40cc106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 --2- 172.21.15.64:0/1835945648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc106970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009820 tx=0x7f40bc02a270 comp rx=0 tx=0).stop 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 -- 172.21.15.64:0/1835945648 shutdown_connections 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 --2- 172.21.15.64:0/1835945648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc106970 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 -- 172.21.15.64:0/1835945648 >> 172.21.15.64:0/1835945648 conn(0x7f40cc101d80 msgr2=0x7f40cc1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:37.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 -- 172.21.15.64:0/1835945648 shutdown_connections 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.049+0000 7f40d27a1640 1 -- 172.21.15.64:0/1835945648 wait complete. 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40d27a1640 1 Processor -- start 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40d27a1640 1 -- start start 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40d27a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc191750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40d27a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc10aad0 con 0x7f40cc106570 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60240/0 (socket says 172.21.15.64:60240) 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.050+0000 7f40cbfff640 1 -- 172.21.15.64:0/3340473583 learned_addr learned my addr 172.21.15.64:0/3340473583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40cbfff640 1 -- 172.21.15.64:0/3340473583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc191c90 con 0x7f40cc106570 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40cbfff640 1 --2- 172.21.15.64:0/3340473583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc191750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f40bc030520 tx=0x7f40bc032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc042070 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f40bc030cc0 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc18fe80 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc037040 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc1903c0 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f40bc045070 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.051+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f40bc0479d0 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.052+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc10ac50 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.054+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f40bc03d9c0 con 0x7f40cc106570 2024-09-19T04:56:37.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.113+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f40cc190700 con 0x7f40cc106570 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.115+0000 7f40c97fa640 1 -- 172.21.15.64:0/3340473583 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f40bc047bd0 con 0x7f40cc106570 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 msgr2=0x7f40cc191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 --2- 172.21.15.64:0/3340473583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc191750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f40bc030520 tx=0x7f40bc032dd0 comp rx=0 tx=0).stop 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 shutdown_connections 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 --2- 172.21.15.64:0/3340473583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc106570 0x7f40cc191750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 >> 172.21.15.64:0/3340473583 conn(0x7f40cc101d80 msgr2=0x7f40cc1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 shutdown_connections 2024-09-19T04:56:37.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:37.116+0000 7f40d27a1640 1 -- 172.21.15.64:0/3340473583 wait complete. 2024-09-19T04:56:37.687 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T04:56:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:56:37.707 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2024-09-19T04:56:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T04:56:37.708 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T04:56:37.950 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:56:37.950 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T04:56:38.103 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5ccce474-7643-11ef-bceb-c7b262605968.target.wants/ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5ccce474-7643-11ef-bceb-c7b262605968@.service. 2024-09-19T04:56:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:38 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3340473583' entity='client.admin' 2024-09-19T04:56:38.400 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 podman[31174]: 2024-09-19 04:56:38.244639371 +0000 UTC m=+0.016191328 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:56:38.651 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:56:38.651 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-19T04:56:38.659 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:56:38.659 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-19T04:56:38.659 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-19T04:56:38.659 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:56:38.667 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:56:38.667 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-19T04:56:38.675 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:56:38.675 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-19T04:56:38.675 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-19T04:56:38.675 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T04:56:38.676 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2024-09-19T04:56:38.676 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2024-09-19T04:56:38.744 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 podman[31174]: 2024-09-19 04:56:38.552587777 +0000 UTC m=+0.324139723 container init da6dc243a6d36ede172c0a929aae5e93e272c42f527f7322440ff71103d03a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T04:56:38.744 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 podman[31174]: 2024-09-19 04:56:38.555397207 +0000 UTC m=+0.326949153 container start da6dc243a6d36ede172c0a929aae5e93e272c42f527f7322440ff71103d03a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T04:56:38.744 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 bash[31174]: da6dc243a6d36ede172c0a929aae5e93e272c42f527f7322440ff71103d03a2a 2024-09-19T04:56:38.745 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 systemd[1]: Started Ceph mgr.x for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:56:38.745 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:38.681+0000 7fe7ab9c4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:56:38.745 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:38.742+0000 7fe7ab9c4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:39.031 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:38.900+0000 7fe7ab9c4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:56:39.031 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:38.973+0000 7fe7ab9c4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:56:39.286 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:39 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1393498698' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:56:39.286 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:39.134+0000 7fe7ab9c4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:39.286 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:39.284+0000 7fe7ab9c4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:56:39.756 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:56:39.757 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:56:39.758 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.759 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:39.760 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:56:23:797082+0000", 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:56:39.761 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:56:23.797748+0000", 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.110+0000 7feac940f640 1 Processor -- start 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.110+0000 7feac940f640 1 -- start start 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.111+0000 7feac940f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac410a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:39.762 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.111+0000 7feac940f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feac407be90 con 0x7feac41080b0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.111+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac410a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.111+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac410a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60270/0 (socket says 172.21.15.64:60270) 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.111+0000 7feac2ffd640 1 -- 172.21.15.64:0/1888808893 learned_addr learned my addr 172.21.15.64:0/1888808893 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.111+0000 7feac2ffd640 1 -- 172.21.15.64:0/1888808893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac410a9e0 con 0x7feac41080b0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.112+0000 7feac2ffd640 1 --2- 172.21.15.64:0/1888808893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac410a4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feab4009820 tx=0x7feab402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95bd8a4946b985f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.112+0000 7feac1ffb640 1 -- 172.21.15.64:0/1888808893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab4037070 con 0x7feac41080b0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.112+0000 7feac1ffb640 1 -- 172.21.15.64:0/1888808893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7feab40328d0 con 0x7feac41080b0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.112+0000 7feac1ffb640 1 -- 172.21.15.64:0/1888808893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab40307d0 con 0x7feac41080b0 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 -- 172.21.15.64:0/1888808893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 msgr2=0x7feac410a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 --2- 172.21.15.64:0/1888808893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac410a4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feab4009820 tx=0x7feab402a270 comp rx=0 tx=0).stop 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 -- 172.21.15.64:0/1888808893 shutdown_connections 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 --2- 172.21.15.64:0/1888808893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac410a4a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:39.763 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 -- 172.21.15.64:0/1888808893 >> 172.21.15.64:0/1888808893 conn(0x7feac4101d90 msgr2=0x7feac41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 -- 172.21.15.64:0/1888808893 shutdown_connections 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.113+0000 7feac940f640 1 -- 172.21.15.64:0/1888808893 wait complete. 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac940f640 1 Processor -- start 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac940f640 1 -- start start 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac940f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac4107930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac940f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feac410ba20 con 0x7feac41080b0 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac4107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac4107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60280/0 (socket says 172.21.15.64:60280) 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.114+0000 7feac2ffd640 1 -- 172.21.15.64:0/1393498698 learned_addr learned my addr 172.21.15.64:0/1393498698 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:39.764 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feac2ffd640 1 -- 172.21.15.64:0/1393498698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac4107e70 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feac2ffd640 1 --2- 172.21.15.64:0/1393498698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac4107930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feab40097f0 tx=0x7feab4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab4037040 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7feab4032bb0 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac4106060 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41065a0 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab403a390 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feab403a600 con 0x7feac41080b0 2024-09-19T04:56:39.765 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.115+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feab4044c40 con 0x7feac41080b0 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.116+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac4108510 con 0x7feac41080b0 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.118+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7feab4032d60 con 0x7feac41080b0 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.181+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feac41068e0 con 0x7feac41080b0 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.181+0000 7feaabfff640 1 -- 172.21.15.64:0/1393498698 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7feab4044400 con 0x7feac41080b0 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 msgr2=0x7feac4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 --2- 172.21.15.64:0/1393498698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac4107930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feab40097f0 tx=0x7feab4030d00 comp rx=0 tx=0).stop 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 shutdown_connections 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 --2- 172.21.15.64:0/1393498698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feac41080b0 0x7feac4107930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 >> 172.21.15.64:0/1393498698 conn(0x7feac4101d90 msgr2=0x7feac41028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 shutdown_connections 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:39.182+0000 7feac940f640 1 -- 172.21.15.64:0/1393498698 wait complete. 2024-09-19T04:56:39.766 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2024-09-19T04:56:39.881 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:39.621+0000 7fe7ab9c4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:56:39.881 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:39.685+0000 7fe7ab9c4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:56:39.881 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:39.879+0000 7fe7ab9c4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:56:41.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:40 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:40.807+0000 7fe7ab9c4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:56:41.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:40 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:40.878+0000 7fe7ab9c4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:56:41.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:41 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:41.639+0000 7fe7ab9c4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:56:41.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:41 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:41.761+0000 7fe7ab9c4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:56:42.207 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:41 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:41.915+0000 7fe7ab9c4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:42.207 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:41 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:41.979+0000 7fe7ab9c4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:56:42.207 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.135+0000 7fe7ab9c4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:56:42.208 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.206+0000 7fe7ab9c4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:56:42.504 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.269+0000 7fe7ab9c4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:56:42.504 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.435+0000 7fe7ab9c4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:56:42.505 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.503+0000 7fe7ab9c4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:56:42.770 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.572+0000 7fe7ab9c4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:56:42.770 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.659+0000 7fe7ab9c4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:42.770 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.721+0000 7fe7ab9c4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:56:43.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:42.785+0000 7fe7ab9c4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:56:43.691 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:43.634+0000 7fe7ab9c4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:56:44.074 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: Activating manager daemon x 2024-09-19T04:56:44.074 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: mgrmap e2: x(active, starting, since 0.00309596s) 2024-09-19T04:56:44.074 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: Manager daemon x is now available 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' 2024-09-19T04:56:44.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:43 smithi064 ceph-mon[30960]: from='mgr.14100 172.21.15.64:0/965913176' entity='mgr.x' 2024-09-19T04:56:44.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:56:44.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:56:44.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:56:44.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:56:44.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:56:44.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:56:44.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:56:44.389 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:56:44.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:56:44.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:56:44.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:56:44.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:56:23:797082+0000", 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:56:44.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:56:23.797748+0000", 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:56:44.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f2840be1640 1 Processor -- start 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f2840be1640 1 -- start start 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f2840be1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c10a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f2840be1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f283c07be00 con 0x7f283c108120 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f283a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c10a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f283a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c10a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60354/0 (socket says 172.21.15.64:60354) 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.744+0000 7f283a59c640 1 -- 172.21.15.64:0/1919231686 learned_addr learned my addr 172.21.15.64:0/1919231686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.745+0000 7f283a59c640 1 -- 172.21.15.64:0/1919231686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c10aa50 con 0x7f283c108120 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.745+0000 7f283a59c640 1 --2- 172.21.15.64:0/1919231686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c10a510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f282c009e80 tx=0x7f282c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107d741676ab6492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f283959a640 1 -- 172.21.15.64:0/1919231686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c036070 con 0x7f283c108120 2024-09-19T04:56:44.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f283959a640 1 -- 172.21.15.64:0/1919231686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f282c031890 con 0x7f283c108120 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f283959a640 1 -- 172.21.15.64:0/1919231686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c007980 con 0x7f283c108120 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 -- 172.21.15.64:0/1919231686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 msgr2=0x7f283c10a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 --2- 172.21.15.64:0/1919231686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c10a510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f282c009e80 tx=0x7f282c02a7a0 comp rx=0 tx=0).stop 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 -- 172.21.15.64:0/1919231686 shutdown_connections 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 --2- 172.21.15.64:0/1919231686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c10a510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 -- 172.21.15.64:0/1919231686 >> 172.21.15.64:0/1919231686 conn(0x7f283c101d80 msgr2=0x7f283c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 -- 172.21.15.64:0/1919231686 shutdown_connections 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.746+0000 7f2840be1640 1 -- 172.21.15.64:0/1919231686 wait complete. 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.747+0000 7f2840be1640 1 Processor -- start 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.747+0000 7f2840be1640 1 -- start start 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.747+0000 7f2840be1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c1079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:44.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.747+0000 7f2840be1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10b9d0 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f283a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c1079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f283a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c1079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60368/0 (socket says 172.21.15.64:60368) 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f283a59c640 1 -- 172.21.15.64:0/512358117 learned_addr learned my addr 172.21.15.64:0/512358117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f283a59c640 1 -- 172.21.15.64:0/512358117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c107ee0 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f283a59c640 1 --2- 172.21.15.64:0/512358117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c1079a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f282c009680 tx=0x7f282c031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c041020 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c1060d0 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f282c03c900 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c036040 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.748+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c106610 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.749+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 2) ==== 46217+0+0 (secure 0 0 0) 0x7f282c044020 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.749+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f282c06df30 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.750+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c108580 con 0x7f283c108120 2024-09-19T04:56:44.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.752+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f282c04b020 con 0x7f283c108120 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.814+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f283c106db0 con 0x7f283c108120 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.815+0000 7f28337fe640 1 -- 172.21.15.64:0/512358117 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f282c038db0 con 0x7f283c108120 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 msgr2=0x7f283c1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 --2- 172.21.15.64:0/512358117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c1079a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f282c009680 tx=0x7f282c031c70 comp rx=0 tx=0).stop 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 shutdown_connections 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 --2- 172.21.15.64:0/512358117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f283c108120 0x7f283c1079a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 >> 172.21.15.64:0/512358117 conn(0x7f283c101d80 msgr2=0x7f283c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 shutdown_connections 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:43.816+0000 7f2840be1640 1 -- 172.21.15.64:0/512358117 wait complete. 2024-09-19T04:56:44.396 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2024-09-19T04:56:45.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:44 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/512358117' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:56:45.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:44 smithi064 ceph-mon[30960]: mgrmap e3: x(active, since 1.00635s) 2024-09-19T04:56:46.802 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:46 smithi064 ceph-mon[30960]: mgrmap e4: x(active, since 2s) 2024-09-19T04:56:47.585 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:56:47.586 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:56:47.587 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:56:47.588 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:56:23:797082+0000", 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:56:47.589 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:56:47.590 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:56:23.797748+0000", 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.816+0000 7f7da674b640 1 Processor -- start 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.816+0000 7f7da674b640 1 -- start start 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.816+0000 7f7da674b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.816+0000 7f7da674b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01080b0 con 0x7f7da01076e0 2024-09-19T04:56:47.591 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.817+0000 7f7d9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.817+0000 7f7d9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60622/0 (socket says 172.21.15.64:60622) 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.817+0000 7f7d9ffff640 1 -- 172.21.15.64:0/728669571 learned_addr learned my addr 172.21.15.64:0/728669571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.817+0000 7f7d9ffff640 1 -- 172.21.15.64:0/728669571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da0108840 con 0x7f7da01076e0 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.818+0000 7f7d9ffff640 1 --2- 172.21.15.64:0/728669571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da0107ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009820 tx=0x7f7d9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a9670d06b554a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.818+0000 7f7d9effd640 1 -- 172.21.15.64:0/728669571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90037070 con 0x7f7da01076e0 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.818+0000 7f7d9effd640 1 -- 172.21.15.64:0/728669571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7d900328d0 con 0x7f7da01076e0 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.818+0000 7f7d9effd640 1 -- 172.21.15.64:0/728669571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d900307d0 con 0x7f7da01076e0 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 -- 172.21.15.64:0/728669571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 msgr2=0x7f7da0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 --2- 172.21.15.64:0/728669571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da0107ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009820 tx=0x7f7d9002a270 comp rx=0 tx=0).stop 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 -- 172.21.15.64:0/728669571 shutdown_connections 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 --2- 172.21.15.64:0/728669571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da0107ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:47.592 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 -- 172.21.15.64:0/728669571 >> 172.21.15.64:0/728669571 conn(0x7f7da0102ed0 msgr2=0x7f7da01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 -- 172.21.15.64:0/728669571 shutdown_connections 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 -- 172.21.15.64:0/728669571 wait complete. 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.819+0000 7f7da674b640 1 Processor -- start 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7da674b640 1 -- start start 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7da674b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7da674b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010bbf0 con 0x7f7da01076e0 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7d9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7d9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60634/0 (socket says 172.21.15.64:60634) 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7d9ffff640 1 -- 172.21.15.64:0/887165057 learned_addr learned my addr 172.21.15.64:0/887165057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.820+0000 7f7d9ffff640 1 -- 172.21.15.64:0/887165057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da019bb50 con 0x7f7da01076e0 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.821+0000 7f7d9ffff640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da019b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009940 tx=0x7f7d90032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.821+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90042070 con 0x7f7da01076e0 2024-09-19T04:56:47.593 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.821+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7d90030c80 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.821+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da019bde0 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.821+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d90037040 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.821+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da019ead0 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.822+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f7d90045070 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.822+0000 7f7d9d7fa640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f7d880391b0 0x7f7d8803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.822+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d9006c050 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.822+0000 7f7d9f7fe640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f7d880391b0 0x7f7d8803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.822+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da010bd70 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.823+0000 7f7d9f7fe640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f7d880391b0 0x7f7d8803b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d940098c0 tx=0x7f7d94006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:46.827+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d9004b170 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.071+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7da019f120 con 0x7f7da01076e0 2024-09-19T04:56:47.594 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.071+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/887165057 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f7d90045520 con 0x7f7da01076e0 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.073+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f7d880391b0 msgr2=0x7f7d8803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.073+0000 7f7da674b640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f7d880391b0 0x7f7d8803b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d940098c0 tx=0x7f7d94006db0 comp rx=0 tx=0).stop 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.073+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 msgr2=0x7f7da019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.073+0000 7f7da674b640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da019b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009940 tx=0x7f7d90032e10 comp rx=0 tx=0).stop 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.074+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 shutdown_connections 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.074+0000 7f7da674b640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f7d880391b0 0x7f7d8803b670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.074+0000 7f7da674b640 1 --2- 172.21.15.64:0/887165057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da01076e0 0x7f7da019b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.074+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 >> 172.21.15.64:0/887165057 conn(0x7f7da0102ed0 msgr2=0x7f7da0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.074+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 shutdown_connections 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.074+0000 7f7da674b640 1 -- 172.21.15.64:0/887165057 wait complete. 2024-09-19T04:56:47.595 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2024-09-19T04:56:47.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:47 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/887165057' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:56:48.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:48 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1148943539' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:56:49.055 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-19T04:56:49.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc6d0b640 1 Processor -- start 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc6d0b640 1 -- start start 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc6d0b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc6d0b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01080b0 con 0x7f2fc01076e0 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc4aa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc4aa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60642/0 (socket says 172.21.15.64:60642) 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc4aa7640 1 -- 172.21.15.64:0/4277492004 learned_addr learned my addr 172.21.15.64:0/4277492004 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:49.057 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.961+0000 7f2fc4aa7640 1 -- 172.21.15.64:0/4277492004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc0108840 con 0x7f2fc01076e0 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.962+0000 7f2fc4aa7640 1 --2- 172.21.15.64:0/4277492004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc0107ae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009820 tx=0x7f2fb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8886e6097eb7b2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.962+0000 7f2fbf7fe640 1 -- 172.21.15.64:0/4277492004 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb4037070 con 0x7f2fc01076e0 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.962+0000 7f2fbf7fe640 1 -- 172.21.15.64:0/4277492004 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2fb40328d0 con 0x7f2fc01076e0 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.962+0000 7f2fbf7fe640 1 -- 172.21.15.64:0/4277492004 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb40307d0 con 0x7f2fc01076e0 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/4277492004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 msgr2=0x7f2fc0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 --2- 172.21.15.64:0/4277492004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc0107ae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009820 tx=0x7f2fb402a270 comp rx=0 tx=0).stop 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/4277492004 shutdown_connections 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 --2- 172.21.15.64:0/4277492004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc0107ae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/4277492004 >> 172.21.15.64:0/4277492004 conn(0x7f2fc0102ed0 msgr2=0x7f2fc01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/4277492004 shutdown_connections 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.963+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/4277492004 wait complete. 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc6d0b640 1 Processor -- start 2024-09-19T04:56:49.058 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc6d0b640 1 -- start start 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc6d0b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc6d0b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010bbf0 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc4aa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc4aa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60656/0 (socket says 172.21.15.64:60656) 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc4aa7640 1 -- 172.21.15.64:0/1148943539 learned_addr learned my addr 172.21.15.64:0/1148943539 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc4aa7640 1 -- 172.21.15.64:0/1148943539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc019bb50 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc4aa7640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc019b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40097f0 tx=0x7f2fb4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb4037040 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc019bde0 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2fb4032bb0 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.964+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb403a390 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.965+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc019ead0 con 0x7f2fc01076e0 2024-09-19T04:56:49.059 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.966+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2fb403a5d0 con 0x7f2fc01076e0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.966+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc010bd70 con 0x7f2fc01076e0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.966+0000 7f2fbdffb640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f2fb00391b0 0x7f2fb003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.966+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2fb406c5b0 con 0x7f2fc01076e0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.966+0000 7f2fbffff640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f2fb00391b0 0x7f2fb003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.967+0000 7f2fbffff640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f2fb00391b0 0x7f2fb003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0098c0 tx=0x7f2fac006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:47.971+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2fb40313a0 con 0x7f2fc01076e0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.150+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2fc0107b60 con 0x7f2fc01076e0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.151+0000 7f2fbdffb640 1 -- 172.21.15.64:0/1148943539 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f2fb406bfd0 con 0x7f2fc01076e0 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f2fb00391b0 msgr2=0x7f2fb003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f2fb00391b0 0x7f2fb003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0098c0 tx=0x7f2fac006db0 comp rx=0 tx=0).stop 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 msgr2=0x7f2fc019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:49.060 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc019b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40097f0 tx=0x7f2fb4030d00 comp rx=0 tx=0).stop 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 shutdown_connections 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f2fb00391b0 0x7f2fb003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 --2- 172.21.15.64:0/1148943539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fc01076e0 0x7f2fc019b610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 >> 172.21.15.64:0/1148943539 conn(0x7f2fc0102ed0 msgr2=0x7f2fc0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 shutdown_connections 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:48.153+0000 7f2fc6d0b640 1 -- 172.21.15.64:0/1148943539 wait complete. 2024-09-19T04:56:49.061 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2024-09-19T04:56:50.010 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:49 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3473636087' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T04:56:50.010 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ignoring --setuser ceph since I am not root 2024-09-19T04:56:50.011 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ignoring --setgroup ceph since I am not root 2024-09-19T04:56:50.011 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:49.839+0000 7f0da29a9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:56:50.011 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:49.899+0000 7f0da29a9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:50.280 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:50.054+0000 7f0da29a9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:56:50.280 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:50.124+0000 7f0da29a9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:56:50.280 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:50.278+0000 7f0da29a9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:50.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.454+0000 7fc08bda7640 1 Processor -- start 2024-09-19T04:56:50.309 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.454+0000 7fc08bda7640 1 -- start start 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.454+0000 7fc08bda7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc084107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.454+0000 7fc08bda7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc0841080b0 con 0x7fc0841076e0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.455+0000 7fc089b43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc084107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.455+0000 7fc089b43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc084107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60662/0 (socket says 172.21.15.64:60662) 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.455+0000 7fc089b43640 1 -- 172.21.15.64:0/1459851565 learned_addr learned my addr 172.21.15.64:0/1459851565 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.455+0000 7fc089b43640 1 -- 172.21.15.64:0/1459851565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc084108840 con 0x7fc0841076e0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.456+0000 7fc089b43640 1 --2- 172.21.15.64:0/1459851565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc084107ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc080009820 tx=0x7fc08002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ed1457a7f20687 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.456+0000 7fc088b41640 1 -- 172.21.15.64:0/1459851565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080037070 con 0x7fc0841076e0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.456+0000 7fc088b41640 1 -- 172.21.15.64:0/1459851565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc0800328d0 con 0x7fc0841076e0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.456+0000 7fc088b41640 1 -- 172.21.15.64:0/1459851565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0800307d0 con 0x7fc0841076e0 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 -- 172.21.15.64:0/1459851565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 msgr2=0x7fc084107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 --2- 172.21.15.64:0/1459851565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc084107ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc080009820 tx=0x7fc08002a270 comp rx=0 tx=0).stop 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 -- 172.21.15.64:0/1459851565 shutdown_connections 2024-09-19T04:56:50.310 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 --2- 172.21.15.64:0/1459851565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc084107ae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 -- 172.21.15.64:0/1459851565 >> 172.21.15.64:0/1459851565 conn(0x7fc084102ed0 msgr2=0x7fc0841052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 -- 172.21.15.64:0/1459851565 shutdown_connections 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 -- 172.21.15.64:0/1459851565 wait complete. 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 Processor -- start 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.457+0000 7fc08bda7640 1 -- start start 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc08bda7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc08419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc08bda7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc08410bbf0 con 0x7fc0841076e0 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc089b43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc08419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc089b43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc08419b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60664/0 (socket says 172.21.15.64:60664) 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc089b43640 1 -- 172.21.15.64:0/3473636087 learned_addr learned my addr 172.21.15.64:0/3473636087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc089b43640 1 -- 172.21.15.64:0/3473636087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08419bb40 con 0x7fc0841076e0 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc089b43640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc08419b600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097f0 tx=0x7fc080030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080037040 con 0x7fc0841076e0 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc080032bb0 con 0x7fc0841076e0 2024-09-19T04:56:50.311 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08419bdd0 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.458+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc08003a390 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.459+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08419eac0 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.459+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc08003a580 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.460+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08410bd70 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.460+0000 7fc07affd640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7fc0700391b0 0x7fc07003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.460+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc08006c560 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.460+0000 7fc089342640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7fc0700391b0 0x7fc07003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.460+0000 7fc089342640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7fc0700391b0 0x7fc07003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc0740098c0 tx=0x7fc074006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.465+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc080032dd0 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.687+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc08419f130 con 0x7fc0841076e0 2024-09-19T04:56:50.312 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.698+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc080031560 con 0x7fc0841076e0 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.700+0000 7fc07affd640 1 -- 172.21.15.64:0/3473636087 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fc080042c20 con 0x7fc0841076e0 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7fc0700391b0 msgr2=0x7fc07003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7fc0700391b0 0x7fc07003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc0740098c0 tx=0x7fc074006db0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 msgr2=0x7fc08419b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc08419b600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097f0 tx=0x7fc080030d00 comp rx=0 tx=0).stop 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 shutdown_connections 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7fc0700391b0 0x7fc07003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 --2- 172.21.15.64:0/3473636087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0841076e0 0x7fc08419b600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:50.313 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 >> 172.21.15.64:0/3473636087 conn(0x7fc084102ed0 msgr2=0x7fc084103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:50.314 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 shutdown_connections 2024-09-19T04:56:50.314 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:49.701+0000 7fc08bda7640 1 -- 172.21.15.64:0/3473636087 wait complete. 2024-09-19T04:56:50.539 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:50.429+0000 7f0da29a9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:56:50.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3473636087' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T04:56:50.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-mon[30960]: mgrmap e5: x(active, since 6s) 2024-09-19T04:56:50.832 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:50.766+0000 7f0da29a9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:56:50.832 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:50.830+0000 7f0da29a9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:56:51.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:51.027+0000 7f0da29a9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:56:51.541 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:56:51.541 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b66a10640 1 Processor -- start 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b66a10640 1 -- start start 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b66a10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b60106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b66a10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60106f40 con 0x7f9b60106570 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b60106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b60106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60686/0 (socket says 172.21.15.64:60686) 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b5ffff640 1 -- 172.21.15.64:0/4074957275 learned_addr learned my addr 172.21.15.64:0/4074957275 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:51.542 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.691+0000 7f9b5ffff640 1 -- 172.21.15.64:0/4074957275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b60107720 con 0x7f9b60106570 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.692+0000 7f9b5ffff640 1 --2- 172.21.15.64:0/4074957275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b60106970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009820 tx=0x7f9b5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3565d65c8c59ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.692+0000 7f9b5effd640 1 -- 172.21.15.64:0/4074957275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50037070 con 0x7f9b60106570 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.692+0000 7f9b5effd640 1 -- 172.21.15.64:0/4074957275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b500328d0 con 0x7f9b60106570 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.692+0000 7f9b5effd640 1 -- 172.21.15.64:0/4074957275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b500307d0 con 0x7f9b60106570 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 -- 172.21.15.64:0/4074957275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 msgr2=0x7f9b60106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 --2- 172.21.15.64:0/4074957275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b60106970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009820 tx=0x7f9b5002a270 comp rx=0 tx=0).stop 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 -- 172.21.15.64:0/4074957275 shutdown_connections 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 --2- 172.21.15.64:0/4074957275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b60106970 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 -- 172.21.15.64:0/4074957275 >> 172.21.15.64:0/4074957275 conn(0x7f9b60101d80 msgr2=0x7f9b601041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 -- 172.21.15.64:0/4074957275 shutdown_connections 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 -- 172.21.15.64:0/4074957275 wait complete. 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.693+0000 7f9b66a10640 1 Processor -- start 2024-09-19T04:56:51.543 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b66a10640 1 -- start start 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b66a10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b6007ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b66a10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6010aad0 con 0x7f9b60106570 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b6007ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b6007ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60692/0 (socket says 172.21.15.64:60692) 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b5ffff640 1 -- 172.21.15.64:0/966572388 learned_addr learned my addr 172.21.15.64:0/966572388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b5ffff640 1 -- 172.21.15.64:0/966572388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6019e410 con 0x7f9b60106570 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.694+0000 7f9b5ffff640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b6007ba30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9b50030520 tx=0x7f9b50032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.695+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50042070 con 0x7f9b60106570 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.695+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b50030cc0 con 0x7f9b60106570 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.695+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019e5e0 con 0x7f9b60106570 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.695+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50037040 con 0x7f9b60106570 2024-09-19T04:56:51.544 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.695+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6019e980 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.695+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f9b50045070 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.696+0000 7f9b5d7fa640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 0x7f9b4803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.696+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b601069f0 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.696+0000 7f9b5f7fe640 1 -- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 msgr2=0x7f9b4803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/4164984314 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.696+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b5006c260 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.696+0000 7f9b5f7fe640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 0x7f9b4803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.701+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9b5002ecc0 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.897+0000 7f9b5f7fe640 1 -- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 msgr2=0x7f9b4803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/4164984314 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.897+0000 7f9b5f7fe640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 0x7f9b4803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.926+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9b6019f1e0 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.926+0000 7f9b5d7fa640 1 -- 172.21.15.64:0/966572388 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9b5004b170 con 0x7f9b60106570 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.928+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 msgr2=0x7f9b4803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:56:51.545 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.928+0000 7f9b66a10640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 0x7f9b4803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.928+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 msgr2=0x7f9b6007ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.928+0000 7f9b66a10640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b6007ba30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9b50030520 tx=0x7f9b50032dd0 comp rx=0 tx=0).stop 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.929+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 shutdown_connections 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.929+0000 7f9b66a10640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f9b48039200 0x7f9b4803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.929+0000 7f9b66a10640 1 --2- 172.21.15.64:0/966572388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b60106570 0x7f9b6007ba30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.929+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 >> 172.21.15.64:0/966572388 conn(0x7f9b60101d80 msgr2=0x7f9b601028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.929+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 shutdown_connections 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:50.929+0000 7f9b66a10640 1 -- 172.21.15.64:0/966572388 wait complete. 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-09-19T04:56:51.546 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2024-09-19T04:56:51.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:51 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/966572388' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:56:52.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:51.967+0000 7f0da29a9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:56:52.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:52.037+0000 7f0da29a9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:56:52.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:52.787+0000 7f0da29a9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:56:53.277 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:52.907+0000 7f0da29a9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:56:53.277 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.056+0000 7f0da29a9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:53.277 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.120+0000 7f0da29a9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:56:53.277 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.275+0000 7f0da29a9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:56:53.577 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.347+0000 7f0da29a9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:56:53.577 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.409+0000 7f0da29a9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:56:53.577 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.575+0000 7f0da29a9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:56:53.862 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.643+0000 7f0da29a9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:56:53.862 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.712+0000 7f0da29a9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:56:53.862 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.799+0000 7f0da29a9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:53.862 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.861+0000 7f0da29a9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:56:54.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:53.924+0000 7f0da29a9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:56:54.814 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:54.759+0000 7f0da29a9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: Active manager daemon x restarted 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: Activating manager daemon x 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: mgrmap e6: x(active, starting, since 0.00325707s) 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:56:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: Manager daemon x is now available 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:56:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:54 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:56:56.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:55 smithi064 ceph-mon[30960]: Found migration_current of "None". Setting to last migration. 2024-09-19T04:56:56.097 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:55 smithi064 ceph-mon[30960]: mgrmap e7: x(active, since 1.00528s) 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.948+0000 7f4120c62640 1 Processor -- start 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.948+0000 7f4120c62640 1 -- start start 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.949+0000 7f4120c62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.949+0000 7f4120c62640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f411c108080 con 0x7f411c107740 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.949+0000 7f411a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.949+0000 7f411a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60706/0 (socket says 172.21.15.64:60706) 2024-09-19T04:56:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.949+0000 7f411a59c640 1 -- 172.21.15.64:0/3875061099 learned_addr learned my addr 172.21.15.64:0/3875061099 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.949+0000 7f411a59c640 1 -- 172.21.15.64:0/3875061099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c108200 con 0x7f411c107740 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.950+0000 7f411a59c640 1 --2- 172.21.15.64:0/3875061099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c107b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f410c009820 tx=0x7f410c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b575c53ab7fafb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.950+0000 7f411959a640 1 -- 172.21.15.64:0/3875061099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410c037070 con 0x7f411c107740 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.950+0000 7f411959a640 1 -- 172.21.15.64:0/3875061099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f410c0328d0 con 0x7f411c107740 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.950+0000 7f411959a640 1 -- 172.21.15.64:0/3875061099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410c0307d0 con 0x7f411c107740 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 -- 172.21.15.64:0/3875061099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 msgr2=0x7f411c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 --2- 172.21.15.64:0/3875061099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c107b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f410c009820 tx=0x7f410c02a270 comp rx=0 tx=0).stop 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 -- 172.21.15.64:0/3875061099 shutdown_connections 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 --2- 172.21.15.64:0/3875061099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c107b40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 -- 172.21.15.64:0/3875061099 >> 172.21.15.64:0/3875061099 conn(0x7f411c102f90 msgr2=0x7f411c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 -- 172.21.15.64:0/3875061099 shutdown_connections 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.951+0000 7f4120c62640 1 -- 172.21.15.64:0/3875061099 wait complete. 2024-09-19T04:56:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f4120c62640 1 Processor -- start 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f4120c62640 1 -- start start 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f4120c62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c07db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f4120c62640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f411c109740 con 0x7f411c107740 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f411a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f411a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c07db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60718/0 (socket says 172.21.15.64:60718) 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f411a59c640 1 -- 172.21.15.64:0/312779698 learned_addr learned my addr 172.21.15.64:0/312779698 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f411a59c640 1 -- 172.21.15.64:0/312779698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c07e090 con 0x7f411c107740 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.952+0000 7f411a59c640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c07db50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f410c030520 tx=0x7f410c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.953+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410c042070 con 0x7f411c107740 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.953+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c07c280 con 0x7f411c107740 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.953+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f410c030cc0 con 0x7f411c107740 2024-09-19T04:56:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.953+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c07c7c0 con 0x7f411c107740 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.953+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410c037040 con 0x7f411c107740 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.954+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f410c045070 con 0x7f411c107740 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.954+0000 7f410b7fe640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.954+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 --> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f410403bdd0 con 0x7f4104039200 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.954+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f410c06c170 con 0x7f411c107740 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.954+0000 7f4119d9b640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 msgr2=0x7f410403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/4164984314 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:51.954+0000 7f4119d9b640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:52.155+0000 7f4119d9b640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 msgr2=0x7f410403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/4164984314 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:52.155+0000 7f4119d9b640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:52.556+0000 7f4119d9b640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 msgr2=0x7f410403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/4164984314 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:52.556+0000 7f4119d9b640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:53.357+0000 7f4119d9b640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 msgr2=0x7f410403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/4164984314 2024-09-19T04:56:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:53.357+0000 7f4119d9b640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:54.763+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f410c06acf0 con 0x7f411c107740 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:54.763+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 msgr2=0x7f410403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:54.763+0000 7f410b7fe640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.763+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f410c03c640 con 0x7f411c107740 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.764+0000 7f410b7fe640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f410403cb30 0x7f410403ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.764+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f410c03ae00 con 0x7f410403cb30 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.764+0000 7f4119d9b640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f410403cb30 0x7f410403ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.765+0000 7f4119d9b640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f410403cb30 0x7f410403ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4110003980 tx=0x7f41100092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.766+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f410c03ae00 con 0x7f410403cb30 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.778+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f411c107bc0 con 0x7f410403cb30 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.778+0000 7f410b7fe640 1 -- 172.21.15.64:0/312779698 <== mgr.14118 v2:172.21.15.64:6800/2584640926 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f411c107bc0 con 0x7f410403cb30 2024-09-19T04:56:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f410403cb30 msgr2=0x7f410403ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f410403cb30 0x7f410403ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4110003980 tx=0x7f41100092b0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 msgr2=0x7f411c07db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c07db50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f410c030520 tx=0x7f410c032dd0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 shutdown_connections 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f410403cb30 0x7f410403ef20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:6800/4164984314,v1:172.21.15.64:6801/4164984314] conn(0x7f4104039200 0x7f410403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 --2- 172.21.15.64:0/312779698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c107740 0x7f411c07db50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 >> 172.21.15.64:0/312779698 conn(0x7f411c102f90 msgr2=0x7f411c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 shutdown_connections 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:55.779+0000 7f4120c62640 1 -- 172.21.15.64:0/312779698 wait complete. 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2024-09-19T04:56:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2024-09-19T04:56:56.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:56 smithi064 ceph-mon[30960]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:56:56.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:56 smithi064 ceph-mon[30960]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:56:56.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:56 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:56:56.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:56 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8d51ac640 1 Processor -- start 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8d51ac640 1 -- start start 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8d51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8d51ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0108080 con 0x7fa8d0107740 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43248/0 (socket says 172.21.15.64:43248) 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.721+0000 7fa8ced9d640 1 -- 172.21.15.64:0/1375905609 learned_addr learned my addr 172.21.15.64:0/1375905609 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.722+0000 7fa8ced9d640 1 -- 172.21.15.64:0/1375905609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d0108200 con 0x7fa8d0107740 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.722+0000 7fa8ced9d640 1 --2- 172.21.15.64:0/1375905609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d0107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009820 tx=0x7fa8c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb06aa7379292ceb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.512 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8ce59c640 1 -- 172.21.15.64:0/1375905609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c0037070 con 0x7fa8d0107740 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8ce59c640 1 -- 172.21.15.64:0/1375905609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa8c00328d0 con 0x7fa8d0107740 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8ce59c640 1 -- 172.21.15.64:0/1375905609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c00307d0 con 0x7fa8d0107740 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 -- 172.21.15.64:0/1375905609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 msgr2=0x7fa8d0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 --2- 172.21.15.64:0/1375905609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d0107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009820 tx=0x7fa8c002a270 comp rx=0 tx=0).stop 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 -- 172.21.15.64:0/1375905609 shutdown_connections 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 --2- 172.21.15.64:0/1375905609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d0107b40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 -- 172.21.15.64:0/1375905609 >> 172.21.15.64:0/1375905609 conn(0x7fa8d0102f90 msgr2=0x7fa8d01053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 -- 172.21.15.64:0/1375905609 shutdown_connections 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.723+0000 7fa8d51ac640 1 -- 172.21.15.64:0/1375905609 wait complete. 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8d51ac640 1 Processor -- start 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8d51ac640 1 -- start start 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8d51ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8d51ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0109740 con 0x7fa8d0107740 2024-09-19T04:56:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43254/0 (socket says 172.21.15.64:43254) 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.724+0000 7fa8ced9d640 1 -- 172.21.15.64:0/3839908967 learned_addr learned my addr 172.21.15.64:0/3839908967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8ced9d640 1 -- 172.21.15.64:0/3839908967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d007e090 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8ced9d640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d007db50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0030520 tx=0x7fa8c0032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c0042070 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa8c0030cc0 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d007c280 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8c0037040 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.725+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d007c7c0 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.726+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fa8c0045070 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.726+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d0107bc0 con 0x7fa8d0107740 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.727+0000 7fa8ccd99640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa8bc0390f0 0x7fa8bc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:57.514 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.727+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa8c006c0b0 con 0x7fa8d0107740 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.727+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa8bc0390f0 0x7fa8bc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.727+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa8bc0390f0 0x7fa8bc03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80098c0 tx=0x7fa8b8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.732+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8c0046e00 con 0x7fa8d0107740 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.815+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa8c003ca70 con 0x7fa8d0107740 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.928+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa8d007d0a0 con 0x7fa8bc0390f0 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.935+0000 7fa8ccd99640 1 -- 172.21.15.64:0/3839908967 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8d007d0a0 con 0x7fa8bc0390f0 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.937+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa8bc0390f0 msgr2=0x7fa8bc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.937+0000 7fa8d51ac640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa8bc0390f0 0x7fa8bc03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80098c0 tx=0x7fa8b8006db0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.937+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 msgr2=0x7fa8d007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.937+0000 7fa8d51ac640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d007db50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0030520 tx=0x7fa8c0032dd0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.938+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 shutdown_connections 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.938+0000 7fa8d51ac640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa8bc0390f0 0x7fa8bc03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.515 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.938+0000 7fa8d51ac640 1 --2- 172.21.15.64:0/3839908967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d0107740 0x7fa8d007db50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:57.516 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.938+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 >> 172.21.15.64:0/3839908967 conn(0x7fa8d0102f90 msgr2=0x7fa8d0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:57.516 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.938+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 shutdown_connections 2024-09-19T04:56:57.516 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:56.938+0000 7fa8d51ac640 1 -- 172.21.15.64:0/3839908967 wait complete. 2024-09-19T04:56:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:57 smithi064 ceph-mon[30960]: mgrmap e8: x(active, since 2s) 2024-09-19T04:56:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:57 smithi064 ceph-mon[30960]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:57 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:56:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:57 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0c90e640 1 Processor -- start 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0c90e640 1 -- start start 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0c90e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0810a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0c90e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa0807be00 con 0x7faa08108120 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0810a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0810a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43268/0 (socket says 172.21.15.64:43268) 2024-09-19T04:56:58.694 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0659c640 1 -- 172.21.15.64:0/221243296 learned_addr learned my addr 172.21.15.64:0/221243296 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.910+0000 7faa0659c640 1 -- 172.21.15.64:0/221243296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0810aa50 con 0x7faa08108120 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.911+0000 7faa0659c640 1 --2- 172.21.15.64:0/221243296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0810a510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009820 tx=0x7fa9f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fda0094236e399 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.911+0000 7faa05d9b640 1 -- 172.21.15.64:0/221243296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9f8037070 con 0x7faa08108120 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.911+0000 7faa05d9b640 1 -- 172.21.15.64:0/221243296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa9f80328d0 con 0x7faa08108120 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.911+0000 7faa05d9b640 1 -- 172.21.15.64:0/221243296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9f80307d0 con 0x7faa08108120 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 -- 172.21.15.64:0/221243296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 msgr2=0x7faa0810a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 --2- 172.21.15.64:0/221243296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0810a510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009820 tx=0x7fa9f802a270 comp rx=0 tx=0).stop 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 -- 172.21.15.64:0/221243296 shutdown_connections 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 --2- 172.21.15.64:0/221243296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0810a510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 -- 172.21.15.64:0/221243296 >> 172.21.15.64:0/221243296 conn(0x7faa08101d80 msgr2=0x7faa081041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 -- 172.21.15.64:0/221243296 shutdown_connections 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.912+0000 7faa0c90e640 1 -- 172.21.15.64:0/221243296 wait complete. 2024-09-19T04:56:58.695 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0c90e640 1 Processor -- start 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0c90e640 1 -- start start 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0c90e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0c90e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810b9d0 con 0x7faa08108120 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0819b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43270/0 (socket says 172.21.15.64:43270) 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0659c640 1 -- 172.21.15.64:0/958551798 learned_addr learned my addr 172.21.15.64:0/958551798 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.913+0000 7faa0659c640 1 -- 172.21.15.64:0/958551798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0819bb40 con 0x7faa08108120 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.914+0000 7faa0659c640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0819b600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097f0 tx=0x7fa9f8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:58.696 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.914+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9f8037040 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.914+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa9f803b3f0 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.914+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0819bdd0 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.914+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0819eac0 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.914+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9f803a520 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.915+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa9f803b5a0 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.915+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa08108580 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.915+0000 7fa9ff7fe640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa9f4039200 0x7fa9f403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.915+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa9f806c3d0 con 0x7faa08108120 2024-09-19T04:56:58.697 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.915+0000 7fa9fffff640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa9f4039200 0x7fa9f403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.916+0000 7fa9fffff640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa9f4039200 0x7fa9f403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00098c0 tx=0x7fa9f0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:57.920+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9f8040210 con 0x7faa08108120 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.097+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7faa08103070 con 0x7fa9f4039200 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.097+0000 7fa9ff7fe640 1 -- 172.21.15.64:0/958551798 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7faa08103070 con 0x7fa9f4039200 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa9f4039200 msgr2=0x7fa9f403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa9f4039200 0x7fa9f403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00098c0 tx=0x7fa9f0006db0 comp rx=0 tx=0).stop 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 msgr2=0x7faa0819b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0819b600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097f0 tx=0x7fa9f8030d00 comp rx=0 tx=0).stop 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 shutdown_connections 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fa9f4039200 0x7fa9f403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 --2- 172.21.15.64:0/958551798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa08108120 0x7faa0819b600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.099+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 >> 172.21.15.64:0/958551798 conn(0x7faa08101d80 msgr2=0x7faa08102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:58.698 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.100+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 shutdown_connections 2024-09-19T04:56:58.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:58.100+0000 7faa0c90e640 1 -- 172.21.15.64:0/958551798 wait complete. 2024-09-19T04:56:58.699 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2024-09-19T04:56:58.977 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-mon[30960]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:58.977 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-mon[30960]: [19/Sep/2024:04:56:58] ENGINE Bus STARTING 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:56:58.885+0000 7f0d09cc3640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:56:58] ENGINE Error in HTTPServer.serve 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: Traceback (most recent call last): 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._connections.run(self.expiration_interval) 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._run(expiration_interval) 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:56:58.978 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: s = self.context.wrap_socket( 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: return self.sslsocket_class._create( 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self.do_handshake() 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._sslobj.do_handshake() 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:56:58.979 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: [19/Sep/2024:04:56:58] ENGINE Serving on http://172.21.15.64:8765 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: [19/Sep/2024:04:56:58] ENGINE Serving on https://172.21.15.64:7150 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: [19/Sep/2024:04:56:58] ENGINE Bus STARTED 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: [19/Sep/2024:04:56:58] ENGINE Error in HTTPServer.serve 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:57:00.025 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:57:00.026 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: Generating ssh key... 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: Generating public/private rsa key pair. 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: Your identification has been saved in /tmp/tmpw0q0l5x9/key 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: Your public key has been saved in /tmp/tmpw0q0l5x9/key.pub 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: The key fingerprint is: 2024-09-19T04:57:00.027 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: SHA256:RNByWdKy7ZyrlAzdKm0GjdN3CbjrUumMVp0hIFY30ZM ceph-5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: The key's randomart image is: 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: +---[RSA 3072]----+ 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | .oo**.. | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | o oo=+E | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | . . +o+.. | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | *ooo. . | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | = S*o+o | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | *+=*. | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | .*X . | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | +*o . | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: | . .o. | 2024-09-19T04:57:00.028 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:56:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: +----[SHA256]-----+ 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19b5bc9640 1 Processor -- start 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19b5bc9640 1 -- start start 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19b5bc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19b5bc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01080b0 con 0x7f19b01076e0 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43282/0 (socket says 172.21.15.64:43282) 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19af7fe640 1 -- 172.21.15.64:0/2104769965 learned_addr learned my addr 172.21.15.64:0/2104769965 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.241+0000 7f19af7fe640 1 -- 172.21.15.64:0/2104769965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b0108840 con 0x7f19b01076e0 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.242+0000 7f19af7fe640 1 --2- 172.21.15.64:0/2104769965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b0107ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009820 tx=0x7f19a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66080cb59d0c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.316 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.242+0000 7f19ae7fc640 1 -- 172.21.15.64:0/2104769965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19a0037070 con 0x7f19b01076e0 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.242+0000 7f19ae7fc640 1 -- 172.21.15.64:0/2104769965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f19a00328d0 con 0x7f19b01076e0 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.242+0000 7f19ae7fc640 1 -- 172.21.15.64:0/2104769965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19a00307d0 con 0x7f19b01076e0 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2104769965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 msgr2=0x7f19b0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 --2- 172.21.15.64:0/2104769965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b0107ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009820 tx=0x7f19a002a270 comp rx=0 tx=0).stop 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2104769965 shutdown_connections 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 --2- 172.21.15.64:0/2104769965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b0107ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2104769965 >> 172.21.15.64:0/2104769965 conn(0x7f19b0102ed0 msgr2=0x7f19b01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2104769965 shutdown_connections 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2104769965 wait complete. 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.243+0000 7f19b5bc9640 1 Processor -- start 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19b5bc9640 1 -- start start 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19b5bc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.317 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19b5bc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f19b010bbf0 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43290/0 (socket says 172.21.15.64:43290) 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19af7fe640 1 -- 172.21.15.64:0/2539220696 learned_addr learned my addr 172.21.15.64:0/2539220696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19af7fe640 1 -- 172.21.15.64:0/2539220696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b007e090 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19af7fe640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b007db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009940 tx=0x7f19a0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19a0042070 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f19a0030c80 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.244+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b007c280 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.245+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19a0037040 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.245+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b007c7c0 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.245+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f19a0045070 con 0x7f19b01076e0 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.246+0000 7f19acff9640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f19980391b0 0x7f199803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.318 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.246+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b0107b60 con 0x7f19b01076e0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.246+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f19a006c450 con 0x7f19b01076e0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.246+0000 7f19aeffd640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f19980391b0 0x7f199803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.246+0000 7f19aeffd640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f19980391b0 0x7f199803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f19a40098c0 tx=0x7f19a4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.251+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19a0045360 con 0x7f19b01076e0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.427+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f19b0104130 con 0x7f19980391b0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.672+0000 7f19acff9640 1 -- 172.21.15.64:0/2539220696 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19b0104130 con 0x7f19980391b0 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f19980391b0 msgr2=0x7f199803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f19980391b0 0x7f199803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f19a40098c0 tx=0x7f19a4006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 msgr2=0x7f19b007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b007db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009940 tx=0x7f19a0032e10 comp rx=0 tx=0).stop 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 shutdown_connections 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f19980391b0 0x7f199803b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.319 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 --2- 172.21.15.64:0/2539220696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19b01076e0 0x7f19b007db50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 >> 172.21.15.64:0/2539220696 conn(0x7f19b0102ed0 msgr2=0x7f19b0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 shutdown_connections 2024-09-19T04:57:00.320 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:56:59.674+0000 7f19b5bc9640 1 -- 172.21.15.64:0/2539220696 wait complete. 2024-09-19T04:57:01.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:01.473 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.706+0000 7f2389a46640 1 Processor -- start 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2389a46640 1 -- start start 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2389a46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f2384105d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2389a46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2384106310 con 0x7f2384105940 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f2384105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f2384105d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43300/0 (socket says 172.21.15.64:43300) 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2382ffd640 1 -- 172.21.15.64:0/1514913406 learned_addr learned my addr 172.21.15.64:0/1514913406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.707+0000 7f2382ffd640 1 -- 172.21.15.64:0/1514913406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2384108920 con 0x7f2384105940 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.708+0000 7f2382ffd640 1 --2- 172.21.15.64:0/1514913406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f2384105d40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f237c009e80 tx=0x7f237c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949ff5c0d10130c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.708+0000 7f2381ffb640 1 -- 172.21.15.64:0/1514913406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f237c036070 con 0x7f2384105940 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.708+0000 7f2381ffb640 1 -- 172.21.15.64:0/1514913406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f237c031890 con 0x7f2384105940 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.708+0000 7f2381ffb640 1 -- 172.21.15.64:0/1514913406 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f237c007980 con 0x7f2384105940 2024-09-19T04:57:01.474 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 -- 172.21.15.64:0/1514913406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 msgr2=0x7f2384105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 --2- 172.21.15.64:0/1514913406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f2384105d40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f237c009e80 tx=0x7f237c02a7a0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 -- 172.21.15.64:0/1514913406 shutdown_connections 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 --2- 172.21.15.64:0/1514913406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f2384105d40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 -- 172.21.15.64:0/1514913406 >> 172.21.15.64:0/1514913406 conn(0x7f2384101490 msgr2=0x7f23841038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 -- 172.21.15.64:0/1514913406 shutdown_connections 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 -- 172.21.15.64:0/1514913406 wait complete. 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.709+0000 7f2389a46640 1 Processor -- start 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2389a46640 1 -- start start 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2389a46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f23841081a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2389a46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f238410bb50 con 0x7f2384105940 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f23841081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.475 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f23841081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43310/0 (socket says 172.21.15.64:43310) 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2382ffd640 1 -- 172.21.15.64:0/1416760840 learned_addr learned my addr 172.21.15.64:0/1416760840 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2382ffd640 1 -- 172.21.15.64:0/1416760840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23841086e0 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2382ffd640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f23841081a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f237c007bb0 tx=0x7f237c031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f237c041020 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f237c03c900 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f237c036040 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.710+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23841068d0 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.711+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2384106e10 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.711+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f237c044020 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.712+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2384105dc0 con 0x7f2384105940 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.712+0000 7f2388a44640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f23700391b0 0x7f237003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:01.476 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.712+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f237c06d770 con 0x7f2384105940 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.712+0000 7f23827fc640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f23700391b0 0x7f237003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.712+0000 7f23827fc640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f23700391b0 0x7f237003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f236c0098c0 tx=0x7f236c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.716+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f237c04b020 con 0x7f2384105940 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.895+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f23841076c0 con 0x7f23700391b0 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.896+0000 7f2388a44640 1 -- 172.21.15.64:0/1416760840 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f23841076c0 con 0x7f23700391b0 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f23700391b0 msgr2=0x7f237003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f23700391b0 0x7f237003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f236c0098c0 tx=0x7f236c006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 msgr2=0x7f23841081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f23841081a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f237c007bb0 tx=0x7f237c031c70 comp rx=0 tx=0).stop 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 shutdown_connections 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f23700391b0 0x7f237003b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 --2- 172.21.15.64:0/1416760840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2384105940 0x7f23841081a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 >> 172.21.15.64:0/1416760840 conn(0x7f2384101490 msgr2=0x7f2384102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:01.478 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 shutdown_connections 2024-09-19T04:57:01.478 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:00.898+0000 7f2389a46640 1 -- 172.21.15.64:0/1416760840 wait complete. 2024-09-19T04:57:01.478 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:57:01.478 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T04:57:01.478 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2024-09-19T04:57:02.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:01 smithi064 ceph-mon[30960]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:03.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:02 smithi064 ceph-mon[30960]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:04.055 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:03 smithi064 ceph-mon[30960]: Deploying cephadm binary to smithi064 2024-09-19T04:57:06.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2024-09-19T04:57:06.546 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.917+0000 7fbdeb817640 1 Processor -- start 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbdeb817640 1 -- start start 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbdeb817640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbdeb817640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4106f40 con 0x7fbde4106570 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbde95b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbde95b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43326/0 (socket says 172.21.15.64:43326) 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbde95b3640 1 -- 172.21.15.64:0/1820105615 learned_addr learned my addr 172.21.15.64:0/1820105615 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.918+0000 7fbde95b3640 1 -- 172.21.15.64:0/1820105615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde4107720 con 0x7fbde4106570 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.919+0000 7fbde95b3640 1 --2- 172.21.15.64:0/1820105615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4106970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009820 tx=0x7fbde002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30e028c4d5e1b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.919+0000 7fbddbfff640 1 -- 172.21.15.64:0/1820105615 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde0037070 con 0x7fbde4106570 2024-09-19T04:57:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.919+0000 7fbddbfff640 1 -- 172.21.15.64:0/1820105615 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbde00328d0 con 0x7fbde4106570 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.919+0000 7fbddbfff640 1 -- 172.21.15.64:0/1820105615 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde00307d0 con 0x7fbde4106570 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 -- 172.21.15.64:0/1820105615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 msgr2=0x7fbde4106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 --2- 172.21.15.64:0/1820105615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4106970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009820 tx=0x7fbde002a270 comp rx=0 tx=0).stop 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 -- 172.21.15.64:0/1820105615 shutdown_connections 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 --2- 172.21.15.64:0/1820105615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4106970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 -- 172.21.15.64:0/1820105615 >> 172.21.15.64:0/1820105615 conn(0x7fbde4101d80 msgr2=0x7fbde41041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 -- 172.21.15.64:0/1820105615 shutdown_connections 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.920+0000 7fbdeb817640 1 -- 172.21.15.64:0/1820105615 wait complete. 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbdeb817640 1 Processor -- start 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbdeb817640 1 -- start start 2024-09-19T04:57:06.548 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbdeb817640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4191750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbdeb817640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbde410aad0 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbde95b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbde95b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43336/0 (socket says 172.21.15.64:43336) 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbde95b3640 1 -- 172.21.15.64:0/2876351535 learned_addr learned my addr 172.21.15.64:0/2876351535 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.921+0000 7fbde95b3640 1 -- 172.21.15.64:0/2876351535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde4191c90 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.922+0000 7fbde95b3640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4191750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbde00097f0 tx=0x7fbde0030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.922+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde0037040 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.922+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbde0032bb0 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.922+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbde003a390 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.922+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde418fe80 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.922+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde41903c0 con 0x7fbde4106570 2024-09-19T04:57:06.549 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.923+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fbde003b3f0 con 0x7fbde4106570 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.923+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde410ac50 con 0x7fbde4106570 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.923+0000 7fbdda7fc640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fbdd00391b0 0x7fbdd003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.923+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbde006c400 con 0x7fbde4106570 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.923+0000 7fbde8db2640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fbdd00391b0 0x7fbdd003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.924+0000 7fbde8db2640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fbdd00391b0 0x7fbdd003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40098c0 tx=0x7fbdd4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:01.928+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbde0040210 con 0x7fbde4106570 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:02.109+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}) -- 0x7fbde40008d0 con 0x7fbdd00391b0 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.982+0000 7fbdda7fc640 1 -- 172.21.15.64:0/2876351535 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbde40008d0 con 0x7fbdd00391b0 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fbdd00391b0 msgr2=0x7fbdd003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fbdd00391b0 0x7fbdd003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40098c0 tx=0x7fbdd4006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 msgr2=0x7fbde4191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4191750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbde00097f0 tx=0x7fbde0030d00 comp rx=0 tx=0).stop 2024-09-19T04:57:06.550 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 shutdown_connections 2024-09-19T04:57:06.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fbdd00391b0 0x7fbdd003b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:06.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 --2- 172.21.15.64:0/2876351535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbde4106570 0x7fbde4191750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:06.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 >> 172.21.15.64:0/2876351535 conn(0x7fbde4101d80 msgr2=0x7fbde4102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:06.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 shutdown_connections 2024-09-19T04:57:06.551 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:05.985+0000 7fbdeb817640 1 -- 172.21.15.64:0/2876351535 wait complete. 2024-09-19T04:57:06.551 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2024-09-19T04:57:07.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:06 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:07.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:06 smithi064 ceph-mon[30960]: Added host smithi064 2024-09-19T04:57:07.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:06 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:07.941 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.006+0000 7f7d0f8de640 1 Processor -- start 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.006+0000 7f7d0f8de640 1 -- start start 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.006+0000 7f7d0f8de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d08107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.006+0000 7f7d0f8de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d081080b0 con 0x7f7d081076e0 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.007+0000 7f7d0d67a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d08107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.007+0000 7f7d0d67a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d08107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37042/0 (socket says 172.21.15.64:37042) 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.007+0000 7f7d0d67a640 1 -- 172.21.15.64:0/140352415 learned_addr learned my addr 172.21.15.64:0/140352415 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.007+0000 7f7d0d67a640 1 -- 172.21.15.64:0/140352415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d08108840 con 0x7f7d081076e0 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0d67a640 1 --2- 172.21.15.64:0/140352415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d08107ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009820 tx=0x7f7d0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5244d49d80b36903 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7cfffff640 1 -- 172.21.15.64:0/140352415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d04037070 con 0x7f7d081076e0 2024-09-19T04:57:07.942 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7cfffff640 1 -- 172.21.15.64:0/140352415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7d040328d0 con 0x7f7d081076e0 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7cfffff640 1 -- 172.21.15.64:0/140352415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d040307d0 con 0x7f7d081076e0 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 -- 172.21.15.64:0/140352415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 msgr2=0x7f7d08107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 --2- 172.21.15.64:0/140352415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d08107ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009820 tx=0x7f7d0402a270 comp rx=0 tx=0).stop 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 -- 172.21.15.64:0/140352415 shutdown_connections 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 --2- 172.21.15.64:0/140352415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d08107ae0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 -- 172.21.15.64:0/140352415 >> 172.21.15.64:0/140352415 conn(0x7f7d08102ed0 msgr2=0x7f7d081052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 -- 172.21.15.64:0/140352415 shutdown_connections 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.008+0000 7f7d0f8de640 1 -- 172.21.15.64:0/140352415 wait complete. 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0f8de640 1 Processor -- start 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0f8de640 1 -- start start 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0f8de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d0807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0f8de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810bbf0 con 0x7f7d081076e0 2024-09-19T04:57:07.943 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0d67a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d0807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0d67a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d0807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37048/0 (socket says 172.21.15.64:37048) 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.009+0000 7f7d0d67a640 1 -- 172.21.15.64:0/2113575267 learned_addr learned my addr 172.21.15.64:0/2113575267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7d0d67a640 1 -- 172.21.15.64:0/2113575267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0807e090 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7d0d67a640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d0807db50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009940 tx=0x7f7d04032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d04042070 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7d04030c80 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0807c280 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d04037040 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.010+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0807c7c0 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.011+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7d04045070 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.011+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0810bd70 con 0x7f7d081076e0 2024-09-19T04:57:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.011+0000 7f7cfe7fc640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f7cf4039200 0x7f7cf403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.011+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7d0406c2f0 con 0x7f7d081076e0 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.011+0000 7f7d0ce79640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f7cf4039200 0x7f7cf403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.012+0000 7f7d0ce79640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f7cf4039200 0x7f7cf403b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80098c0 tx=0x7f7cf8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.016+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d04045360 con 0x7f7d081076e0 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.209+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7d0807d050 con 0x7f7cf4039200 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.214+0000 7f7cfe7fc640 1 -- 172.21.15.64:0/2113575267 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7d0807d050 con 0x7f7cf4039200 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f7cf4039200 msgr2=0x7f7cf403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f7cf4039200 0x7f7cf403b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80098c0 tx=0x7f7cf8006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 msgr2=0x7f7d0807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:07.945 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d0807db50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009940 tx=0x7f7d04032e10 comp rx=0 tx=0).stop 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 shutdown_connections 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f7cf4039200 0x7f7cf403b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 --2- 172.21.15.64:0/2113575267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d081076e0 0x7f7d0807db50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 >> 172.21.15.64:0/2113575267 conn(0x7f7d08102ed0 msgr2=0x7f7d08103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 shutdown_connections 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:07.217+0000 7f7d0f8de640 1 -- 172.21.15.64:0/2113575267 wait complete. 2024-09-19T04:57:07.946 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2024-09-19T04:57:08.494 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:08 smithi064 ceph-mon[30960]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:08.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:08 smithi064 ceph-mon[30960]: Saving service mon spec with placement count:5 2024-09-19T04:57:08.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:08 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.345+0000 7f21a5432640 1 Processor -- start 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.345+0000 7f21a5432640 1 -- start start 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.345+0000 7f21a5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a010a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.345+0000 7f21a5432640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21a007be00 con 0x7f21a0108120 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.345+0000 7f219effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a010a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.346+0000 7f219effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a010a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37064/0 (socket says 172.21.15.64:37064) 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.346+0000 7f219effd640 1 -- 172.21.15.64:0/1510220306 learned_addr learned my addr 172.21.15.64:0/1510220306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.346+0000 7f219effd640 1 -- 172.21.15.64:0/1510220306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a010aa50 con 0x7f21a0108120 2024-09-19T04:57:09.266 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f219effd640 1 --2- 172.21.15.64:0/1510220306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a010a510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2198009820 tx=0x7f219802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41564ad18b3c1ead server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f219dffb640 1 -- 172.21.15.64:0/1510220306 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2198037070 con 0x7f21a0108120 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f219dffb640 1 -- 172.21.15.64:0/1510220306 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f21980328d0 con 0x7f21a0108120 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f219dffb640 1 -- 172.21.15.64:0/1510220306 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21980307d0 con 0x7f21a0108120 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f21a5432640 1 -- 172.21.15.64:0/1510220306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 msgr2=0x7f21a010a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f21a5432640 1 --2- 172.21.15.64:0/1510220306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a010a510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2198009820 tx=0x7f219802a270 comp rx=0 tx=0).stop 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f21a5432640 1 -- 172.21.15.64:0/1510220306 shutdown_connections 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f21a5432640 1 --2- 172.21.15.64:0/1510220306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a010a510 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.347+0000 7f21a5432640 1 -- 172.21.15.64:0/1510220306 >> 172.21.15.64:0/1510220306 conn(0x7f21a0101d80 msgr2=0x7f21a01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.348+0000 7f21a5432640 1 -- 172.21.15.64:0/1510220306 shutdown_connections 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.348+0000 7f21a5432640 1 -- 172.21.15.64:0/1510220306 wait complete. 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.348+0000 7f21a5432640 1 Processor -- start 2024-09-19T04:57:09.267 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.348+0000 7f21a5432640 1 -- start start 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f21a5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f21a5432640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010b9d0 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f219effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f219effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a019b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37074/0 (socket says 172.21.15.64:37074) 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f219effd640 1 -- 172.21.15.64:0/263249425 learned_addr learned my addr 172.21.15.64:0/263249425 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f219effd640 1 -- 172.21.15.64:0/263249425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019bb40 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f219effd640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a019b600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2198030520 tx=0x7f2198032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2198042070 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2198030cc0 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2198037040 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a019bdd0 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.349+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a019eac0 con 0x7f21a0108120 2024-09-19T04:57:09.268 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.350+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2198045070 con 0x7f21a0108120 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.350+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a0108580 con 0x7f21a0108120 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.351+0000 7f2197fff640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f218c0391b0 0x7f218c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.351+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f219806c260 con 0x7f21a0108120 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.351+0000 7f219e7fc640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f218c0391b0 0x7f218c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.351+0000 7f219e7fc640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f218c0391b0 0x7f218c03b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21880098c0 tx=0x7f2188006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.355+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f219803d7b0 con 0x7f21a0108120 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.545+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f21a0103070 con 0x7f218c0391b0 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.550+0000 7f2197fff640 1 -- 172.21.15.64:0/263249425 <== mgr.14118 v2:172.21.15.64:6800/2584640926 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f21a0103070 con 0x7f218c0391b0 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f218c0391b0 msgr2=0x7f218c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f218c0391b0 0x7f218c03b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21880098c0 tx=0x7f2188006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 msgr2=0x7f21a019b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a019b600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2198030520 tx=0x7f2198032dd0 comp rx=0 tx=0).stop 2024-09-19T04:57:09.269 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 shutdown_connections 2024-09-19T04:57:09.270 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f218c0391b0 0x7f218c03b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:09.270 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 --2- 172.21.15.64:0/263249425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21a0108120 0x7f21a019b600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:09.270 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 >> 172.21.15.64:0/263249425 conn(0x7f21a0101d80 msgr2=0x7f21a0102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:09.270 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 shutdown_connections 2024-09-19T04:57:09.270 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:08.552+0000 7f21a5432640 1 -- 172.21.15.64:0/263249425 wait complete. 2024-09-19T04:57:09.734 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:09 smithi064 ceph-mon[30960]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:09.735 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:09 smithi064 ceph-mon[30960]: Saving service mgr spec with placement count:2 2024-09-19T04:57:09.735 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:09 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff685f5640 1 Processor -- start 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff685f5640 1 -- start start 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff685f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff6010a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff685f5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff6007be40 con 0x7eff601080e0 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff66391640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff6010a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff66391640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff6010a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37090/0 (socket says 172.21.15.64:37090) 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.730+0000 7eff66391640 1 -- 172.21.15.64:0/3637382655 learned_addr learned my addr 172.21.15.64:0/3637382655 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.731+0000 7eff66391640 1 -- 172.21.15.64:0/3637382655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6010aa10 con 0x7eff601080e0 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.732+0000 7eff66391640 1 --2- 172.21.15.64:0/3637382655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff6010a4d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009820 tx=0x7eff5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269afa4c76fc12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.732+0000 7eff6538f640 1 -- 172.21.15.64:0/3637382655 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c037070 con 0x7eff601080e0 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.732+0000 7eff6538f640 1 -- 172.21.15.64:0/3637382655 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7eff5c0328d0 con 0x7eff601080e0 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.732+0000 7eff6538f640 1 -- 172.21.15.64:0/3637382655 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c0307d0 con 0x7eff601080e0 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 -- 172.21.15.64:0/3637382655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 msgr2=0x7eff6010a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 --2- 172.21.15.64:0/3637382655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff6010a4d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009820 tx=0x7eff5c02a270 comp rx=0 tx=0).stop 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 -- 172.21.15.64:0/3637382655 shutdown_connections 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 --2- 172.21.15.64:0/3637382655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff6010a4d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 -- 172.21.15.64:0/3637382655 >> 172.21.15.64:0/3637382655 conn(0x7eff60101d80 msgr2=0x7eff601041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 -- 172.21.15.64:0/3637382655 shutdown_connections 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.733+0000 7eff685f5640 1 -- 172.21.15.64:0/3637382655 wait complete. 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff685f5640 1 Processor -- start 2024-09-19T04:57:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff685f5640 1 -- start start 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff685f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff60107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff685f5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010b9f0 con 0x7eff601080e0 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff66391640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff60107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff66391640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff60107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37100/0 (socket says 172.21.15.64:37100) 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff66391640 1 -- 172.21.15.64:0/3578227127 learned_addr learned my addr 172.21.15.64:0/3578227127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.734+0000 7eff66391640 1 -- 172.21.15.64:0/3578227127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60107ea0 con 0x7eff601080e0 2024-09-19T04:57:10.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.735+0000 7eff66391640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff60107960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0097f0 tx=0x7eff5c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:10.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.735+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c037040 con 0x7eff601080e0 2024-09-19T04:57:10.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.735+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7eff5c03b3f0 con 0x7eff601080e0 2024-09-19T04:57:10.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.735+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c03a520 con 0x7eff601080e0 2024-09-19T04:57:10.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.735+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff60106090 con 0x7eff601080e0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.735+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601065d0 con 0x7eff601080e0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.736+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7eff5c03a710 con 0x7eff601080e0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.736+0000 7eff577fe640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7eff4c0391b0 0x7eff4c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.736+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff5c06c490 con 0x7eff601080e0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.736+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff60108540 con 0x7eff601080e0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.736+0000 7eff65b90640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7eff4c0391b0 0x7eff4c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.737+0000 7eff65b90640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7eff4c0391b0 0x7eff4c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff500098c0 tx=0x7eff50006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:10.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.741+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff5c032820 con 0x7eff601080e0 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.918+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7eff60106c90 con 0x7eff601080e0 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.920+0000 7eff577fe640 1 -- 172.21.15.64:0/3578227127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7eff5c0319b0 con 0x7eff601080e0 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7eff4c0391b0 msgr2=0x7eff4c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7eff4c0391b0 0x7eff4c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff500098c0 tx=0x7eff50006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 msgr2=0x7eff60107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff60107960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0097f0 tx=0x7eff5c030d00 comp rx=0 tx=0).stop 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 shutdown_connections 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7eff4c0391b0 0x7eff4c03b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 --2- 172.21.15.64:0/3578227127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff601080e0 0x7eff60107960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 >> 172.21.15.64:0/3578227127 conn(0x7eff60101d80 msgr2=0x7eff60102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 shutdown_connections 2024-09-19T04:57:10.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:09.923+0000 7eff685f5640 1 -- 172.21.15.64:0/3578227127 wait complete. 2024-09-19T04:57:11.285 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:10 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3578227127' entity='client.admin' 2024-09-19T04:57:11.285 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:10 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:11.857 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac3cf6640 1 Processor -- start 2024-09-19T04:57:11.857 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac3cf6640 1 -- start start 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac3cf6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac3cf6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc106f40 con 0x7fdabc106570 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac1a92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac1a92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37112/0 (socket says 172.21.15.64:37112) 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.094+0000 7fdac1a92640 1 -- 172.21.15.64:0/3666104438 learned_addr learned my addr 172.21.15.64:0/3666104438 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.095+0000 7fdac1a92640 1 -- 172.21.15.64:0/3666104438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc107720 con 0x7fdabc106570 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.095+0000 7fdac1a92640 1 --2- 172.21.15.64:0/3666104438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc106970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009820 tx=0x7fdab002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3fbbad0578257d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.095+0000 7fdac0a90640 1 -- 172.21.15.64:0/3666104438 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0037070 con 0x7fdabc106570 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac0a90640 1 -- 172.21.15.64:0/3666104438 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab00328f0 con 0x7fdabc106570 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac0a90640 1 -- 172.21.15.64:0/3666104438 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0030740 con 0x7fdabc106570 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3666104438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 msgr2=0x7fdabc106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:11.858 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 --2- 172.21.15.64:0/3666104438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc106970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009820 tx=0x7fdab002a270 comp rx=0 tx=0).stop 2024-09-19T04:57:11.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3666104438 shutdown_connections 2024-09-19T04:57:11.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 --2- 172.21.15.64:0/3666104438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc106970 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:11.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3666104438 >> 172.21.15.64:0/3666104438 conn(0x7fdabc101d80 msgr2=0x7fdabc1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:11.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3666104438 shutdown_connections 2024-09-19T04:57:11.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.096+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3666104438 wait complete. 2024-09-19T04:57:11.859 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac3cf6640 1 Processor -- start 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac3cf6640 1 -- start start 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac3cf6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc191750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac3cf6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc10aad0 con 0x7fdabc106570 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac1a92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac1a92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37126/0 (socket says 172.21.15.64:37126) 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac1a92640 1 -- 172.21.15.64:0/3739953127 learned_addr learned my addr 172.21.15.64:0/3739953127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.097+0000 7fdac1a92640 1 -- 172.21.15.64:0/3739953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc191c90 con 0x7fdabc106570 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.098+0000 7fdac1a92640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc191750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdab00097f0 tx=0x7fdab0030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.098+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0042070 con 0x7fdabc106570 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.098+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab0030e80 con 0x7fdabc106570 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.098+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc18fe80 con 0x7fdabc106570 2024-09-19T04:57:11.860 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.098+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0037040 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.098+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc1903c0 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.099+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdab0045070 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.099+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fdaa80391b0 0x7fdaa803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.099+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdabc10ac50 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.099+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdab006c300 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.099+0000 7fdac1291640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fdaa80391b0 0x7fdaa803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.099+0000 7fdac1291640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fdaa80391b0 0x7fdaa803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098c0 tx=0x7fdab8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.104+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdab004b170 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.283+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdabc1954f0 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.288+0000 7fdaaeffd640 1 -- 172.21.15.64:0/3739953127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdab002fd90 con 0x7fdabc106570 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fdaa80391b0 msgr2=0x7fdaa803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fdaa80391b0 0x7fdaa803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098c0 tx=0x7fdab8006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 msgr2=0x7fdabc191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc191750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdab00097f0 tx=0x7fdab0030c70 comp rx=0 tx=0).stop 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 shutdown_connections 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fdaa80391b0 0x7fdaa803b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 --2- 172.21.15.64:0/3739953127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdabc106570 0x7fdabc191750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 >> 172.21.15.64:0/3739953127 conn(0x7fdabc101d80 msgr2=0x7fdabc102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 shutdown_connections 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:11.290+0000 7fdac3cf6640 1 -- 172.21.15.64:0/3739953127 wait complete. 2024-09-19T04:57:11.862 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2024-09-19T04:57:12.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:12 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:12.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:12 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3739953127' entity='client.admin' 2024-09-19T04:57:13.452 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3028885079' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T04:57:13.452 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-mon[30960]: from='mgr.14118 172.21.15.64:0/287045077' entity='mgr.x' 2024-09-19T04:57:13.452 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ignoring --setuser ceph since I am not root 2024-09-19T04:57:13.452 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ignoring --setgroup ceph since I am not root 2024-09-19T04:57:13.452 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:13.345+0000 7f85a8899040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:57:13.452 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:13.404+0000 7f85a8899040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:57:13.792 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:13.564+0000 7f85a8899040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:57:13.793 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:13.637+0000 7f85a8899040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:57:13.793 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:13.790+0000 7f85a8899040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:57:13.834 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.259+0000 7f89a7d5d640 1 Processor -- start 2024-09-19T04:57:13.834 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.259+0000 7f89a7d5d640 1 -- start start 2024-09-19T04:57:13.834 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.260+0000 7f89a7d5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.260+0000 7f89a7d5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89a007bd80 con 0x7f89a01081a0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.260+0000 7f89a5af9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.260+0000 7f89a5af9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37128/0 (socket says 172.21.15.64:37128) 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.260+0000 7f89a5af9640 1 -- 172.21.15.64:0/2540850455 learned_addr learned my addr 172.21.15.64:0/2540850455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.260+0000 7f89a5af9640 1 -- 172.21.15.64:0/2540850455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a010aad0 con 0x7f89a01081a0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a5af9640 1 --2- 172.21.15.64:0/2540850455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a010a590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f899c009820 tx=0x7f899c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf87af7d4c08c46e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a4af7640 1 -- 172.21.15.64:0/2540850455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c037070 con 0x7f89a01081a0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a4af7640 1 -- 172.21.15.64:0/2540850455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f899c0328f0 con 0x7f89a01081a0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a4af7640 1 -- 172.21.15.64:0/2540850455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c030740 con 0x7f89a01081a0 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a7d5d640 1 -- 172.21.15.64:0/2540850455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 msgr2=0x7f89a010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a7d5d640 1 --2- 172.21.15.64:0/2540850455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a010a590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f899c009820 tx=0x7f899c02a270 comp rx=0 tx=0).stop 2024-09-19T04:57:13.835 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a7d5d640 1 -- 172.21.15.64:0/2540850455 shutdown_connections 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a7d5d640 1 --2- 172.21.15.64:0/2540850455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a010a590 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.261+0000 7f89a7d5d640 1 -- 172.21.15.64:0/2540850455 >> 172.21.15.64:0/2540850455 conn(0x7f89a0101d80 msgr2=0x7f89a01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.262+0000 7f89a7d5d640 1 -- 172.21.15.64:0/2540850455 shutdown_connections 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.262+0000 7f89a7d5d640 1 -- 172.21.15.64:0/2540850455 wait complete. 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.262+0000 7f89a7d5d640 1 Processor -- start 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.262+0000 7f89a7d5d640 1 -- start start 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.262+0000 7f89a7d5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.262+0000 7f89a7d5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89a010ba30 con 0x7f89a01081a0 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a5af9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a5af9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37134/0 (socket says 172.21.15.64:37134) 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a5af9640 1 -- 172.21.15.64:0/3028885079 learned_addr learned my addr 172.21.15.64:0/3028885079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a5af9640 1 -- 172.21.15.64:0/3028885079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a019bb80 con 0x7f89a01081a0 2024-09-19T04:57:13.836 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a5af9640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a019b640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f899c0097f0 tx=0x7f899c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c042070 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f899c030e80 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a019be10 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c037040 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.263+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a019eb00 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.264+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f899c045070 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.264+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a0108600 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.264+0000 7f8992ffd640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f898c0391b0 0x7f898c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.264+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f899c06c300 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.264+0000 7f89a52f8640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f898c0391b0 0x7f898c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.265+0000 7f89a52f8640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f898c0391b0 0x7f898c03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f89940098c0 tx=0x7f8994006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.269+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f899c04b170 con 0x7f89a01081a0 2024-09-19T04:57:13.837 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:12.512+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f89a019ecd0 con 0x7f89a01081a0 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.191+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f899c02fd90 con 0x7f89a01081a0 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.193+0000 7f8992ffd640 1 -- 172.21.15.64:0/3028885079 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f899c03db70 con 0x7f89a01081a0 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f898c0391b0 msgr2=0x7f898c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f898c0391b0 0x7f898c03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f89940098c0 tx=0x7f8994006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 msgr2=0x7f89a019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a019b640 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f899c0097f0 tx=0x7f899c030c70 comp rx=0 tx=0).stop 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 shutdown_connections 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f898c0391b0 0x7f898c03b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 --2- 172.21.15.64:0/3028885079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89a01081a0 0x7f89a019b640 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 >> 172.21.15.64:0/3028885079 conn(0x7f89a0101d80 msgr2=0x7f89a01029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 shutdown_connections 2024-09-19T04:57:13.839 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:13.195+0000 7f89a7d5d640 1 -- 172.21.15.64:0/3028885079 wait complete. 2024-09-19T04:57:14.093 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:13.950+0000 7f85a8899040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:57:14.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:14 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3028885079' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T04:57:14.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:14 smithi064 ceph-mon[30960]: mgrmap e9: x(active, since 18s) 2024-09-19T04:57:14.353 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:14.286+0000 7f85a8899040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:57:14.353 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:14.350+0000 7f85a8899040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:57:14.650 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:14.548+0000 7f85a8899040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:57:15.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:15 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3830403217' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:57:15.217 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:57:15.217 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T04:57:15.217 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:57:15.217 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.267+0000 7f7268a88640 1 Processor -- start 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.267+0000 7f7268a88640 1 -- start start 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.268+0000 7f7268a88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f7264107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.268+0000 7f7268a88640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72641080b0 con 0x7f72641076e0 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.268+0000 7f726259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f7264107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.268+0000 7f726259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f7264107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49092/0 (socket says 172.21.15.64:49092) 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.268+0000 7f726259c640 1 -- 172.21.15.64:0/1201775752 learned_addr learned my addr 172.21.15.64:0/1201775752 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.268+0000 7f726259c640 1 -- 172.21.15.64:0/1201775752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7264108840 con 0x7f72641076e0 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.269+0000 7f726259c640 1 --2- 172.21.15.64:0/1201775752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f7264107ae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7254009820 tx=0x7f725402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f0cf70e5c38658 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.269+0000 7f726159a640 1 -- 172.21.15.64:0/1201775752 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7254037070 con 0x7f72641076e0 2024-09-19T04:57:15.218 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.269+0000 7f726159a640 1 -- 172.21.15.64:0/1201775752 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72540328f0 con 0x7f72641076e0 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.269+0000 7f726159a640 1 -- 172.21.15.64:0/1201775752 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7254030740 con 0x7f72641076e0 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 -- 172.21.15.64:0/1201775752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 msgr2=0x7f7264107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 --2- 172.21.15.64:0/1201775752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f7264107ae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7254009820 tx=0x7f725402a270 comp rx=0 tx=0).stop 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 -- 172.21.15.64:0/1201775752 shutdown_connections 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 --2- 172.21.15.64:0/1201775752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f7264107ae0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 -- 172.21.15.64:0/1201775752 >> 172.21.15.64:0/1201775752 conn(0x7f7264102ed0 msgr2=0x7f72641052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 -- 172.21.15.64:0/1201775752 shutdown_connections 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.270+0000 7f7268a88640 1 -- 172.21.15.64:0/1201775752 wait complete. 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f7268a88640 1 Processor -- start 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f7268a88640 1 -- start start 2024-09-19T04:57:15.219 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f7268a88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f726419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f7268a88640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f726410bbf0 con 0x7f72641076e0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f726259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f726419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f726259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f726419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49096/0 (socket says 172.21.15.64:49096) 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f726259c640 1 -- 172.21.15.64:0/3830403217 learned_addr learned my addr 172.21.15.64:0/3830403217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.271+0000 7f726259c640 1 -- 172.21.15.64:0/3830403217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726419bbd0 con 0x7f72641076e0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.272+0000 7f726259c640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f726419b690 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f72540097f0 tx=0x7f7254030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.272+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7254042070 con 0x7f72641076e0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.272+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7254030e80 con 0x7f72641076e0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.272+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726419be60 con 0x7f72641076e0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.272+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7254037040 con 0x7f72641076e0 2024-09-19T04:57:15.220 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.272+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726419eb50 con 0x7f72641076e0 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.273+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f7254045070 con 0x7f72641076e0 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.273+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726410bd70 con 0x7f72641076e0 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.273+0000 7f72537fe640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 0x7f724c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.273+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f725406c320 con 0x7f72641076e0 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.273+0000 7f7261d9b640 1 -- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 msgr2=0x7f724c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/2584640926 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.273+0000 7f7261d9b640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 0x7f724c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:57:15.221 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.278+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f725404b170 con 0x7f72641076e0 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.474+0000 7f7261d9b640 1 -- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 msgr2=0x7f724c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/2584640926 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.474+0000 7f7261d9b640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 0x7f724c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.503+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f726419ed20 con 0x7f72641076e0 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.503+0000 7f72537fe640 1 -- 172.21.15.64:0/3830403217 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f725402fd90 con 0x7f72641076e0 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 msgr2=0x7f724c03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 0x7f724c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 msgr2=0x7f726419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f726419b690 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f72540097f0 tx=0x7f7254030c70 comp rx=0 tx=0).stop 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 shutdown_connections 2024-09-19T04:57:15.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7f724c039260 0x7f724c03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:15.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 --2- 172.21.15.64:0/3830403217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72641076e0 0x7f726419b690 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:15.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 >> 172.21.15.64:0/3830403217 conn(0x7f7264102ed0 msgr2=0x7f7264103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:15.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 shutdown_connections 2024-09-19T04:57:15.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:14.505+0000 7f7268a88640 1 -- 172.21.15.64:0/3830403217 wait complete. 2024-09-19T04:57:15.223 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-09-19T04:57:15.223 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2024-09-19T04:57:15.754 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:15.481+0000 7f85a8899040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:57:15.754 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:15.554+0000 7f85a8899040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:57:16.610 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.337+0000 7f85a8899040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:57:16.610 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.458+0000 7f85a8899040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:57:16.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.608+0000 7f85a8899040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:57:16.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.672+0000 7f85a8899040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:57:16.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.829+0000 7f85a8899040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:57:17.196 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.899+0000 7f85a8899040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:57:17.196 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:16.961+0000 7f85a8899040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:57:17.196 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:17.126+0000 7f85a8899040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:57:17.196 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:17.194+0000 7f85a8899040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:57:17.474 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:17.263+0000 7f85a8899040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:57:17.474 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:17.349+0000 7f85a8899040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:57:17.475 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:17.411+0000 7f85a8899040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:57:17.475 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:17.472+0000 7f85a8899040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:57:18.559 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:18.296+0000 7f85a8899040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:57:18.559 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: Active manager daemon x restarted 2024-09-19T04:57:18.559 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: Activating manager daemon x 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: mgrmap e10: x(active, starting, since 0.00336796s) 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: Manager daemon x is now available 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:57:18.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:57:20.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-mon[30960]: mgrmap e11: x(active, since 1.00659s) 2024-09-19T04:57:20.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:20.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-mon[30960]: [19/Sep/2024:04:57:20] ENGINE Bus STARTING 2024-09-19T04:57:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:57:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T04:57:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:57:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:57:20.377 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.683+0000 7fb174266640 1 Processor -- start 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.683+0000 7fb174266640 1 -- start start 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.684+0000 7fb174266640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.684+0000 7fb174266640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c106f40 con 0x7fb16c106570 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.684+0000 7fb172002640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.684+0000 7fb172002640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49112/0 (socket says 172.21.15.64:49112) 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.684+0000 7fb172002640 1 -- 172.21.15.64:0/1064632331 learned_addr learned my addr 172.21.15.64:0/1064632331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.684+0000 7fb172002640 1 -- 172.21.15.64:0/1064632331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c107720 con 0x7fb16c106570 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.685+0000 7fb172002640 1 --2- 172.21.15.64:0/1064632331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c106970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009e80 tx=0x7fb15c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847cb82e1e4f1abd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.685+0000 7fb171000640 1 -- 172.21.15.64:0/1064632331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb15c036070 con 0x7fb16c106570 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.685+0000 7fb171000640 1 -- 172.21.15.64:0/1064632331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb15c0318b0 con 0x7fb16c106570 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.685+0000 7fb171000640 1 -- 172.21.15.64:0/1064632331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb15c007930 con 0x7fb16c106570 2024-09-19T04:57:20.378 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 -- 172.21.15.64:0/1064632331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 msgr2=0x7fb16c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 --2- 172.21.15.64:0/1064632331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c106970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009e80 tx=0x7fb15c02a7a0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 -- 172.21.15.64:0/1064632331 shutdown_connections 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 --2- 172.21.15.64:0/1064632331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c106970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 -- 172.21.15.64:0/1064632331 >> 172.21.15.64:0/1064632331 conn(0x7fb16c101d80 msgr2=0x7fb16c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 -- 172.21.15.64:0/1064632331 shutdown_connections 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.686+0000 7fb174266640 1 -- 172.21.15.64:0/1064632331 wait complete. 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb174266640 1 Processor -- start 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb174266640 1 -- start start 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb174266640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb174266640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c10aad0 con 0x7fb16c106570 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb172002640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:20.379 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb172002640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49120/0 (socket says 172.21.15.64:49120) 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.687+0000 7fb172002640 1 -- 172.21.15.64:0/4189466430 learned_addr learned my addr 172.21.15.64:0/4189466430 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb172002640 1 -- 172.21.15.64:0/4189466430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c19aab0 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb172002640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c19a570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb15c02acc0 tx=0x7fb15c031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb15c036050 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb15c037040 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c19ad40 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb15c03d950 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.688+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c19da30 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.689+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fb15c044020 con 0x7fb16c106570 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.690+0000 7fb16affd640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:20.380 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.690+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 --> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb15403be30 con 0x7fb154039260 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.690+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb15c042020 con 0x7fb16c106570 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.690+0000 7fb171801640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 msgr2=0x7fb15403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/2584640926 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.690+0000 7fb171801640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.891+0000 7fb171801640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 msgr2=0x7fb15403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/2584640926 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:15.891+0000 7fb171801640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:16.292+0000 7fb171801640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 msgr2=0x7fb15403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/2584640926 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:16.292+0000 7fb171801640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:17.093+0000 7fb171801640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 msgr2=0x7fb15403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/2584640926 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:17.093+0000 7fb171801640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:18.300+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fb15c06ba20 con 0x7fb16c106570 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:18.300+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 msgr2=0x7fb15403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:18.300+0000 7fb16affd640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.381 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.302+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fb15c04bcc0 con 0x7fb16c106570 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.302+0000 7fb16affd640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb15403cd20 0x7fb15403f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.302+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb15c066090 con 0x7fb15403cd20 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.302+0000 7fb171801640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb15403cd20 0x7fb15403f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.303+0000 7fb171801640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb15403cd20 0x7fb15403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb160003980 tx=0x7fb1600092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.304+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb15c066090 con 0x7fb15403cd20 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb16c10ac50 con 0x7fb15403cd20 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb16affd640 1 -- 172.21.15.64:0/4189466430 <== mgr.14150 v2:172.21.15.64:6800/3907259856 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb16c10ac50 con 0x7fb15403cd20 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb15403cd20 msgr2=0x7fb15403f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb15403cd20 0x7fb15403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb160003980 tx=0x7fb1600092b0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 msgr2=0x7fb16c19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c19a570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb15c02acc0 tx=0x7fb15c031c90 comp rx=0 tx=0).stop 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 shutdown_connections 2024-09-19T04:57:20.382 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb15403cd20 0x7fb15403f110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:6800/2584640926,v1:172.21.15.64:6801/2584640926] conn(0x7fb154039260 0x7fb15403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 --2- 172.21.15.64:0/4189466430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb16c106570 0x7fb16c19a570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.311+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 >> 172.21.15.64:0/4189466430 conn(0x7fb16c101d80 msgr2=0x7fb16c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.312+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 shutdown_connections 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:19.312+0000 7fb174266640 1 -- 172.21.15.64:0/4189466430 wait complete. 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2024-09-19T04:57:20.383 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:20.410+0000 7f850e362640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:57:20] ENGINE Error in HTTPServer.serve 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: Traceback (most recent call last): 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._connections.run(self.expiration_interval) 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._run(expiration_interval) 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:57:20.651 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: s = self.context.wrap_socket( 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: return self.sslsocket_class._create( 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self.do_handshake() 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._sslobj.do_handshake() 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:57:20.652 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:57:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f22114be640 1 Processor -- start 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f22114be640 1 -- start start 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f22114be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c109ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f22114be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10a270 con 0x7f220c1098a0 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f220affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c109ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f220affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c109ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49200/0 (socket says 172.21.15.64:49200) 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f220affd640 1 -- 172.21.15.64:0/3176684994 learned_addr learned my addr 172.21.15.64:0/3176684994 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.809+0000 7f220affd640 1 -- 172.21.15.64:0/3176684994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c10aa20 con 0x7f220c1098a0 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.810+0000 7f220affd640 1 --2- 172.21.15.64:0/3176684994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c109ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009820 tx=0x7f21fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e2c60f4ae78508 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.810+0000 7f220a7fc640 1 -- 172.21.15.64:0/3176684994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21fc037070 con 0x7f220c1098a0 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.810+0000 7f220a7fc640 1 -- 172.21.15.64:0/3176684994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21fc0328f0 con 0x7f220c1098a0 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.810+0000 7f220a7fc640 1 -- 172.21.15.64:0/3176684994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21fc030740 con 0x7f220c1098a0 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 -- 172.21.15.64:0/3176684994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 msgr2=0x7f220c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 --2- 172.21.15.64:0/3176684994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c109ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009820 tx=0x7f21fc02a270 comp rx=0 tx=0).stop 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 -- 172.21.15.64:0/3176684994 shutdown_connections 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 --2- 172.21.15.64:0/3176684994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c109ca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 -- 172.21.15.64:0/3176684994 >> 172.21.15.64:0/3176684994 conn(0x7f220c07cf20 msgr2=0x7f220c07d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 -- 172.21.15.64:0/3176684994 shutdown_connections 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 -- 172.21.15.64:0/3176684994 wait complete. 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 Processor -- start 2024-09-19T04:57:21.684 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 -- start start 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.811+0000 7f22114be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f22114be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10ddd0 con 0x7f220c1098a0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f220affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f220affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49208/0 (socket says 172.21.15.64:49208) 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f220affd640 1 -- 172.21.15.64:0/3549152020 learned_addr learned my addr 172.21.15.64:0/3549152020 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f220affd640 1 -- 172.21.15.64:0/3549152020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c19dd50 con 0x7f220c1098a0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f220affd640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c19d810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0097f0 tx=0x7f21fc030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21fc042070 con 0x7f220c1098a0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21fc030d80 con 0x7f220c1098a0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c19dfe0 con 0x7f220c1098a0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21fc037040 con 0x7f220c1098a0 2024-09-19T04:57:21.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.812+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a0cd0 con 0x7f220c1098a0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.813+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f21fc045070 con 0x7f220c1098a0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.813+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c10df50 con 0x7f220c1098a0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.813+0000 7f2208ff9640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f21f8039260 0x7f21f803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.814+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f21fc06c4c0 con 0x7f220c1098a0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.814+0000 7f2203fff640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f21f8039260 0x7f21f803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.814+0000 7f2203fff640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f21f8039260 0x7f21f803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21f40098c0 tx=0x7f21f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.819+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21fc045360 con 0x7f220c1098a0 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:20.998+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f220c107c40 con 0x7f21f8039260 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.102+0000 7f2208ff9640 1 -- 172.21.15.64:0/3549152020 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f220c107c40 con 0x7f21f8039260 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f21f8039260 msgr2=0x7f21f803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f21f8039260 0x7f21f803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21f40098c0 tx=0x7f21f4006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 msgr2=0x7f220c19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:21.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c19d810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0097f0 tx=0x7f21fc030c70 comp rx=0 tx=0).stop 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 shutdown_connections 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f21f8039260 0x7f21f803b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 --2- 172.21.15.64:0/3549152020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c1098a0 0x7f220c19d810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 >> 172.21.15.64:0/3549152020 conn(0x7f220c07cf20 msgr2=0x7f220c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 shutdown_connections 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:21.104+0000 7f22114be640 1 -- 172.21.15.64:0/3549152020 wait complete. 2024-09-19T04:57:21.687 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2024-09-19T04:57:21.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: [19/Sep/2024:04:57:20] ENGINE Serving on http://172.21.15.64:8765 2024-09-19T04:57:21.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: [19/Sep/2024:04:57:20] ENGINE Serving on https://172.21.15.64:7150 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: [19/Sep/2024:04:57:20] ENGINE Bus STARTED 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: [19/Sep/2024:04:57:20] ENGINE Error in HTTPServer.serve 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:57:21.802 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: mgrmap e12: x(active, since 2s) 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:21.803 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:23.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:23.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Y0mrpiU.k3S4O8AdCdTFu.gkGtG/73gxKodaQPf8Xp9PUaKmAfFG6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726721842, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.120+0000 7f7f05ad0640 1 Processor -- start 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.120+0000 7f7f05ad0640 1 -- start start 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.121+0000 7f7f05ad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f00106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.121+0000 7f7f05ad0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00106f40 con 0x7f7f00106570 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.121+0000 7f7eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f00106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:23.468 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.121+0000 7f7eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f00106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49212/0 (socket says 172.21.15.64:49212) 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.121+0000 7f7eff7fe640 1 -- 172.21.15.64:0/777497513 learned_addr learned my addr 172.21.15.64:0/777497513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.121+0000 7f7eff7fe640 1 -- 172.21.15.64:0/777497513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f00107720 con 0x7f7f00106570 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.122+0000 7f7eff7fe640 1 --2- 172.21.15.64:0/777497513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f00106970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009820 tx=0x7f7ef002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8ff6f8a300337f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.122+0000 7f7efe7fc640 1 -- 172.21.15.64:0/777497513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0037070 con 0x7f7f00106570 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.122+0000 7f7efe7fc640 1 -- 172.21.15.64:0/777497513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef00328f0 con 0x7f7f00106570 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.122+0000 7f7efe7fc640 1 -- 172.21.15.64:0/777497513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0030740 con 0x7f7f00106570 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 -- 172.21.15.64:0/777497513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 msgr2=0x7f7f00106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 --2- 172.21.15.64:0/777497513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f00106970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009820 tx=0x7f7ef002a270 comp rx=0 tx=0).stop 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 -- 172.21.15.64:0/777497513 shutdown_connections 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 --2- 172.21.15.64:0/777497513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f00106970 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 -- 172.21.15.64:0/777497513 >> 172.21.15.64:0/777497513 conn(0x7f7f00101d80 msgr2=0x7f7f001041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:23.469 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 -- 172.21.15.64:0/777497513 shutdown_connections 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.123+0000 7f7f05ad0640 1 -- 172.21.15.64:0/777497513 wait complete. 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7f05ad0640 1 Processor -- start 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7f05ad0640 1 -- start start 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7f05ad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f0019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7f05ad0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0010aad0 con 0x7f7f00106570 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f0019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f0019b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49226/0 (socket says 172.21.15.64:49226) 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7eff7fe640 1 -- 172.21.15.64:0/558665323 learned_addr learned my addr 172.21.15.64:0/558665323 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.124+0000 7f7eff7fe640 1 -- 172.21.15.64:0/558665323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0019bbf0 con 0x7f7f00106570 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.125+0000 7f7eff7fe640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f0019b6b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009940 tx=0x7f7ef00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.125+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0042070 con 0x7f7f00106570 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.125+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0019be80 con 0x7f7f00106570 2024-09-19T04:57:23.470 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.125+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef0030d00 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.125+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0037040 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.125+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0019c2e0 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.126+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f001069f0 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.126+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f7ef0045070 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.126+0000 7f7efcff9640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7ee803d610 0x7f7ee803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.126+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ef006c3f0 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.127+0000 7f7efeffd640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7ee803d610 0x7f7ee803fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.127+0000 7f7efeffd640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7ee803d610 0x7f7ee803fad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40098c0 tx=0x7f7ef4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.131+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7ef0046e00 con 0x7f7f00106570 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.336+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f7f001955f0 con 0x7f7ee803d610 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.585+0000 7f7efcff9640 1 -- 172.21.15.64:0/558665323 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f7f001955f0 con 0x7f7ee803d610 2024-09-19T04:57:23.471 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.587+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7ee803d610 msgr2=0x7f7ee803fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.587+0000 7f7f05ad0640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7ee803d610 0x7f7ee803fad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40098c0 tx=0x7f7ef4006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.587+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 msgr2=0x7f7f0019b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.587+0000 7f7f05ad0640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f0019b6b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009940 tx=0x7f7ef00327b0 comp rx=0 tx=0).stop 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.588+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 shutdown_connections 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.588+0000 7f7f05ad0640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7ee803d610 0x7f7ee803fad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.588+0000 7f7f05ad0640 1 --2- 172.21.15.64:0/558665323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f00106570 0x7f7f0019b6b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.588+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 >> 172.21.15.64:0/558665323 conn(0x7f7f00101d80 msgr2=0x7f7f00102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.588+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 shutdown_connections 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:22.588+0000 7f7f05ad0640 1 -- 172.21.15.64:0/558665323 wait complete. 2024-09-19T04:57:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2024-09-19T04:57:24.088 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:23 smithi064 ceph-mon[30960]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:25.060 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:24 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1560847212' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T04:57:25.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T04:57:25.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.264+0000 7fb363b78640 1 Processor -- start 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.264+0000 7fb363b78640 1 -- start start 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.264+0000 7fb363b78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.264+0000 7fb363b78640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c106f10 con 0x7fb35c106540 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.265+0000 7fb361914640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.265+0000 7fb361914640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36630/0 (socket says 172.21.15.64:36630) 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.265+0000 7fb361914640 1 -- 172.21.15.64:0/1759925824 learned_addr learned my addr 172.21.15.64:0/1759925824 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.265+0000 7fb361914640 1 -- 172.21.15.64:0/1759925824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c107740 con 0x7fb35c106540 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb361914640 1 --2- 172.21.15.64:0/1759925824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c106940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb358009820 tx=0x7fb35802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f599ef3ec72387 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb360912640 1 -- 172.21.15.64:0/1759925824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb358037070 con 0x7fb35c106540 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb360912640 1 -- 172.21.15.64:0/1759925824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3580328f0 con 0x7fb35c106540 2024-09-19T04:57:25.068 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb360912640 1 -- 172.21.15.64:0/1759925824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb358030740 con 0x7fb35c106540 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 -- 172.21.15.64:0/1759925824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 msgr2=0x7fb35c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 --2- 172.21.15.64:0/1759925824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c106940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb358009820 tx=0x7fb35802a270 comp rx=0 tx=0).stop 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 -- 172.21.15.64:0/1759925824 shutdown_connections 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 --2- 172.21.15.64:0/1759925824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c106940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 -- 172.21.15.64:0/1759925824 >> 172.21.15.64:0/1759925824 conn(0x7fb35c101d90 msgr2=0x7fb35c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 -- 172.21.15.64:0/1759925824 shutdown_connections 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.266+0000 7fb363b78640 1 -- 172.21.15.64:0/1759925824 wait complete. 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb363b78640 1 Processor -- start 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb363b78640 1 -- start start 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb363b78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb363b78640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10aaf0 con 0x7fb35c106540 2024-09-19T04:57:25.069 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb361914640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb361914640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c19b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36632/0 (socket says 172.21.15.64:36632) 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.267+0000 7fb361914640 1 -- 172.21.15.64:0/1560847212 learned_addr learned my addr 172.21.15.64:0/1560847212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb361914640 1 -- 172.21.15.64:0/1560847212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c19bb90 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb361914640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c19b650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097f0 tx=0x7fb358030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb358042070 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb358030d80 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c19be20 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb358037040 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.268+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c19eb10 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.269+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb358045070 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.269+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c10ac70 con 0x7fb35c106540 2024-09-19T04:57:25.070 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.269+0000 7fb352ffd640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb348039200 0x7fb34803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.269+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb35806c430 con 0x7fb35c106540 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.269+0000 7fb361113640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb348039200 0x7fb34803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.270+0000 7fb361113640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb348039200 0x7fb34803b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0098c0 tx=0x7fb34c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.274+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb35803d430 con 0x7fb35c106540 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.449+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb35c02d9b0 con 0x7fb35c106540 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.450+0000 7fb352ffd640 1 -- 172.21.15.64:0/1560847212 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb35803db00 con 0x7fb35c106540 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb348039200 msgr2=0x7fb34803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb348039200 0x7fb34803b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0098c0 tx=0x7fb34c006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 msgr2=0x7fb35c19b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c19b650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097f0 tx=0x7fb358030c70 comp rx=0 tx=0).stop 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 shutdown_connections 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb348039200 0x7fb34803b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:25.071 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 --2- 172.21.15.64:0/1560847212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb35c106540 0x7fb35c19b650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:25.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 >> 172.21.15.64:0/1560847212 conn(0x7fb35c101d90 msgr2=0x7fb35c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:25.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 shutdown_connections 2024-09-19T04:57:25.072 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:24.452+0000 7fb363b78640 1 -- 172.21.15.64:0/1560847212 wait complete. 2024-09-19T04:57:25.074 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:57:25.075 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-19T04:57:25.082 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:57:25.082 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-19T04:57:25.082 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-19T04:57:25.082 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T04:57:25.084 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2024-09-19T04:57:25.084 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:25.084 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-09-19T04:57:25.084 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2024-09-19T04:57:25.084 INFO:teuthology.orchestra.run.smithi064.stdout: Password: haewbf48de 2024-09-19T04:57:25.084 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:25.085 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config directory 2024-09-19T04:57:26.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:25 smithi064 ceph-mon[30960]: mgrmap e13: x(active, since 6s) 2024-09-19T04:57:26.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:25 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/161216928' entity='client.admin' 2024-09-19T04:57:26.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.485+0000 7f0994c21640 1 Processor -- start 2024-09-19T04:57:26.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.485+0000 7f0994c21640 1 -- start start 2024-09-19T04:57:26.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.486+0000 7f0994c21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f0990107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.390 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.486+0000 7f0994c21640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f09901080b0 con 0x7f09901076e0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.486+0000 7f098e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f0990107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.486+0000 7f098e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f0990107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36642/0 (socket says 172.21.15.64:36642) 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.486+0000 7f098e59c640 1 -- 172.21.15.64:0/1230701665 learned_addr learned my addr 172.21.15.64:0/1230701665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.486+0000 7f098e59c640 1 -- 172.21.15.64:0/1230701665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0990108840 con 0x7f09901076e0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.487+0000 7f098e59c640 1 --2- 172.21.15.64:0/1230701665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f0990107ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0980009820 tx=0x7f098002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7365106e12a0c868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.487+0000 7f098d59a640 1 -- 172.21.15.64:0/1230701665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0980037070 con 0x7f09901076e0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.487+0000 7f098d59a640 1 -- 172.21.15.64:0/1230701665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09800328f0 con 0x7f09901076e0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.487+0000 7f098d59a640 1 -- 172.21.15.64:0/1230701665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0980030740 con 0x7f09901076e0 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 -- 172.21.15.64:0/1230701665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 msgr2=0x7f0990107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 --2- 172.21.15.64:0/1230701665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f0990107ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0980009820 tx=0x7f098002a270 comp rx=0 tx=0).stop 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 -- 172.21.15.64:0/1230701665 shutdown_connections 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 --2- 172.21.15.64:0/1230701665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f0990107ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.391 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 -- 172.21.15.64:0/1230701665 >> 172.21.15.64:0/1230701665 conn(0x7f0990102ed0 msgr2=0x7f09901052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 -- 172.21.15.64:0/1230701665 shutdown_connections 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.488+0000 7f0994c21640 1 -- 172.21.15.64:0/1230701665 wait complete. 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f0994c21640 1 Processor -- start 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f0994c21640 1 -- start start 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f0994c21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f099019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f0994c21640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f099010bbf0 con 0x7f09901076e0 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f098e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f099019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f098e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f099019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36644/0 (socket says 172.21.15.64:36644) 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f098e59c640 1 -- 172.21.15.64:0/161216928 learned_addr learned my addr 172.21.15.64:0/161216928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f098e59c640 1 -- 172.21.15.64:0/161216928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099019dda0 con 0x7f09901076e0 2024-09-19T04:57:26.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.489+0000 7f098e59c640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f099019d860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0980009940 tx=0x7f0980030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.490+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0980042070 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.490+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099019e030 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.490+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0980030e30 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.490+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0980037040 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.490+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09901a0d20 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.491+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0980045070 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.491+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099010bd70 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.491+0000 7f097f7fe640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0978039300 0x7f097803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.491+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f098006c530 con 0x7f09901076e0 2024-09-19T04:57:26.393 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.491+0000 7f098dd9b640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0978039300 0x7f097803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.491+0000 7f098dd9b640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0978039300 0x7f097803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f09840098c0 tx=0x7f0984006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.496+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0980045360 con 0x7f09901076e0 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.759+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f09901a0ef0 con 0x7f09901076e0 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.762+0000 7f097f7fe640 1 -- 172.21.15.64:0/161216928 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f098002ed90 con 0x7f09901076e0 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0978039300 msgr2=0x7f097803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0978039300 0x7f097803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f09840098c0 tx=0x7f0984006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 msgr2=0x7f099019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f099019d860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0980009940 tx=0x7f0980030cc0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 shutdown_connections 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0978039300 0x7f097803b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 --2- 172.21.15.64:0/161216928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09901076e0 0x7f099019d860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.394 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 >> 172.21.15.64:0/161216928 conn(0x7f0990102ed0 msgr2=0x7f09901039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 shutdown_connections 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-19T04:57:25.764+0000 7f0994c21640 1 -- 172.21.15.64:0/161216928 wait complete. 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.395 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2024-09-19T04:57:26.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.396 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2024-09-19T04:57:26.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.396 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T04:57:26.396 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:26.396 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2024-09-19T04:57:26.429 INFO:tasks.cephadm:Fetching config... 2024-09-19T04:57:26.429 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:57:26.429 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T04:57:26.446 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T04:57:26.447 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:57:26.447 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T04:57:26.503 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T04:57:26.504 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:57:26.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T04:57:26.570 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T04:57:26.570 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:57:26.570 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T04:57:26.627 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T04:57:26.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:57:26.705 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:26.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:57:26.868 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:26.880 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:57:27.073 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0d1gFwPK8XOKe/DRAQ8OBtiH9dTLjB99q0v6ozshzK6qFK1tm5/I1QUj2zZAHuvsHhekVFV8m5X7o8E5fp2Wt5hj63ew6kgs4MHNzx5Ru3DshCcRoZBb0xcoorDfuUAabwdjcyl8hJFDVhMp0sMybmPAEnmqVIQ0DfYMAyMmkdgrOgYM8SIzaSpi54/yFrkCfLXkwRiqFNvOpNQQKnq1P4L1IjyCAd0vnGAx0fAarueh9U2OKWcOr+yZajTHTHz0wIn7Igwf9s0R31LTTf1OOVDNxPZLRf8uk+yBIjLWItdvRKgdK+Nc4bGiRw7L9WvwpJ1RcKUX9MyK0uK13phe1pLbHoHMbysa5nF0TM8QCaE1dtOCTJfxpyxeipH8MocNoRqONzSZbr/dTXfgen6QO/CiPLSVzLbWs60SMZ1dTDhjuXliwGHD0DbyOODTZVY/VuYwArvgL5IZKmGRgimgp9HLx0BSKgNKrCGHb+nHn/PKxLLTwkGAvBXcFUkkh+bU= ceph-5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:27.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T04:57:27.378 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:28.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.027+0000 7f98453b3640 1 -- 172.21.15.64:0/2166645688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 msgr2=0x7f9840100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.027+0000 7f98453b3640 1 --2- 172.21.15.64:0/2166645688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840100910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9830009880 tx=0x7f983002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.027+0000 7f98453b3640 1 -- 172.21.15.64:0/2166645688 shutdown_connections 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.028+0000 7f98453b3640 1 --2- 172.21.15.64:0/2166645688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840100910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.028+0000 7f98453b3640 1 -- 172.21.15.64:0/2166645688 >> 172.21.15.64:0/2166645688 conn(0x7f98400fbc80 msgr2=0x7f98400fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.028+0000 7f98453b3640 1 -- 172.21.15.64:0/2166645688 shutdown_connections 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.028+0000 7f98453b3640 1 -- 172.21.15.64:0/2166645688 wait complete. 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.028+0000 7f98453b3640 1 Processor -- start 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f98453b3640 1 -- start start 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f98453b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f98453b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98401064d0 con 0x7f9840100510 2024-09-19T04:57:28.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f983effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f983effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36658/0 (socket says 172.21.15.64:36658) 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f983effd640 1 -- 172.21.15.64:0/3548735611 learned_addr learned my addr 172.21.15.64:0/3548735611 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f983effd640 1 -- 172.21.15.64:0/3548735611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840197200 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f983effd640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840196cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9830009230 tx=0x7f98300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.029+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f983003f070 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.030+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9830003aa0 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.030+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9840197490 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.030+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9830035030 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.030+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984019a180 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.031+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9830042070 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.031+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9840106c50 con 0x7f9840100510 2024-09-19T04:57:28.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.031+0000 7f983cff9640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f982c039300 0x7f982c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.031+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f983006d150 con 0x7f9840100510 2024-09-19T04:57:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.031+0000 7f9837fff640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f982c039300 0x7f982c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:28.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.032+0000 7f9837fff640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f982c039300 0x7f982c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98280098c0 tx=0x7f9828006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:28.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.036+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f983004c350 con 0x7f9840100510 2024-09-19T04:57:28.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.215+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f984002d830 con 0x7f9840100510 2024-09-19T04:57:28.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.217+0000 7f983cff9640 1 -- 172.21.15.64:0/3548735611 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9830036260 con 0x7f9840100510 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f982c039300 msgr2=0x7f982c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f982c039300 0x7f982c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98280098c0 tx=0x7f9828006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 msgr2=0x7f9840196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840196cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9830009230 tx=0x7f98300048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 shutdown_connections 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f982c039300 0x7f982c03b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 --2- 172.21.15.64:0/3548735611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9840100510 0x7f9840196cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 >> 172.21.15.64:0/3548735611 conn(0x7f98400fbc80 msgr2=0x7f984018dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.219+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 shutdown_connections 2024-09-19T04:57:28.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:28.220+0000 7f98453b3640 1 -- 172.21.15.64:0/3548735611 wait complete. 2024-09-19T04:57:29.160 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T04:57:29.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3548735611' entity='client.admin' 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 -- 172.21.15.64:0/2024148566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 msgr2=0x7f59d81044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 --2- 172.21.15.64:0/2024148566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d81044c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009880 tx=0x7f59d002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 -- 172.21.15.64:0/2024148566 shutdown_connections 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 --2- 172.21.15.64:0/2024148566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d81044c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 -- 172.21.15.64:0/2024148566 >> 172.21.15.64:0/2024148566 conn(0x7f59d80fbcd0 msgr2=0x7f59d80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 -- 172.21.15.64:0/2024148566 shutdown_connections 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.919+0000 7f59e0599640 1 -- 172.21.15.64:0/2024148566 wait complete. 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59e0599640 1 Processor -- start 2024-09-19T04:57:29.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59e0599640 1 -- start start 2024-09-19T04:57:29.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59e0599640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:29.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59e0599640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8107520 con 0x7f59d81020d0 2024-09-19T04:57:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59de335640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59de335640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d8196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36684/0 (socket says 172.21.15.64:36684) 2024-09-19T04:57:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59de335640 1 -- 172.21.15.64:0/4059287888 learned_addr learned my addr 172.21.15.64:0/4059287888 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59de335640 1 -- 172.21.15.64:0/4059287888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d8197150 con 0x7f59d81020d0 2024-09-19T04:57:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.920+0000 7f59de335640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d8196c10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009230 tx=0x7f59d00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.921+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d003f070 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.921+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d0003aa0 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.921+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d8197440 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.921+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0035030 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.921+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d819a0d0 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.922+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f59d0042070 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.922+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d8107ca0 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.922+0000 7f59d77fe640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c00392a0 0x7f59c003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.922+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59d006d150 con 0x7f59d81020d0 2024-09-19T04:57:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.922+0000 7f59ddb34640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c00392a0 0x7f59c003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.922+0000 7f59ddb34640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c00392a0 0x7f59c003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098c0 tx=0x7f59cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:29.927+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59d004a260 con 0x7f59d81020d0 2024-09-19T04:57:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.104+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f59d802d9a0 con 0x7f59c00392a0 2024-09-19T04:57:30.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.108+0000 7f59d77fe640 1 -- 172.21.15.64:0/4059287888 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59d802d9a0 con 0x7f59c00392a0 2024-09-19T04:57:30.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c00392a0 msgr2=0x7f59c003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:30.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c00392a0 0x7f59c003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098c0 tx=0x7f59cc006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 msgr2=0x7f59d8196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d8196c10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009230 tx=0x7f59d00048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 shutdown_connections 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c00392a0 0x7f59c003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 --2- 172.21.15.64:0/4059287888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59d81020d0 0x7f59d8196c10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 >> 172.21.15.64:0/4059287888 conn(0x7f59d80fbcd0 msgr2=0x7f59d80fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 shutdown_connections 2024-09-19T04:57:30.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:30.110+0000 7f59e0599640 1 -- 172.21.15.64:0/4059287888 wait complete. 2024-09-19T04:57:30.854 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2024-09-19T04:57:30.854 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:57:30.854 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:57:30.872 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:57:30.873 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:30.930 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2024-09-19T04:57:30.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch host add smithi103 2024-09-19T04:57:31.185 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T04:57:31.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:31.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:31.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 -- 172.21.15.64:0/875275514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 msgr2=0x7f4e9c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 --2- 172.21.15.64:0/875275514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009880 tx=0x7f4e8c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 -- 172.21.15.64:0/875275514 shutdown_connections 2024-09-19T04:57:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 --2- 172.21.15.64:0/875275514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 -- 172.21.15.64:0/875275514 >> 172.21.15.64:0/875275514 conn(0x7f4e9c0fd290 msgr2=0x7f4e9c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 -- 172.21.15.64:0/875275514 shutdown_connections 2024-09-19T04:57:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.610+0000 7f4ea207a640 1 -- 172.21.15.64:0/875275514 wait complete. 2024-09-19T04:57:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.611+0000 7f4ea207a640 1 Processor -- start 2024-09-19T04:57:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.611+0000 7f4ea207a640 1 -- start start 2024-09-19T04:57:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.611+0000 7f4ea207a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.611+0000 7f4ea207a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c10bb40 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36710/0 (socket says 172.21.15.64:36710) 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/2548888732 learned_addr learned my addr 172.21.15.64:0/2548888732 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e9b7fe640 1 -- 172.21.15.64:0/2548888732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c197560 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e9b7fe640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c197020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009230 tx=0x7f4e8c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e8c03f070 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c1977f0 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e8c004d10 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e8c035030 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.612+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c19b8f0 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.613+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4e8c042070 con 0x7f4e9c107660 2024-09-19T04:57:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.613+0000 7f4e997fa640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4e880392a0 0x7f4e8803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.613+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4e8c06cff0 con 0x7f4e9c107660 2024-09-19T04:57:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.613+0000 7f4e92ffd640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4e880392a0 0x7f4e8803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.613+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c102cd0 con 0x7f4e9c107660 2024-09-19T04:57:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.614+0000 7f4e92ffd640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4e880392a0 0x7f4e8803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098c0 tx=0x7f4e84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:31.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.618+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e8c039350 con 0x7f4e9c107660 2024-09-19T04:57:31.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:31.796+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}) -- 0x7f4e9c107ac0 con 0x7f4e880392a0 2024-09-19T04:57:32.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:32 smithi064 ceph-mon[30960]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:57:32.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:32 smithi064 ceph-mon[30960]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:32.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:32 smithi064 ceph-mon[30960]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:57:33.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:33 smithi064 ceph-mon[30960]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:34.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:34 smithi064 ceph-mon[30960]: Deploying cephadm binary to smithi103 2024-09-19T04:57:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.645+0000 7f4e997fa640 1 -- 172.21.15.64:0/2548888732 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4e9c107ac0 con 0x7f4e880392a0 2024-09-19T04:57:35.646 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi103' with addr '172.21.15.103' 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4e880392a0 msgr2=0x7f4e8803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4e880392a0 0x7f4e8803b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098c0 tx=0x7f4e84006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 msgr2=0x7f4e9c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c197020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009230 tx=0x7f4e8c0048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 shutdown_connections 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4e880392a0 0x7f4e8803b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 --2- 172.21.15.64:0/2548888732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e9c107660 0x7f4e9c197020 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 >> 172.21.15.64:0/2548888732 conn(0x7f4e9c0fd290 msgr2=0x7f4e9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 shutdown_connections 2024-09-19T04:57:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:35.647+0000 7f4ea207a640 1 -- 172.21.15.64:0/2548888732 wait complete. 2024-09-19T04:57:36.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:57:36.511 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:36.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:36.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:36 smithi064 ceph-mon[30960]: Added host smithi103 2024-09-19T04:57:36.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:36.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:36.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.947+0000 7f072fe53640 1 -- 172.21.15.64:0/1126454955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 msgr2=0x7f0728107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:36.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.947+0000 7f072fe53640 1 --2- 172.21.15.64:0/1126454955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0724009880 tx=0x7f072402a670 comp rx=0 tx=0).stop 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 -- 172.21.15.64:0/1126454955 shutdown_connections 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 --2- 172.21.15.64:0/1126454955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 -- 172.21.15.64:0/1126454955 >> 172.21.15.64:0/1126454955 conn(0x7f07280fb930 msgr2=0x7f07280fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 -- 172.21.15.64:0/1126454955 shutdown_connections 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 -- 172.21.15.64:0/1126454955 wait complete. 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 Processor -- start 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.948+0000 7f072fe53640 1 -- start start 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072fe53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072fe53640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f072810bb00 con 0x7f0728107660 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072dbef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072dbef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728100bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33698/0 (socket says 172.21.15.64:33698) 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072dbef640 1 -- 172.21.15.64:0/929731667 learned_addr learned my addr 172.21.15.64:0/929731667 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072dbef640 1 -- 172.21.15.64:0/929731667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0728102ab0 con 0x7f0728107660 2024-09-19T04:57:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072dbef640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728100bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0724009230 tx=0x7f07240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f072403f070 con 0x7f0728107660 2024-09-19T04:57:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0724003aa0 con 0x7f0728107660 2024-09-19T04:57:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0724035030 con 0x7f0728107660 2024-09-19T04:57:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0728101160 con 0x7f0728107660 2024-09-19T04:57:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.949+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0728101620 con 0x7f0728107660 2024-09-19T04:57:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.950+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0724042070 con 0x7f0728107660 2024-09-19T04:57:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.950+0000 7f071effd640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0714039300 0x7f071403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.950+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0728107ac0 con 0x7f0728107660 2024-09-19T04:57:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.951+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f072406ce90 con 0x7f0728107660 2024-09-19T04:57:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.951+0000 7f072d3ee640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0714039300 0x7f071403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.951+0000 7f072d3ee640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0714039300 0x7f071403b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f07180098c0 tx=0x7f0718006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:36.955+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f072402fdc0 con 0x7f0728107660 2024-09-19T04:57:37.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.136+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0728102ce0 con 0x7f0714039300 2024-09-19T04:57:37.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.137+0000 7f071effd640 1 -- 172.21.15.64:0/929731667 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f0728102ce0 con 0x7f0714039300 2024-09-19T04:57:37.139 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:37.139 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0714039300 msgr2=0x7f071403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0714039300 0x7f071403b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f07180098c0 tx=0x7f0718006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 msgr2=0x7f0728100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728100bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0724009230 tx=0x7f07240048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 shutdown_connections 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0714039300 0x7f071403b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 --2- 172.21.15.64:0/929731667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0728107660 0x7f0728100bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.139+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 >> 172.21.15.64:0/929731667 conn(0x7f07280fb930 msgr2=0x7f07280fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:37.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.140+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 shutdown_connections 2024-09-19T04:57:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:37.140+0000 7f072fe53640 1 -- 172.21.15.64:0/929731667 wait complete. 2024-09-19T04:57:37.732 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-09-19T04:57:37.732 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:57:37.732 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:57:37.752 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:57:37.752 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:37.811 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-09-19T04:57:37.812 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch host add smithi174 2024-09-19T04:57:38.064 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:38.283 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:37 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:38.283 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:37 smithi064 ceph-mon[30960]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:57:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.500+0000 7fd1c0529640 1 -- 172.21.15.64:0/3239085933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 msgr2=0x7fd1b8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.500+0000 7fd1c0529640 1 --2- 172.21.15.64:0/3239085933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b8107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009880 tx=0x7fd1b402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 -- 172.21.15.64:0/3239085933 shutdown_connections 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 --2- 172.21.15.64:0/3239085933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b8107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 -- 172.21.15.64:0/3239085933 >> 172.21.15.64:0/3239085933 conn(0x7fd1b80fd270 msgr2=0x7fd1b80ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 -- 172.21.15.64:0/3239085933 shutdown_connections 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 -- 172.21.15.64:0/3239085933 wait complete. 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 Processor -- start 2024-09-19T04:57:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.501+0000 7fd1c0529640 1 -- start start 2024-09-19T04:57:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1c0529640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1c0529640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b810bb40 con 0x7fd1b8107660 2024-09-19T04:57:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1be2c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1be2c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b81992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33728/0 (socket says 172.21.15.64:33728) 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1be2c5640 1 -- 172.21.15.64:0/3195955779 learned_addr learned my addr 172.21.15.64:0/3195955779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1be2c5640 1 -- 172.21.15.64:0/3195955779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b81997e0 con 0x7fd1b8107660 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1be2c5640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b81992a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009230 tx=0x7fd1b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.502+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b403f070 con 0x7fd1b8107660 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.503+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1b4004d10 con 0x7fd1b8107660 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.503+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4035030 con 0x7fd1b8107660 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.503+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b8199a70 con 0x7fd1b8107660 2024-09-19T04:57:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.503+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b8199ed0 con 0x7fd1b8107660 2024-09-19T04:57:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.504+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd1b4042070 con 0x7fd1b8107660 2024-09-19T04:57:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.504+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b8102cd0 con 0x7fd1b8107660 2024-09-19T04:57:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.504+0000 7fd1af7fe640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd1a40392a0 0x7fd1a403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.504+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1b406d020 con 0x7fd1b8107660 2024-09-19T04:57:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.504+0000 7fd1bdac4640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd1a40392a0 0x7fd1a403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.504+0000 7fd1bdac4640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd1a40392a0 0x7fd1a403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098c0 tx=0x7fd1a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.509+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1b4039350 con 0x7fd1b8107660 2024-09-19T04:57:38.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:38.687+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7fd1b819a600 con 0x7fd1a40392a0 2024-09-19T04:57:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:39 smithi064 ceph-mon[30960]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:39 smithi064 ceph-mon[30960]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:41.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:41.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:41.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: Deploying cephadm binary to smithi174 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:41.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:41 smithi064 ceph-mon[30960]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:41 smithi064 ceph-mon[30960]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:57:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.513+0000 7fd1af7fe640 1 -- 172.21.15.64:0/3195955779 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd1b819a600 con 0x7fd1a40392a0 2024-09-19T04:57:42.514 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-09-19T04:57:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd1a40392a0 msgr2=0x7fd1a403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd1a40392a0 0x7fd1a403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098c0 tx=0x7fd1a8006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 msgr2=0x7fd1b81992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b81992a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009230 tx=0x7fd1b40048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 shutdown_connections 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd1a40392a0 0x7fd1a403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 --2- 172.21.15.64:0/3195955779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1b8107660 0x7fd1b81992a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 >> 172.21.15.64:0/3195955779 conn(0x7fd1b80fd270 msgr2=0x7fd1b80ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 shutdown_connections 2024-09-19T04:57:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:42.516+0000 7fd1c0529640 1 -- 172.21.15.64:0/3195955779 wait complete. 2024-09-19T04:57:43.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:57:43.347 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:43.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:43 smithi064 ceph-mon[30960]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:43.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:43.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:43 smithi064 ceph-mon[30960]: Added host smithi174 2024-09-19T04:57:43.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:43.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:43.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 -- 172.21.15.64:0/1421367203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 msgr2=0x7f8fe0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 --2- 172.21.15.64:0/1421367203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe0107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009880 tx=0x7f8fc802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 -- 172.21.15.64:0/1421367203 shutdown_connections 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 --2- 172.21.15.64:0/1421367203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe0107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 -- 172.21.15.64:0/1421367203 >> 172.21.15.64:0/1421367203 conn(0x7f8fe00fd290 msgr2=0x7f8fe00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 -- 172.21.15.64:0/1421367203 shutdown_connections 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.845+0000 7f8fe4c19640 1 -- 172.21.15.64:0/1421367203 wait complete. 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fe4c19640 1 Processor -- start 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fe4c19640 1 -- start start 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fe4c19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fe4c19640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe010bb40 con 0x7f8fe0107660 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fde59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fde59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33750/0 (socket says 172.21.15.64:33750) 2024-09-19T04:57:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.846+0000 7f8fde59c640 1 -- 172.21.15.64:0/2294821773 learned_addr learned my addr 172.21.15.64:0/2294821773 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fde59c640 1 -- 172.21.15.64:0/2294821773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe00763e0 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fde59c640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe00744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009230 tx=0x7f8fc80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc803f070 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc8004d10 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe0074a30 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fc8035030 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.847+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe0074ef0 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.848+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8fc8042070 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.848+0000 7f8fd77fe640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8fcc039300 0x7f8fcc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.848+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fc806cf90 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.848+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe0102cd0 con 0x7f8fe0107660 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.848+0000 7f8fddd9b640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8fcc039300 0x7f8fcc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.849+0000 7f8fddd9b640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8fcc039300 0x7f8fcc03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098c0 tx=0x7f8fd0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:43.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:43.853+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8fc8039350 con 0x7f8fe0107660 2024-09-19T04:57:44.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.037+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fe0075a20 con 0x7f8fcc039300 2024-09-19T04:57:44.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.038+0000 7f8fd77fe640 1 -- 172.21.15.64:0/2294821773 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f8fe0075a20 con 0x7f8fcc039300 2024-09-19T04:57:44.039 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:57:44.039 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-09-19T04:57:44.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8fcc039300 msgr2=0x7f8fcc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:44.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8fcc039300 0x7f8fcc03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098c0 tx=0x7f8fd0006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 msgr2=0x7f8fe00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe00744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009230 tx=0x7f8fc80048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 shutdown_connections 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8fcc039300 0x7f8fcc03b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 --2- 172.21.15.64:0/2294821773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0107660 0x7f8fe00744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 >> 172.21.15.64:0/2294821773 conn(0x7f8fe00fd290 msgr2=0x7f8fe00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 shutdown_connections 2024-09-19T04:57:44.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:44.040+0000 7f8fe4c19640 1 -- 172.21.15.64:0/2294821773 wait complete. 2024-09-19T04:57:44.663 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T04:57:44.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T04:57:44.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:44 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:44.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:44 smithi064 ceph-mon[30960]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:57:44.921 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:45.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 -- 172.21.15.64:0/2061375890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 msgr2=0x7fef5c102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 --2- 172.21.15.64:0/2061375890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c102b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009810 tx=0x7fef4c02a540 comp rx=0 tx=0).stop 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 -- 172.21.15.64:0/2061375890 shutdown_connections 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 --2- 172.21.15.64:0/2061375890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c102b10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 -- 172.21.15.64:0/2061375890 >> 172.21.15.64:0/2061375890 conn(0x7fef5c077310 msgr2=0x7fef5c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 -- 172.21.15.64:0/2061375890 shutdown_connections 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.364+0000 7fef60beb640 1 -- 172.21.15.64:0/2061375890 wait complete. 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef60beb640 1 Processor -- start 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef60beb640 1 -- start start 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef60beb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef60beb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c10bb30 con 0x7fef5c102730 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:45.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50652/0 (socket says 172.21.15.64:50652) 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.365+0000 7fef5a59c640 1 -- 172.21.15.64:0/3703572433 learned_addr learned my addr 172.21.15.64:0/3703572433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef5a59c640 1 -- 172.21.15.64:0/3703572433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c102280 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef5a59c640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c100390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef4c030b00 tx=0x7fef4c030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef4c040070 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1008d0 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef4c0318b0 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef4c003de0 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.366+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c100d90 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.367+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fef4c039040 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.367+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c069f90 con 0x7fef5c102730 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.367+0000 7fef4b7fe640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef44039300 0x7fef4403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.367+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef4c06d890 con 0x7fef5c102730 2024-09-19T04:57:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.367+0000 7fef59d9b640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef44039300 0x7fef4403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.368+0000 7fef59d9b640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef44039300 0x7fef4403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef500098c0 tx=0x7fef50006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.372+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef4c043b60 con 0x7fef5c102730 2024-09-19T04:57:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.553+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fef5c1018c0 con 0x7fef5c102730 2024-09-19T04:57:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.857+0000 7fef4b7fe640 1 -- 172.21.15.64:0/3703572433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fef4c04a210 con 0x7fef5c102730 2024-09-19T04:57:45.858 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-09-19T04:57:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef44039300 msgr2=0x7fef4403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef44039300 0x7fef4403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef500098c0 tx=0x7fef50006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 msgr2=0x7fef5c100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:45.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c100390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef4c030b00 tx=0x7fef4c030b30 comp rx=0 tx=0).stop 2024-09-19T04:57:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 shutdown_connections 2024-09-19T04:57:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef44039300 0x7fef4403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 --2- 172.21.15.64:0/3703572433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef5c102730 0x7fef5c100390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 >> 172.21.15.64:0/3703572433 conn(0x7fef5c077310 msgr2=0x7fef5c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 shutdown_connections 2024-09-19T04:57:45.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:57:45.859+0000 7fef60beb640 1 -- 172.21.15.64:0/3703572433 wait complete. 2024-09-19T04:57:46.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:45 smithi064 ceph-mon[30960]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:46.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:45 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3703572433' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:57:46.461 INFO:tasks.cephadm:Adding mon.a on smithi064 2024-09-19T04:57:46.461 INFO:tasks.cephadm:Adding mon.b on smithi103 2024-09-19T04:57:46.461 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-09-19T04:57:46.462 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c' 2024-09-19T04:57:46.751 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T04:57:46.796 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T04:57:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3703572433' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:57:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:57:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:47.245 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.243+0000 7f6cb914e640 1 -- 172.21.15.174:0/2726496650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 msgr2=0x7f6cb4100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:47.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.243+0000 7f6cb914e640 1 --2- 172.21.15.174:0/2726496650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb4100070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009880 tx=0x7f6ca402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:47.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.244+0000 7f6cb914e640 1 -- 172.21.15.174:0/2726496650 shutdown_connections 2024-09-19T04:57:47.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.244+0000 7f6cb914e640 1 --2- 172.21.15.174:0/2726496650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb4100070 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:47.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.244+0000 7f6cb914e640 1 -- 172.21.15.174:0/2726496650 >> 172.21.15.174:0/2726496650 conn(0x7f6cb40fb400 msgr2=0x7f6cb40fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:47.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.244+0000 7f6cb914e640 1 -- 172.21.15.174:0/2726496650 shutdown_connections 2024-09-19T04:57:47.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.244+0000 7f6cb914e640 1 -- 172.21.15.174:0/2726496650 wait complete. 2024-09-19T04:57:47.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.244+0000 7f6cb914e640 1 Processor -- start 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb914e640 1 -- start start 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb914e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb40fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb914e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4105c30 con 0x7f6cb40ffc70 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb40fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb40fc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.174:60890/0 (socket says 172.21.15.174:60890) 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb2d9d640 1 -- 172.21.15.174:0/854035972 learned_addr learned my addr 172.21.15.174:0/854035972 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.245+0000 7f6cb2d9d640 1 -- 172.21.15.174:0/854035972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb4199a30 con 0x7f6cb40ffc70 2024-09-19T04:57:47.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.246+0000 7f6cb2d9d640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb40fc2b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009230 tx=0x7f6ca40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:47.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.246+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ca403f070 con 0x7f6cb40ffc70 2024-09-19T04:57:47.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.246+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca4003aa0 con 0x7f6cb40ffc70 2024-09-19T04:57:47.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.246+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb4199c00 con 0x7f6cb40ffc70 2024-09-19T04:57:47.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.246+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ca4035030 con 0x7f6cb40ffc70 2024-09-19T04:57:47.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.246+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb4199fa0 con 0x7f6cb40ffc70 2024-09-19T04:57:47.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.247+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb41063b0 con 0x7f6cb40ffc70 2024-09-19T04:57:47.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.247+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6ca4042070 con 0x7f6cb40ffc70 2024-09-19T04:57:47.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.247+0000 7f6c9bfff640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6c9c039300 0x7f6c9c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:47.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.247+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6ca406d190 con 0x7f6cb40ffc70 2024-09-19T04:57:47.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.248+0000 7f6cb259c640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6c9c039300 0x7f6c9c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:47.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.248+0000 7f6cb259c640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6c9c039300 0x7f6c9c03b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80098c0 tx=0x7f6ca8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:47.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.252+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ca402f300 con 0x7f6cb40ffc70 2024-09-19T04:57:47.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.438+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) -- 0x7f6cb40fd110 con 0x7f6c9c039300 2024-09-19T04:57:47.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.443+0000 7f6c9bfff640 1 -- 172.21.15.174:0/854035972 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6cb40fd110 con 0x7f6c9c039300 2024-09-19T04:57:47.445 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-09-19T04:57:47.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.445+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6c9c039300 msgr2=0x7f6c9c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:47.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.445+0000 7f6cb914e640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6c9c039300 0x7f6c9c03b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80098c0 tx=0x7f6ca8006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:47.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.445+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 msgr2=0x7f6cb40fc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:47.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.445+0000 7f6cb914e640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb40fc2b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009230 tx=0x7f6ca40048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:47.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.446+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 shutdown_connections 2024-09-19T04:57:47.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.446+0000 7f6cb914e640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6c9c039300 0x7f6c9c03b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:47.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.446+0000 7f6cb914e640 1 --2- 172.21.15.174:0/854035972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb40ffc70 0x7f6cb40fc2b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:47.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.446+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 >> 172.21.15.174:0/854035972 conn(0x7f6cb40fb400 msgr2=0x7f6cb40fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:47.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.446+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 shutdown_connections 2024-09-19T04:57:47.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:47.446+0000 7f6cb914e640 1 -- 172.21.15.174:0/854035972 wait complete. 2024-09-19T04:57:48.019 DEBUG:teuthology.orchestra.run.smithi103:mon.b> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T04:57:48.022 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T04:57:48.024 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:57:48.025 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:57:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T04:57:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:57:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:57:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:48.295 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:57:48.717 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/1043931267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 msgr2=0x7fcb18106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:48.717 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 --2- 172.21.15.174:0/1043931267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009880 tx=0x7fcb0802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:57:48.717 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/1043931267 shutdown_connections 2024-09-19T04:57:48.717 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 --2- 172.21.15.174:0/1043931267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:48.717 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/1043931267 >> 172.21.15.174:0/1043931267 conn(0x7fcb180fc130 msgr2=0x7fcb180fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/1043931267 shutdown_connections 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.713+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/1043931267 wait complete. 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb1d8ed640 1 Processor -- start 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb1d8ed640 1 -- start start 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb1d8ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb1d8ed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1810aa20 con 0x7fcb18106540 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.174:60920/0 (socket says 172.21.15.174:60920) 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.714+0000 7fcb16ffd640 1 -- 172.21.15.174:0/3377943777 learned_addr learned my addr 172.21.15.174:0/3377943777 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb16ffd640 1 -- 172.21.15.174:0/3377943777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb18197580 con 0x7fcb18106540 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb16ffd640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18197040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009230 tx=0x7fcb080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0803f070 con 0x7fcb18106540 2024-09-19T04:57:48.719 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb18197810 con 0x7fcb18106540 2024-09-19T04:57:48.719 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb08004d10 con 0x7fcb18106540 2024-09-19T04:57:48.719 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08035030 con 0x7fcb18106540 2024-09-19T04:57:48.719 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.715+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1819b910 con 0x7fcb18106540 2024-09-19T04:57:48.720 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.716+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcb08042070 con 0x7fcb18106540 2024-09-19T04:57:48.720 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.716+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb18101bb0 con 0x7fcb18106540 2024-09-19T04:57:48.720 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.717+0000 7fcb1c8eb640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcb04039300 0x7fcb0403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:48.720 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.717+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcb0806d0c0 con 0x7fcb18106540 2024-09-19T04:57:48.720 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.717+0000 7fcb0ffff640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcb04039300 0x7fcb0403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:48.720 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.717+0000 7fcb0ffff640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcb04039300 0x7fcb0403b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098c0 tx=0x7fcb00006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:48.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.722+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb0802f300 con 0x7fcb18106540 2024-09-19T04:57:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.958+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcb181069a0 con 0x7fcb18106540 2024-09-19T04:57:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.959+0000 7fcb1c8eb640 1 -- 172.21.15.174:0/3377943777 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fcb08042c60 con 0x7fcb18106540 2024-09-19T04:57:48.961 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:57:48.961 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:56:20.817470Z","created":"2024-09-19T04:56:20.817470Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T04:57:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-19T04:57:48.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcb04039300 msgr2=0x7fcb0403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:48.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcb04039300 0x7fcb0403b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098c0 tx=0x7fcb00006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:48.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 msgr2=0x7fcb18197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18197040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009230 tx=0x7fcb080048f0 comp rx=0 tx=0).stop 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 shutdown_connections 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcb04039300 0x7fcb0403b7c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 --2- 172.21.15.174:0/3377943777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb18106540 0x7fcb18197040 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 >> 172.21.15.174:0/3377943777 conn(0x7fcb180fc130 msgr2=0x7fcb18069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 shutdown_connections 2024-09-19T04:57:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:48.962+0000 7fcb1d8ed640 1 -- 172.21.15.174:0/3377943777 wait complete. 2024-09-19T04:57:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c;count:3 2024-09-19T04:57:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: Reconfiguring daemon mon.a on smithi064 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:57:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:50.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:49 smithi064 ceph-mon[30960]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:50.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:49 smithi064 ceph-mon[30960]: Deploying daemon mon.c on smithi174 2024-09-19T04:57:50.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:49 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/3377943777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:57:50.577 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:57:50.577 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:57:51.283 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:57:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:51 smithi064 ceph-mon[30960]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:52.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 -- 172.21.15.174:0/2147825420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 msgr2=0x7f8504106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 --2- 172.21.15.174:0/2147825420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504106920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009810 tx=0x7f84f802a540 comp rx=0 tx=0).stop 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 -- 172.21.15.174:0/2147825420 shutdown_connections 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 --2- 172.21.15.174:0/2147825420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504106920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 -- 172.21.15.174:0/2147825420 >> 172.21.15.174:0/2147825420 conn(0x7f8504077310 msgr2=0x7f85040fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 -- 172.21.15.174:0/2147825420 shutdown_connections 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.229+0000 7f85097dd640 1 -- 172.21.15.174:0/2147825420 wait complete. 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f85097dd640 1 Processor -- start 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f85097dd640 1 -- start start 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f85097dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f85097dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f850410aa20 con 0x7f8504106540 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f8502ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f8502ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.174:60924/0 (socket says 172.21.15.174:60924) 2024-09-19T04:57:52.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.230+0000 7f8502ffd640 1 -- 172.21.15.174:0/1974199023 learned_addr learned my addr 172.21.15.174:0/1974199023 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f8502ffd640 1 -- 172.21.15.174:0/1974199023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8504197510 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f8502ffd640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504196fd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f84f8030b00 tx=0x7f84f8030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f8040070 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85041977a0 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84f80318b0 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f8003de0 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.231+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850419b8a0 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.232+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f84f8039040 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.232+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8504101bb0 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.232+0000 7f84ebfff640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f84ec039300 0x7f84ec03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.232+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f84f806d860 con 0x7f8504106540 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.232+0000 7f85027fc640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f84ec039300 0x7f84ec03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.233+0000 7f85027fc640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f84ec039300 0x7f84ec03b7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84f40098c0 tx=0x7f84f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:52.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.237+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84f8043bd0 con 0x7f8504106540 2024-09-19T04:57:52.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.482+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8504198330 con 0x7f8504106540 2024-09-19T04:57:52.485 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:57:52.485 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:56:20.817470Z","created":"2024-09-19T04:56:20.817470Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T04:57:52.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.483+0000 7f84ebfff640 1 -- 172.21.15.174:0/1974199023 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f84f804d3a0 con 0x7f8504106540 2024-09-19T04:57:52.486 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f84ec039300 msgr2=0x7f84ec03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f84ec039300 0x7f84ec03b7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84f40098c0 tx=0x7f84f4006db0 comp rx=0 tx=0).stop 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 msgr2=0x7f8504196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504196fd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f84f8030b00 tx=0x7f84f8030b30 comp rx=0 tx=0).stop 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 shutdown_connections 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f84ec039300 0x7f84ec03b7c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 --2- 172.21.15.174:0/1974199023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504106540 0x7f8504196fd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:52.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 >> 172.21.15.174:0/1974199023 conn(0x7f8504077310 msgr2=0x7f85040696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:52.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 shutdown_connections 2024-09-19T04:57:52.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:52.485+0000 7f85097dd640 1 -- 172.21.15.174:0/1974199023 wait complete. 2024-09-19T04:57:53.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:52 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/1974199023' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:57:53.276 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 podman[34469]: 2024-09-19 04:57:53.167564605 +0000 UTC m=+0.015784875 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:57:53.276 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 podman[34469]: 2024-09-19 04:57:53.275476319 +0000 UTC m=+0.123696581 container create fdede484ad090cb2c23c21a25730cdb4492852dd2dca79529d0307ca55ef489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 podman[34469]: 2024-09-19 04:57:53.557353871 +0000 UTC m=+0.405574141 container init fdede484ad090cb2c23c21a25730cdb4492852dd2dca79529d0307ca55ef489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 podman[34469]: 2024-09-19 04:57:53.560552925 +0000 UTC m=+0.408773188 container start fdede484ad090cb2c23c21a25730cdb4492852dd2dca79529d0307ca55ef489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: pidfile_write: ignore empty --pid-file 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: load: jerasure load: lrc 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Git sha 0 2024-09-19T04:57:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: DB SUMMARY 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: DB Session ID: T0L3A2IY8LS9892WTE8M 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:57:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:57:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.env: 0x558370f1d940 2024-09-19T04:57:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:57:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.info_log: 0x55837323d6e0 2024-09-19T04:57:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:57:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.statistics: (nil) 2024-09-19T04:57:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.use_fsync: 0 2024-09-19T04:57:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:57:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:57:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.db_log_dir: 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.wal_dir: 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:57:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.write_buffer_manager: 0x55837324cdc0 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:57:54.033 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.unordered_write: 0 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.row_cache: None 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.wal_filter: None 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:57:54.034 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.wal_compression: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:57:54.035 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_open_files: -1 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:57:54.036 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Compression algorithms supported: 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kZSTD supported: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:57:54.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.merge_operator: 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_filter: None 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55837323d320) 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:57:54.038 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x5583732311f0 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-09-19T04:57:54.039 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-09-19T04:57:54.040 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression: NoCompression 2024-09-19T04:57:54.041 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.num_levels: 7 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:57:54.042 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:57:54.043 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:57:54.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:57:54.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:57:54.046 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.ttl: 2592000 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:57:54.047 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6c768e01-fcd5-41a5-979b-0eee982f4769 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721873587916, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721873588332, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726721873, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6c768e01-fcd5-41a5-979b-0eee982f4769", "db_session_id": "T0L3A2IY8LS9892WTE8M", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:57:54.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721873588422, "job": 1, "event": "recovery_finished"} 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55837325ee00 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: DB pointer 0x55837335a000 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:57:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T04:57:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x5583732311f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T04:57:54.051 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(???) e0 preinit fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: btime 2024-09-19T04:56:23:797082+0000 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-09-19T04:57:54.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.053 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:57:54.054 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:57:54.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3703572433' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3703572433' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c;count:3 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Reconfiguring daemon mon.a on smithi064 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.057 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: Deploying daemon mon.c on smithi174 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/3377943777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/1974199023' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 ceph-mon[34488]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 bash[34469]: fdede484ad090cb2c23c21a25730cdb4492852dd2dca79529d0307ca55ef489a 2024-09-19T04:57:54.058 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:53 smithi174 systemd[1]: Started Ceph mon.c for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:57:54.306 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:57:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: mon.a calling monitor election 2024-09-19T04:57:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: mon.c calling monitor election 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: monmap epoch 2 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: last_changed 2024-09-19T04:57:53.613049+0000 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: min_mon_release 19 (squid) 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: election_strategy: 1 2024-09-19T04:57:59.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: fsmap 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: mgrmap e13: x(active, since 40s) 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: overall HEALTH_OK 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:57:59.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:59.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 -- 172.21.15.174:0/29641614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 msgr2=0x7fecbc007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:59.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 --2- 172.21.15.174:0/29641614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7fecbc007bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feccc1016a0 tx=0x7fecc002c490 comp rx=0 tx=0).stop 2024-09-19T04:57:59.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 -- 172.21.15.174:0/29641614 shutdown_connections 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 --2- 172.21.15.174:0/29641614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7fecbc007bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 -- 172.21.15.174:0/29641614 >> 172.21.15.174:0/29641614 conn(0x7feccc0fc220 msgr2=0x7feccc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 -- 172.21.15.174:0/29641614 shutdown_connections 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.131+0000 7fecd31c0640 1 -- 172.21.15.174:0/29641614 wait complete. 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd31c0640 1 Processor -- start 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd31c0640 1 -- start start 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd31c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7feccc194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd31c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd31c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feccc107550 con 0x7fecbc005740 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd31c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1073d0 con 0x7feccc1954d0 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7feccbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7feccbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43932/0 (socket says 172.21.15.174:43932) 2024-09-19T04:57:59.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7feccbfff640 1 -- 172.21.15.174:0/1302694628 learned_addr learned my addr 172.21.15.174:0/1302694628 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.132+0000 7fecd0f5c640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7feccc194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7feccbfff640 1 -- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 msgr2=0x7feccc199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7feccbfff640 1 -- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 msgr2=0x7feccc199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7feccbfff640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7feccbfff640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7fecd0f5c640 1 -- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 msgr2=0x7feccc199860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7fecd0f5c640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:59.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7fecd0f5c640 1 -- 172.21.15.174:0/1302694628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc195af0 con 0x7fecbc005740 2024-09-19T04:57:59.137 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7fecd0f5c640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7feccc194f90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fecc002ca80 tx=0x7fecc00339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:59.137 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.133+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fecc0005200 con 0x7fecbc005740 2024-09-19T04:57:59.137 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.134+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc0032ca0 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.134+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc199da0 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.134+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fecc0004b90 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.134+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc19a330 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.135+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fecc0043020 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.135+0000 7fecc9ffb640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecb0039090 0x7fecb003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.135+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fecc0041020 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.135+0000 7feccbfff640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecb0039090 0x7fecb003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.135+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc101ca0 con 0x7fecbc005740 2024-09-19T04:57:59.138 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.136+0000 7feccbfff640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecb0039090 0x7fecb003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009100 tx=0x7fecb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:59.142 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.141+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecc004a020 con 0x7fecbc005740 2024-09-19T04:57:59.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:59.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: mon.a calling monitor election 2024-09-19T04:57:59.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:59.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: mon.c calling monitor election 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: monmap epoch 2 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: last_changed 2024-09-19T04:57:53.613049+0000 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: min_mon_release 19 (squid) 2024-09-19T04:57:59.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: election_strategy: 1 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: fsmap 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: mgrmap e13: x(active, since 40s) 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: overall HEALTH_OK 2024-09-19T04:57:59.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:57:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:57:59.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:59.375 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.373+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feccc19a5e0 con 0x7fecbc005740 2024-09-19T04:57:59.375 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.374+0000 7fecc9ffb640 1 -- 172.21.15.174:0/1302694628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fecc003ab60 con 0x7fecbc005740 2024-09-19T04:57:59.376 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:57:59.376 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:57:53.613049Z","created":"2024-09-19T04:56:20.817470Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:57:59.376 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecb0039090 msgr2=0x7fecb003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecb0039090 0x7fecb003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009100 tx=0x7fecb8008040 comp rx=0 tx=0).stop 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 msgr2=0x7feccc194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7feccc194f90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fecc002ca80 tx=0x7fecc00339a0 comp rx=0 tx=0).stop 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 shutdown_connections 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecb0039090 0x7fecb003b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc1954d0 0x7feccc199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 --2- 172.21.15.174:0/1302694628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecbc005740 0x7feccc194f90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 >> 172.21.15.174:0/1302694628 conn(0x7feccc0fc220 msgr2=0x7feccc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:59.378 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 shutdown_connections 2024-09-19T04:57:59.379 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:57:59.376+0000 7fecd31c0640 1 -- 172.21.15.174:0/1302694628 wait complete. 2024-09-19T04:58:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:59 smithi174 ceph-mon[34488]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:59 smithi174 ceph-mon[34488]: Deploying daemon mon.b on smithi103 2024-09-19T04:58:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:59 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/1302694628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:58:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:57:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:58:00.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:59 smithi064 ceph-mon[30960]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:00.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:59 smithi064 ceph-mon[30960]: Deploying daemon mon.b on smithi103 2024-09-19T04:58:00.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:59 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/1302694628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:58:00.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:57:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:58:01.096 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:58:01.096 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:58:01.349 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:58:01.737 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:01 smithi174 ceph-mon[34488]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:01.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 -- 172.21.15.174:0/1615887622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c08107670 msgr2=0x7f5c08107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:01.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 --2- 172.21.15.174:0/1615887622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c08107670 0x7f5c08107a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009880 tx=0x7f5c0402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:58:01.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 -- 172.21.15.174:0/1615887622 shutdown_connections 2024-09-19T04:58:01.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 --2- 172.21.15.174:0/1615887622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c08107670 0x7f5c08107a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:01.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 -- 172.21.15.174:0/1615887622 >> 172.21.15.174:0/1615887622 conn(0x7f5c080fd330 msgr2=0x7f5c080ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 -- 172.21.15.174:0/1615887622 shutdown_connections 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.816+0000 7f5c10465640 1 -- 172.21.15.174:0/1615887622 wait complete. 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c10465640 1 Processor -- start 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c10465640 1 -- start start 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c10465640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c10465640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c081977b0 0x7f5c0819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c10465640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0810bbe0 con 0x7f5c081977b0 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c10465640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0810ba60 con 0x7f5c08107670 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43940/0 (socket says 172.21.15.174:43940) 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 -- 172.21.15.174:0/3333501218 learned_addr learned my addr 172.21.15.174:0/3333501218 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 -- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 msgr2=0x7f5c08197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:58:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 -- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 msgr2=0x7f5c08197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0e201640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.817+0000 7f5c0da00640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c081977b0 0x7f5c0819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5c0da00640 1 -- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 msgr2=0x7f5c08197270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5c0da00640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5c0da00640 1 -- 172.21.15.174:0/3333501218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c08197dd0 con 0x7f5c081977b0 2024-09-19T04:58:01.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5c0da00640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c081977b0 0x7f5c0819bb40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b3d0 tx=0x7f5bfc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5bfc011630 con 0x7f5c081977b0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0819c080 con 0x7f5c081977b0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bfc011c80 con 0x7f5c081977b0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5bfc010d80 con 0x7f5c081977b0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.818+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0819c5c0 con 0x7f5c081977b0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.819+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c08102db0 con 0x7f5c081977b0 2024-09-19T04:58:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.819+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f5bfc018020 con 0x7f5c081977b0 2024-09-19T04:58:01.823 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.819+0000 7f5bfb7fe640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5bf4039040 0x7f5bf403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:01.823 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.819+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5bfc04bf90 con 0x7f5c081977b0 2024-09-19T04:58:01.823 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.819+0000 7f5c0e201640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5bf4039040 0x7f5bf403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:01.823 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.820+0000 7f5c0e201640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5bf4039040 0x7f5bf403b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c040314f0 tx=0x7f5c04031420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:01.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:01.824+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5bfc01f020 con 0x7f5c081977b0 2024-09-19T04:58:02.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.053+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5c0819c8d0 con 0x7f5c081977b0 2024-09-19T04:58:02.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.053+0000 7f5bfb7fe640 1 -- 172.21.15.174:0/3333501218 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5bfc0189a0 con 0x7f5c081977b0 2024-09-19T04:58:02.055 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:58:02.055 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:57:53.613049Z","created":"2024-09-19T04:56:20.817470Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:58:02.055 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5bf4039040 msgr2=0x7f5bf403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5bf4039040 0x7f5bf403b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c040314f0 tx=0x7f5c04031420 comp rx=0 tx=0).stop 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c081977b0 msgr2=0x7f5c0819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c081977b0 0x7f5c0819bb40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b3d0 tx=0x7f5bfc00b890 comp rx=0 tx=0).stop 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 shutdown_connections 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5bf4039040 0x7f5bf403b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c081977b0 0x7f5c0819bb40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:02.058 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 --2- 172.21.15.174:0/3333501218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c08107670 0x7f5c08197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:02.058 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 >> 172.21.15.174:0/3333501218 conn(0x7f5c080fd330 msgr2=0x7f5c080fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:02.058 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 shutdown_connections 2024-09-19T04:58:02.058 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:02.055+0000 7f5c10465640 1 -- 172.21.15.174:0/3333501218 wait complete. 2024-09-19T04:58:02.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:01 smithi064 ceph-mon[30960]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:02.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:02 smithi103 systemd[1]: Started Ceph mon.b for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:58:03.654 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:58:03.654 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:58:03.903 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.331+0000 7fef86fc2640 1 -- 172.21.15.174:0/331115164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c003530 msgr2=0x7fef6c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.331+0000 7fef86fc2640 1 --2- 172.21.15.174:0/331115164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c003530 0x7fef6c0059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef80102b00 tx=0x7fef7802bce0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.331+0000 7fef86fc2640 1 -- 172.21.15.174:0/331115164 shutdown_connections 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.331+0000 7fef86fc2640 1 --2- 172.21.15.174:0/331115164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef6c003530 0x7fef6c0059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.331+0000 7fef86fc2640 1 -- 172.21.15.174:0/331115164 >> 172.21.15.174:0/331115164 conn(0x7fef800fd330 msgr2=0x7fef800ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.332+0000 7fef86fc2640 1 -- 172.21.15.174:0/331115164 shutdown_connections 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.332+0000 7fef86fc2640 1 -- 172.21.15.174:0/331115164 wait complete. 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.332+0000 7fef86fc2640 1 Processor -- start 2024-09-19T04:58:07.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.332+0000 7fef86fc2640 1 -- start start 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef86fc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 0x7fef800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef86fc2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef86fc2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef801a3520 0x7fef801a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef86fc2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fef801095e0 con 0x7fef801a3520 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef86fc2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef80109460 con 0x7fef6c003530 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef86fc2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef80109760 con 0x7fef80074a30 2024-09-19T04:58:07.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55178/0 (socket says 172.21.15.174:55178) 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 -- 172.21.15.174:0/3434699858 learned_addr learned my addr 172.21.15.174:0/3434699858 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef8555f640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef801a3520 0x7fef801a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef84d5e640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 0x7fef800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 msgr2=0x7fef80074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 msgr2=0x7fef80074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef7ffff640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef84d5e640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 msgr2=0x7fef800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef84d5e640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 msgr2=0x7fef800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:58:07.337 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef84d5e640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 0x7fef800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef84d5e640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 0x7fef800744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef8555f640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 msgr2=0x7fef80074e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef8555f640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef8555f640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 msgr2=0x7fef800744f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef8555f640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 0x7fef800744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.333+0000 7fef8555f640 1 -- 172.21.15.174:0/3434699858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801a5fe0 con 0x7fef801a3520 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.334+0000 7fef8555f640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef801a3520 0x7fef801a5910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef68009fd0 tx=0x7fef6800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.334+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68015020 con 0x7fef801a3520 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.334+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef68011ce0 con 0x7fef801a3520 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.334+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801a62d0 con 0x7fef801a3520 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.334+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68010e40 con 0x7fef801a3520 2024-09-19T04:58:07.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.334+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a6810 con 0x7fef801a3520 2024-09-19T04:58:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.335+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fef68018020 con 0x7fef801a3520 2024-09-19T04:58:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.335+0000 7fef7dffb640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef60039730 0x7fef6003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:07.339 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.335+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fef6804bef0 con 0x7fef801a3520 2024-09-19T04:58:07.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.336+0000 7fef84d5e640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef60039730 0x7fef6003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:07.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.336+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8010a710 con 0x7fef801a3520 2024-09-19T04:58:07.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.336+0000 7fef84d5e640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef60039730 0x7fef6003bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fef78005940 tx=0x7fef780058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:07.342 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.341+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef6801f020 con 0x7fef801a3520 2024-09-19T04:58:07.585 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.583+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fef8002d7c0 con 0x7fef801a3520 2024-09-19T04:58:07.585 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: mon.c calling monitor election 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: mon.a calling monitor election 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:07.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: monmap epoch 3 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: last_changed 2024-09-19T04:58:02.299203+0000 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: min_mon_release 19 (squid) 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: election_strategy: 1 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: fsmap 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:58:07.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: mgrmap e13: x(active, since 49s) 2024-09-19T04:58:07.588 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: overall HEALTH_OK 2024-09-19T04:58:07.588 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.588 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.588 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.588 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.588 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.584+0000 7fef7dffb640 1 -- 172.21.15.174:0/3434699858 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fef68018310 con 0x7fef801a3520 2024-09-19T04:58:07.588 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:58:07.588 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:58:02.299203Z","created":"2024-09-19T04:56:20.817470Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:58:07.589 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-09-19T04:58:07.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.586+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef60039730 msgr2=0x7fef6003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:07.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.586+0000 7fef86fc2640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef60039730 0x7fef6003bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fef78005940 tx=0x7fef780058b0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.586+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef801a3520 msgr2=0x7fef801a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.586+0000 7fef86fc2640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef801a3520 0x7fef801a5910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef68009fd0 tx=0x7fef6800b870 comp rx=0 tx=0).stop 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 shutdown_connections 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fef60039730 0x7fef6003bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef801a3520 0x7fef801a5910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef80074a30 0x7fef80074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 --2- 172.21.15.174:0/3434699858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef6c003530 0x7fef800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 >> 172.21.15.174:0/3434699858 conn(0x7fef800fd330 msgr2=0x7fef80106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 shutdown_connections 2024-09-19T04:58:07.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:07.587+0000 7fef86fc2640 1 -- 172.21.15.174:0/3434699858 wait complete. 2024-09-19T04:58:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: mon.c calling monitor election 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: mon.a calling monitor election 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: monmap epoch 3 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: last_changed 2024-09-19T04:58:02.299203+0000 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: min_mon_release 19 (squid) 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: election_strategy: 1 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-19T04:58:07.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: fsmap 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: mgrmap e13: x(active, since 49s) 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: overall HEALTH_OK 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:07.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:08.163 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T04:58:08.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T04:58:08.428 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:58:08.433 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:08.433 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:08 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/3434699858' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:58:08.433 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:08.433 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:08.549 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:08.549 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:08 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/3434699858' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:58:08.549 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:08.549 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 -- 172.21.15.64:0/1849263015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad01003d0 msgr2=0x7f0ad01007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 --2- 172.21.15.64:0/1849263015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad01003d0 0x7f0ad01007b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009880 tx=0x7f0acc02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:58:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 -- 172.21.15.64:0/1849263015 shutdown_connections 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 --2- 172.21.15.64:0/1849263015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad01003d0 0x7f0ad01007b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 -- 172.21.15.64:0/1849263015 >> 172.21.15.64:0/1849263015 conn(0x7f0ad00fc280 msgr2=0x7f0ad00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 -- 172.21.15.64:0/1849263015 shutdown_connections 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.846+0000 7f0ad7811640 1 -- 172.21.15.64:0/1849263015 wait complete. 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 Processor -- start 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 -- start start 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 0x7f0ad0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ad0199bc0 0x7f0ad019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 0x7f0ad019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad010de40 con 0x7f0ad019e490 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad010dcc0 con 0x7f0ad01003d0 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad7811640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad010dfc0 con 0x7f0ad0199bc0 2024-09-19T04:58:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad5dae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 0x7f0ad019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad5dae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 0x7f0ad019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53024/0 (socket says 172.21.15.64:53024) 2024-09-19T04:58:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad4dac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ad0199bc0 0x7f0ad019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad5dae640 1 -- 172.21.15.64:0/2111397997 learned_addr learned my addr 172.21.15.64:0/2111397997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:58:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.847+0000 7f0ad55ad640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 0x7f0ad0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad4dac640 1 -- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 msgr2=0x7f0ad0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad55ad640 1 -- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 msgr2=0x7f0ad0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad55ad640 1 -- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 msgr2=0x7f0ad0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad55ad640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 0x7f0ad0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad55ad640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 0x7f0ad0199680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad4dac640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 0x7f0ad0199680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad4dac640 1 -- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 msgr2=0x7f0ad019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad4dac640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 0x7f0ad019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad4dac640 1 -- 172.21.15.64:0/2111397997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad01a6200 con 0x7f0ad0199bc0 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad5dae640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 0x7f0ad019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad4dac640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ad0199bc0 0x7f0ad019df50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000b450 tx=0x7f0ac000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0004230 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac00043d0 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0003be0 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.848+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad01a6430 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.849+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad01a6890 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.849+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0ac000f050 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.850+0000 7f0ac67fc640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0abc039730 0x7f0abc03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.850+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0ac004bff0 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.850+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad0100830 con 0x7f0ad0199bc0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.850+0000 7f0ad55ad640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0abc039730 0x7f0abc03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.850+0000 7f0ad55ad640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0abc039730 0x7f0abc03bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0acc031440 tx=0x7f0acc02aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:08.855+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ac000d040 con 0x7f0ad0199bc0 2024-09-19T04:58:09.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.032+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0ad01a6be0 con 0x7f0ad0199bc0 2024-09-19T04:58:09.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.033+0000 7f0ac67fc640 1 -- 172.21.15.64:0/2111397997 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f0ac000f340 con 0x7f0ad0199bc0 2024-09-19T04:58:09.035 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:09.036 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-09-19T04:58:09.036 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:09.036 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-19T04:58:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0abc039730 msgr2=0x7f0abc03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0abc039730 0x7f0abc03bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0acc031440 tx=0x7f0acc02aad0 comp rx=0 tx=0).stop 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ad0199bc0 msgr2=0x7f0ad019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ad0199bc0 0x7f0ad019df50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000b450 tx=0x7f0ac000b910 comp rx=0 tx=0).stop 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 shutdown_connections 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0abc039730 0x7f0abc03bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad019e490 0x7f0ad019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ad0199bc0 0x7f0ad019df50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 --2- 172.21.15.64:0/2111397997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ad01003d0 0x7f0ad0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 >> 172.21.15.64:0/2111397997 conn(0x7f0ad00fc280 msgr2=0x7f0ad00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 shutdown_connections 2024-09-19T04:58:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:09.035+0000 7f0ad7811640 1 -- 172.21.15.64:0/2111397997 wait complete. 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.c calling monitor election 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.a calling monitor election 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: monmap epoch 3 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: last_changed 2024-09-19T04:58:02.299203+0000 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: min_mon_release 19 (squid) 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: election_strategy: 1 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:58:09.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: fsmap 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mgrmap e13: x(active, since 49s) 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: overall HEALTH_OK 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/3434699858' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:09.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:09.766 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T04:58:09.767 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:58:09.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:58:09.796 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:58:09.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:58:09.863 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:58:09.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:58:09.890 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:58:09.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:58:09.955 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:58:09.955 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:58:09.983 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:58:09.983 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:58:10.049 INFO:tasks.cephadm:Adding mgr.x on smithi064 2024-09-19T04:58:10.049 INFO:tasks.cephadm:Adding mgr.y on smithi103 2024-09-19T04:58:10.049 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-09-19T04:58:10.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi064=x;smithi103=y;smithi174=z' 2024-09-19T04:58:10.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: mon.b calling monitor election 2024-09-19T04:58:10.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: mon.b calling monitor election 2024-09-19T04:58:10.111 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: mon.c calling monitor election 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: mon.a calling monitor election 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: monmap epoch 3 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: last_changed 2024-09-19T04:58:02.299203+0000 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: min_mon_release 19 (squid) 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: election_strategy: 1 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-19T04:58:10.112 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: fsmap 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: mgrmap e13: x(active, since 51s) 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: overall HEALTH_OK 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:10.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:10.148 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: mon.b calling monitor election 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: mon.b calling monitor election 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: mon.c calling monitor election 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: mon.a calling monitor election 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: monmap epoch 3 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: last_changed 2024-09-19T04:58:02.299203+0000 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: min_mon_release 19 (squid) 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: election_strategy: 1 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:58:10.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: fsmap 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: mgrmap e13: x(active, since 51s) 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: overall HEALTH_OK 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:10.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.b calling monitor election 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.b calling monitor election 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.c calling monitor election 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.a calling monitor election 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: monmap epoch 3 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: last_changed 2024-09-19T04:58:02.299203+0000 2024-09-19T04:58:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: created 2024-09-19T04:56:20.817470+0000 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: min_mon_release 19 (squid) 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: election_strategy: 1 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: fsmap 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: mgrmap e13: x(active, since 51s) 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: overall HEALTH_OK 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:10.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:10.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:10.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:10.325 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:58:10.730 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 -- 172.21.15.174:0/532230846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 msgr2=0x7f50f41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:10.730 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/532230846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 0x7f50f41009f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009880 tx=0x7f50ec02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 -- 172.21.15.174:0/532230846 shutdown_connections 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/532230846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 0x7f50f41009f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 -- 172.21.15.174:0/532230846 >> 172.21.15.174:0/532230846 conn(0x7f50f40fbda0 msgr2=0x7f50f40fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 -- 172.21.15.174:0/532230846 shutdown_connections 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.700+0000 7f50f8e3b640 1 -- 172.21.15.174:0/532230846 wait complete. 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 Processor -- start 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 -- start start 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 0x7f50f4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 0x7f50f419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f4199ac0 0x7f50f41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4105470 con 0x7f50f41005f0 2024-09-19T04:58:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41052f0 con 0x7f50f4199ac0 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f8e3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41055f0 con 0x7f50f4199640 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.701+0000 7f50f1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 0x7f50f419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 0x7f50f419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55180/0 (socket says 172.21.15.174:55180) 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 -- 172.21.15.174:0/2926492935 learned_addr learned my addr 172.21.15.174:0/2926492935 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f259c640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 0x7f50f4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f2d9d640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f4199ac0 0x7f50f41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 -- 172.21.15.174:0/2926492935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f4199ac0 msgr2=0x7f50f41a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f4199ac0 0x7f50f41a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 -- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 msgr2=0x7f50f4199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 0x7f50f4199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 -- 172.21.15.174:0/2926492935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41a1500 con 0x7f50f4199640 2024-09-19T04:58:10.732 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f1d9b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 0x7f50f419e6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b520 tx=0x7f50dc00b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc004230 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41a17f0 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50dc0043d0 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003cf0 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.702+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a1db0 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.703+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f50dc00e070 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.703+0000 7f50eb7fe640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f50e0039720 0x7f50e003bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.703+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4192640 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.703+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f50dc04bd90 con 0x7f50f4199640 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.703+0000 7f50f259c640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f50e0039720 0x7f50e003bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.704+0000 7f50f259c640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f50e0039720 0x7f50e003bbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0314e0 tx=0x7f50ec005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:10.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.708+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50dc014020 con 0x7f50f4199640 2024-09-19T04:58:10.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.888+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}) -- 0x7f50f40626c0 con 0x7f50e0039720 2024-09-19T04:58:10.897 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.896+0000 7f50eb7fe640 1 -- 172.21.15.174:0/2926492935 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f50f40626c0 con 0x7f50e0039720 2024-09-19T04:58:10.897 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-09-19T04:58:10.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f50e0039720 msgr2=0x7f50e003bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f50e0039720 0x7f50e003bbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0314e0 tx=0x7f50ec005bd0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 msgr2=0x7f50f419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 0x7f50f419e6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00b520 tx=0x7f50dc00b9e0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 shutdown_connections 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f50e0039720 0x7f50e003bbe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50f4199ac0 0x7f50f41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4199640 0x7f50f419e6b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 --2- 172.21.15.174:0/2926492935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50f41005f0 0x7f50f4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 >> 172.21.15.174:0/2926492935 conn(0x7f50f40fbda0 msgr2=0x7f50f40fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 shutdown_connections 2024-09-19T04:58:10.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:58:10.898+0000 7f50f8e3b640 1 -- 172.21.15.174:0/2926492935 wait complete. 2024-09-19T04:58:11.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:10 smithi174 ceph-mon[34488]: Reconfiguring mon.a (monmap changed)... 2024-09-19T04:58:11.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:10 smithi174 ceph-mon[34488]: Reconfiguring daemon mon.a on smithi064 2024-09-19T04:58:11.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:10 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:11.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:10 smithi064 ceph-mon[30960]: Reconfiguring mon.a (monmap changed)... 2024-09-19T04:58:11.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:10 smithi064 ceph-mon[30960]: Reconfiguring daemon mon.a on smithi064 2024-09-19T04:58:11.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:10 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:11.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:10 smithi103 ceph-mon[34446]: Reconfiguring mon.a (monmap changed)... 2024-09-19T04:58:11.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:10 smithi103 ceph-mon[34446]: Reconfiguring daemon mon.a on smithi064 2024-09-19T04:58:11.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:10 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:58:11.513 DEBUG:teuthology.orchestra.run.smithi103:mgr.y> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T04:58:11.516 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T04:58:11.556 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T04:58:11.556 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T04:58:11.557 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:58:11.574 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:58:11.574 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-09-19T04:58:11.580 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T04:58:11.298+0000 7f85475a2640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T04:58:11.594 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 04:57:27.315662404 +0000 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 04:53:16.749842248 +0000 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 04:53:16.749842248 +0000 2024-09-19T04:58:11.595 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 04:53:16.749842248 +0000 2024-09-19T04:58:11.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:58:11.661 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T04:58:11.661 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T04:58:11.661 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00026366 s, 1.9 MB/s 2024-09-19T04:58:11.663 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:58:11.720 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-09-19T04:58:11.775 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 04:57:27.357661617 +0000 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 04:53:17.077836164 +0000 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 04:53:17.077836164 +0000 2024-09-19T04:58:11.776 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 04:53:17.077836164 +0000 2024-09-19T04:58:11.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:58:11.840 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T04:58:11.840 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T04:58:11.840 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000233513 s, 2.2 MB/s 2024-09-19T04:58:11.841 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:58:11.899 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 04:57:27.398660849 +0000 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 04:53:17.400830174 +0000 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 04:53:17.400830174 +0000 2024-09-19T04:58:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 04:53:17.400830174 +0000 2024-09-19T04:58:11.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:58:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T04:58:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T04:58:12.019 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000185526 s, 2.8 MB/s 2024-09-19T04:58:12.020 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:58:12.077 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-09-19T04:58:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 04:57:27.432660212 +0000 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 04:53:17.729824071 +0000 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 04:53:17.729824071 +0000 2024-09-19T04:58:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-19 04:53:17.729824071 +0000 2024-09-19T04:58:12.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:58:12.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:12.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:12.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: Saving service mgr spec with placement smithi064=x;smithi103=y;smithi174=z;count:3 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:12.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:11 smithi064 ceph-mon[30960]: Reconfiguring daemon mgr.x on smithi064 2024-09-19T04:58:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-19T04:58:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-19T04:58:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000181962 s, 2.8 MB/s 2024-09-19T04:58:12.175 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:58:12.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: Saving service mgr spec with placement smithi064=x;smithi103=y;smithi174=z;count:3 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:12.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:11 smithi103 ceph-mon[34446]: Reconfiguring daemon mgr.x on smithi064 2024-09-19T04:58:12.232 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T04:58:12.232 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:58:12.251 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:58:12.252 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-19T04:58:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: Saving service mgr spec with placement smithi064=x;smithi103=y;smithi174=z;count:3 2024-09-19T04:58:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:11 smithi174 ceph-mon[34488]: Reconfiguring daemon mgr.x on smithi064 2024-09-19T04:58:12.310 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:58:12.310 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 04:58:01.750711684 +0000 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 04:53:14.136859068 +0000 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 04:53:14.136859068 +0000 2024-09-19T04:58:12.311 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 04:53:14.136859068 +0000 2024-09-19T04:58:12.311 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:58:12.377 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T04:58:12.378 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T04:58:12.378 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.00020286 s, 2.5 MB/s 2024-09-19T04:58:12.379 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:58:12.437 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-19T04:58:12.492 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:58:12.492 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:12.492 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T04:58:12.493 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:12.493 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:12.493 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 04:58:01.751711664 +0000 2024-09-19T04:58:12.493 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 04:53:14.473852287 +0000 2024-09-19T04:58:12.493 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 04:53:14.473852287 +0000 2024-09-19T04:58:12.493 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 04:53:14.473852287 +0000 2024-09-19T04:58:12.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:58:12.556 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T04:58:12.557 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T04:58:12.557 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000182679 s, 2.8 MB/s 2024-09-19T04:58:12.558 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:58:12.615 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 04:58:01.751711664 +0000 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 04:53:14.799845728 +0000 2024-09-19T04:58:12.672 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 04:53:14.799845728 +0000 2024-09-19T04:58:12.673 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 04:53:14.799845728 +0000 2024-09-19T04:58:12.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:58:12.735 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T04:58:12.735 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T04:58:12.735 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000195883 s, 2.6 MB/s 2024-09-19T04:58:12.736 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:58:12.794 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-19T04:58:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:58:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 04:58:01.751711664 +0000 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 04:53:15.130839067 +0000 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 04:53:15.130839067 +0000 2024-09-19T04:58:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 04:53:15.130839067 +0000 2024-09-19T04:58:12.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:58:12.915 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T04:58:12.916 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T04:58:12.916 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000187765 s, 2.7 MB/s 2024-09-19T04:58:12.917 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:58:12.973 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T04:58:12.974 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:58:12.994 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:58:12.994 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 04:57:52.640401284 +0000 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 04:53:14.335152396 +0000 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 04:53:14.335152396 +0000 2024-09-19T04:58:13.053 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 04:53:14.335152396 +0000 2024-09-19T04:58:13.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:58:13.121 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T04:58:13.121 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T04:58:13.121 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000262934 s, 1.9 MB/s 2024-09-19T04:58:13.122 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:58:13.181 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 04:57:52.640401284 +0000 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 04:53:14.676147735 +0000 2024-09-19T04:58:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 04:53:14.676147735 +0000 2024-09-19T04:58:13.239 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 04:53:14.676147735 +0000 2024-09-19T04:58:13.239 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:58:13.303 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T04:58:13.303 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T04:58:13.303 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000203582 s, 2.5 MB/s 2024-09-19T04:58:13.304 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:58:13.361 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-09-19T04:58:13.417 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:58:13.417 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:13.417 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T04:58:13.418 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:13.418 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:13.418 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 04:57:52.640401284 +0000 2024-09-19T04:58:13.418 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 04:53:15.002143277 +0000 2024-09-19T04:58:13.418 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 04:53:15.002143277 +0000 2024-09-19T04:58:13.418 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 04:53:15.002143277 +0000 2024-09-19T04:58:13.418 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:58:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:58:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:13.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:13.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:13.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:13 smithi174 ceph-mon[34488]: Reconfiguring daemon mon.b on smithi103 2024-09-19T04:58:13.481 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T04:58:13.481 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T04:58:13.481 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000193379 s, 2.6 MB/s 2024-09-19T04:58:13.483 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:58:13.540 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-09-19T04:58:13.595 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 04:57:52.641401270 +0000 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 04:53:15.347138559 +0000 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 04:53:15.347138559 +0000 2024-09-19T04:58:13.596 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 04:53:15.347138559 +0000 2024-09-19T04:58:13.597 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:13 smithi064 ceph-mon[30960]: Reconfiguring daemon mon.b on smithi103 2024-09-19T04:58:13.658 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T04:58:13.658 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T04:58:13.659 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000188948 s, 2.7 MB/s 2024-09-19T04:58:13.660 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:58:13.709 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:13.710 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:13 smithi103 ceph-mon[34446]: Reconfiguring daemon mon.b on smithi103 2024-09-19T04:58:13.716 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2024-09-19T04:58:13.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:58:13.976 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:58:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:58:14.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:14.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:14.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:14.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:14 smithi064 ceph-mon[30960]: Reconfiguring daemon mon.c on smithi174 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:14.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:14 smithi174 ceph-mon[34488]: Reconfiguring daemon mon.c on smithi174 2024-09-19T04:58:14.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:14.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:14.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:58:14.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:58:14.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:58:14.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:14.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:14 smithi103 ceph-mon[34446]: Reconfiguring daemon mon.c on smithi174 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:58:16.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:15 smithi103 ceph-mon[34446]: Deploying daemon mgr.z on smithi174 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:16.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:15 smithi174 ceph-mon[34488]: Deploying daemon mgr.z on smithi174 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:16.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:58:16.256 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:16.256 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.256 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:15 smithi064 ceph-mon[30960]: Deploying daemon mgr.z on smithi174 2024-09-19T04:58:16.258 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:58:16.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-09-19T04:58:16.540 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:58:16.777 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:16 smithi174 systemd[1]: Starting Ceph mgr.z for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T04:58:16.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 -- 172.21.15.64:0/616372700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434107960 msgr2=0x7ff434109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:16.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 --2- 172.21.15.64:0/616372700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434107960 0x7ff434109d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009900 tx=0x7ff42c02a410 comp rx=0 tx=0).stop 2024-09-19T04:58:16.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 -- 172.21.15.64:0/616372700 shutdown_connections 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 --2- 172.21.15.64:0/616372700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434107960 0x7ff434109d50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 --2- 172.21.15.64:0/616372700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434105030 0x7ff434107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 --2- 172.21.15.64:0/616372700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434102700 0x7ff434104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 -- 172.21.15.64:0/616372700 >> 172.21.15.64:0/616372700 conn(0x7ff4340fc2a0 msgr2=0x7ff4340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 -- 172.21.15.64:0/616372700 shutdown_connections 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 -- 172.21.15.64:0/616372700 wait complete. 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.952+0000 7ff439fa5640 1 Processor -- start 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 -- start start 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 0x7ff434197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434105030 0x7ff4341983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:16.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434107960 0x7ff43419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d7d0 con 0x7ff434102700 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d650 con 0x7ff434105030 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff439fa5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d950 con 0x7ff434107960 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 0x7ff434197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 0x7ff434197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37404/0 (socket says 172.21.15.64:37404) 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 -- 172.21.15.64:0/2141573877 learned_addr learned my addr 172.21.15.64:0/2141573877 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff433fff640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434107960 0x7ff43419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff432ffd640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434105030 0x7ff4341983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 -- 172.21.15.64:0/2141573877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434107960 msgr2=0x7ff43419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434107960 0x7ff43419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 -- 172.21.15.64:0/2141573877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434105030 msgr2=0x7ff4341983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:16.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434105030 0x7ff4341983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff4337fe640 1 -- 172.21.15.64:0/2141573877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43419fab0 con 0x7ff434102700 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff432ffd640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434105030 0x7ff4341983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.953+0000 7ff433fff640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434107960 0x7ff43419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.954+0000 7ff4337fe640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 0x7ff434197e80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff42400ba50 tx=0x7ff42400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.954+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424019070 con 0x7ff434102700 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.954+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff42400cbe0 con 0x7ff434102700 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.954+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4240075d0 con 0x7ff434102700 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.954+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43419fda0 con 0x7ff434102700 2024-09-19T04:58:16.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.954+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a0280 con 0x7ff434102700 2024-09-19T04:58:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.955+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff42401c070 con 0x7ff434102700 2024-09-19T04:58:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.955+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4340fdd90 con 0x7ff434102700 2024-09-19T04:58:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.955+0000 7ff430ff9640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff41c039500 0x7ff41c03b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.955+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff42404bdf0 con 0x7ff434102700 2024-09-19T04:58:16.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.955+0000 7ff432ffd640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff41c039500 0x7ff41c03b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:16.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.955+0000 7ff432ffd640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff41c039500 0x7ff41c03b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4341993a0 tx=0x7ff428005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:16.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:16.960+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff424012500 con 0x7ff434102700 2024-09-19T04:58:17.118 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:16 smithi174 podman[36272]: 2024-09-19 04:58:16.75980125 +0000 UTC m=+0.015529131 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:17.118 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:16 smithi174 podman[36272]: 2024-09-19 04:58:16.870767495 +0000 UTC m=+0.126495428 container create a3c9f68e21d3f7ec2302928f88f5677015a76863fab310941c740aa35dd78a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T04:58:17.118 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 podman[36272]: 2024-09-19 04:58:17.089481543 +0000 UTC m=+0.345209409 container init a3c9f68e21d3f7ec2302928f88f5677015a76863fab310941c740aa35dd78a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T04:58:17.118 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 podman[36272]: 2024-09-19 04:58:17.092969777 +0000 UTC m=+0.348697635 container start a3c9f68e21d3f7ec2302928f88f5677015a76863fab310941c740aa35dd78a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T04:58:17.118 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mgr[36292]: -- 172.21.15.174:0/2599655900 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddb75ad0e0 con 0x55ddb758b400 2024-09-19T04:58:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:17.146+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff434062630 con 0x7ff41c039500 2024-09-19T04:58:17.421 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 bash[36272]: a3c9f68e21d3f7ec2302928f88f5677015a76863fab310941c740aa35dd78a92 2024-09-19T04:58:17.421 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 systemd[1]: Started Ceph mgr.z for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:58:17.421 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:17.214+0000 7f8435b52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:58:17.421 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:17.274+0000 7f8435b52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:58:17.421 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:17.420+0000 7f8435b52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:58:17.771 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:17.486+0000 7f8435b52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:58:17.771 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:17.633+0000 7f8435b52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:58:17.771 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:17.770+0000 7f8435b52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:58:18.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:18.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:18.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:17 smithi174 ceph-mon[34488]: Deploying daemon mgr.y on smithi103 2024-09-19T04:58:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:18.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:17 smithi064 ceph-mon[30960]: Deploying daemon mgr.y on smithi103 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:18.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:58:18.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:58:18.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:58:18.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:18.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:17 smithi103 ceph-mon[34446]: Deploying daemon mgr.y on smithi103 2024-09-19T04:58:18.527 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:18.085+0000 7f8435b52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:58:18.527 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:18.144+0000 7f8435b52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:58:18.527 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:18.327+0000 7f8435b52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:58:19.219 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 systemd[1]: Starting Ceph mgr.y for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T04:58:19.522 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 podman[35791]: 2024-09-19 04:58:19.125083486 +0000 UTC m=+0.015361697 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:58:19.522 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 podman[35791]: 2024-09-19 04:58:19.2360669 +0000 UTC m=+0.126345096 container create 445aed7eeb37a82c0726fa5453b00ec472562c6b16d4cac1b1cde825ef45612f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T04:58:19.522 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 podman[35791]: 2024-09-19 04:58:19.438321923 +0000 UTC m=+0.328600125 container init 445aed7eeb37a82c0726fa5453b00ec472562c6b16d4cac1b1cde825ef45612f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T04:58:19.522 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 podman[35791]: 2024-09-19 04:58:19.4410943 +0000 UTC m=+0.331372497 container start 445aed7eeb37a82c0726fa5453b00ec472562c6b16d4cac1b1cde825ef45612f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T04:58:19.522 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mgr[35811]: -- 172.21.15.103:0/2474123702 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3776dd0e0 con 0x55e3776ba800 2024-09-19T04:58:19.523 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 bash[35791]: 445aed7eeb37a82c0726fa5453b00ec472562c6b16d4cac1b1cde825ef45612f 2024-09-19T04:58:19.527 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:19.205+0000 7f8435b52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:58:19.527 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:19.271+0000 7f8435b52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:58:19.784 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 systemd[1]: Started Ceph mgr.y for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T04:58:19.785 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:19.568+0000 7f5ecbc23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:58:19.785 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:19.628+0000 7f5ecbc23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:58:19.785 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:19.783+0000 7f5ecbc23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:58:20.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:20.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2586556415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee9cdd-278a-45e4-8b61-5fa0991e9841"}]: dispatch 2024-09-19T04:58:20.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2586556415' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ee9cdd-278a-45e4-8b61-5fa0991e9841"}]': finished 2024-09-19T04:58:20.077 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:58:20.077 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:20.077 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.077 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.077 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.077 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.078 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:20.174 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:19.853+0000 7f5ecbc23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:58:20.174 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:20.012+0000 7f5ecbc23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:58:20.174 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:20.173+0000 7f5ecbc23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:58:20.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:20.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2586556415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee9cdd-278a-45e4-8b61-5fa0991e9841"}]: dispatch 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2586556415' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ee9cdd-278a-45e4-8b61-5fa0991e9841"}]': finished 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:20.277 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.014+0000 7f8435b52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:58:20.277 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.143+0000 7f8435b52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2586556415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee9cdd-278a-45e4-8b61-5fa0991e9841"}]: dispatch 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2586556415' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ee9cdd-278a-45e4-8b61-5fa0991e9841"}]': finished 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:20.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:20.579 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.287+0000 7f8435b52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:58:20.580 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.350+0000 7f8435b52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:58:20.580 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.511+0000 7f8435b52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:58:20.580 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.579+0000 7f8435b52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:58:20.718 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:20.501+0000 7f5ecbc23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:58:20.718 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:20.564+0000 7f5ecbc23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:58:20.873 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.641+0000 7f8435b52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:58:20.873 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.800+0000 7f8435b52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:58:20.873 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.872+0000 7f8435b52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:58:21.143 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:20.939+0000 7f8435b52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:58:21.148 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:21.025+0000 7f8435b52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:58:21.148 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:21.084+0000 7f8435b52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:58:21.148 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:21.142+0000 7f8435b52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1836865185' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:21.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.218 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:20.759+0000 7f5ecbc23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1836865185' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1836865185' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:21.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:21.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:21.944 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:21.648+0000 7f5ecbc23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:58:21.944 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:21.714+0000 7f5ecbc23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:58:22.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:22.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: Standby manager daemon z started 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:58:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:21 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:58:22.277 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T04:58:21.926+0000 7f8435b52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:58:22.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: Standby manager daemon z started 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:58:22.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:21 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:58:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:22.368+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7ff42401d400 con 0x7ff434102700 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: Standby manager daemon z started 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:58:22.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:58:22.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:21 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/1393508830' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:58:22.681 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:22.426+0000 7f5ecbc23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:58:22.681 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:22.540+0000 7f5ecbc23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:58:22.681 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:22.680+0000 7f5ecbc23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:58:22.957 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:22.741+0000 7f5ecbc23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:58:22.957 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:22.888+0000 7f5ecbc23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:58:22.957 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:22.956+0000 7f5ecbc23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:58:23.218 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.015+0000 7f5ecbc23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:58:23.218 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.172+0000 7f5ecbc23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:58:23.505 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-mon[34446]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:23.505 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-mon[34446]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-19T04:58:23.505 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:58:23.505 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:23.505 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.238+0000 7f5ecbc23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:58:23.506 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.304+0000 7f5ecbc23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:58:23.506 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.387+0000 7f5ecbc23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:58:23.506 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.445+0000 7f5ecbc23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:58:23.506 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:23.504+0000 7f5ecbc23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:58:23.637 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:23 smithi064 ceph-mon[30960]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:23.637 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:23 smithi064 ceph-mon[30960]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-19T04:58:23.637 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:58:23.637 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:23 smithi174 ceph-mon[34488]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:23 smithi174 ceph-mon[34488]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-19T04:58:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:58:23.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:24.378+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7ff4240133a0 con 0x7ff434102700 2024-09-19T04:58:24.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:24 smithi064 ceph-mon[30960]: Standby manager daemon y started 2024-09-19T04:58:24.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:24 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:58:24.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:24 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:58:24.625 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:24 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:58:24.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:24 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:58:24.718 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 04:58:24 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T04:58:24.282+0000 7f5ecbc23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:58:24.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:24 smithi103 ceph-mon[34446]: Standby manager daemon y started 2024-09-19T04:58:24.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:24 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:58:24.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:24 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:58:24.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:24 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:58:24.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:24 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:58:24.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:24 smithi174 ceph-mon[34488]: Standby manager daemon y started 2024-09-19T04:58:24.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:24 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:58:24.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:24 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:58:24.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:24 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:58:24.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:24 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/4198836910' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:58:25.548 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:25 smithi064 ceph-mon[30960]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:25.548 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:25 smithi064 ceph-mon[30960]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T04:58:25.548 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:58:25.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:25 smithi103 ceph-mon[34446]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:25.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:25 smithi103 ceph-mon[34446]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T04:58:25.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:58:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:25 smithi174 ceph-mon[34488]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:25 smithi174 ceph-mon[34488]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-19T04:58:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:58:27.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:27 smithi064 ceph-mon[30960]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:27 smithi103 ceph-mon[34446]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:27.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:27 smithi174 ceph-mon[34488]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:58:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:28 smithi064 ceph-mon[30960]: Deploying daemon osd.0 on smithi064 2024-09-19T04:58:28.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:58:28.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:28.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:28 smithi103 ceph-mon[34446]: Deploying daemon osd.0 on smithi064 2024-09-19T04:58:28.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:58:28.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:28.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:28 smithi174 ceph-mon[34488]: Deploying daemon osd.0 on smithi064 2024-09-19T04:58:29.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:29 smithi103 ceph-mon[34446]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:29 smithi064 ceph-mon[30960]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:29.678 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:29 smithi174 ceph-mon[34488]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:31 smithi064 ceph-mon[30960]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:31 smithi103 ceph-mon[34446]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:31 smithi174 ceph-mon[34488]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:33 smithi064 ceph-mon[30960]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:33.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:33 smithi103 ceph-mon[34446]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:33.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:33 smithi174 ceph-mon[34488]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:35 smithi064 ceph-mon[30960]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:35 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:35 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:35 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:35.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:35 smithi103 ceph-mon[34446]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:35.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:35 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:35 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:35 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:35 smithi174 ceph-mon[34488]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:35 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:35.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:35 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:35.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:35 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:38 smithi064 ceph-mon[30960]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:38.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:38.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:38.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:38 smithi103 ceph-mon[34446]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:38 smithi174 ceph-mon[34488]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:38.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:38.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.710+0000 7ff430ff9640 1 -- 172.21.15.64:0/2141573877 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff434062630 con 0x7ff41c039500 2024-09-19T04:58:38.712 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2024-09-19T04:58:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff41c039500 msgr2=0x7ff41c03b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff41c039500 0x7ff41c03b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4341993a0 tx=0x7ff428005e50 comp rx=0 tx=0).stop 2024-09-19T04:58:38.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 msgr2=0x7ff434197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 0x7ff434197e80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff42400ba50 tx=0x7ff42400bf10 comp rx=0 tx=0).stop 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 shutdown_connections 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff41c039500 0x7ff41c03b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff434107960 0x7ff43419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff434105030 0x7ff4341983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 --2- 172.21.15.64:0/2141573877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff434102700 0x7ff434197e80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 >> 172.21.15.64:0/2141573877 conn(0x7ff4340fc2a0 msgr2=0x7ff4341030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 shutdown_connections 2024-09-19T04:58:38.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:58:38.713+0000 7ff439fa5640 1 -- 172.21.15.64:0/2141573877 wait complete. 2024-09-19T04:58:39.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:39.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:39.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:39.279 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T04:58:39.282 INFO:tasks.cephadm:Deploying osd.1 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-19T04:58:39.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:58:39.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:39.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:39.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:39.523 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:58:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:40.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:40 smithi103 ceph-mon[34446]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:40.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:40 smithi064 ceph-mon[30960]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:40.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:40 smithi174 ceph-mon[34488]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:41.128 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:58:40 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0[42571]: 2024-09-19T04:58:40.813+0000 7f122a861640 -1 osd.0 0 log_to_monitors true 2024-09-19T04:58:41.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:41 smithi174 ceph-mon[34488]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:58:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:41 smithi064 ceph-mon[30960]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:58:41.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:41 smithi103 ceph-mon[34446]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:58:41.834 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:58:41.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-19T04:58:42.117 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:58:42.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:42.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:58:42.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.402 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:42.402 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:58:42.402 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:42.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:42.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:42.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:42.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:58:42.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.616+0000 7f7c4e238640 1 -- 172.21.15.103:0/2700370240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c48101c00 msgr2=0x7f7c48102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:42.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.616+0000 7f7c4e238640 1 --2- 172.21.15.103:0/2700370240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c48101c00 0x7f7c48102000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009900 tx=0x7f7c3802a410 comp rx=0 tx=0).stop 2024-09-19T04:58:42.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.616+0000 7f7c4e238640 1 -- 172.21.15.103:0/2700370240 shutdown_connections 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.616+0000 7f7c4e238640 1 --2- 172.21.15.103:0/2700370240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c481037c0 0x7f7c4810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 --2- 172.21.15.103:0/2700370240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48102e00 0x7f7c48103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 --2- 172.21.15.103:0/2700370240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c48101c00 0x7f7c48102000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 -- 172.21.15.103:0/2700370240 >> 172.21.15.103:0/2700370240 conn(0x7f7c480fd390 msgr2=0x7f7c480ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 -- 172.21.15.103:0/2700370240 shutdown_connections 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 -- 172.21.15.103:0/2700370240 wait complete. 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 Processor -- start 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.617+0000 7f7c4e238640 1 -- start start 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48101c00 0x7f7c48075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c48102e00 0x7f7c480745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 0x7f7c48074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:42.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4810d710 con 0x7f7c48102e00 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4810d590 con 0x7f7c481037c0 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4810d890 con 0x7f7c48101c00 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 0x7f7c48074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 0x7f7c48074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42406/0 (socket says 172.21.15.103:42406) 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 -- 172.21.15.103:0/1947766320 learned_addr learned my addr 172.21.15.103:0/1947766320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c46ffd640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c48102e00 0x7f7c480745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c477fe640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48101c00 0x7f7c48075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 -- 172.21.15.103:0/1947766320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48101c00 msgr2=0x7f7c48075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48101c00 0x7f7c48075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 -- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c48102e00 msgr2=0x7f7c480745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c48102e00 0x7f7c480745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:42.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 -- 172.21.15.103:0/1947766320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c480753e0 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c477fe640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48101c00 0x7f7c48075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c47fff640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 0x7f7c48074b20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009fd0 tx=0x7f7c4000def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40017070 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c40003e20 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c481a1f50 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40010460 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.618+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481a23b0 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.619+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c48109050 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.620+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f7c40003970 con 0x7f7c481037c0 2024-09-19T04:58:42.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.621+0000 7f7c44ff9640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7c300a31b0 0x7f7c300a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:42.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.621+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f7c400cda90 con 0x7f7c481037c0 2024-09-19T04:58:42.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.621+0000 7f7c477fe640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7c300a31b0 0x7f7c300a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:42.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.622+0000 7f7c477fe640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7c300a31b0 0x7f7c300a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006010 tx=0x7f7c38000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:42.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.626+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c4009c340 con 0x7f7c481037c0 2024-09-19T04:58:42.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:58:42.800+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7c48062630 con 0x7f7c300a31b0 2024-09-19T04:58:43.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi064 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:43.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:43 smithi064 ceph-mon[30960]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' 2024-09-19T04:58:43.401 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0[42571]: 2024-09-19T04:58:42.967+0000 7f1225fc1640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T04:58:43.401 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:58:42 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0[42571]: 2024-09-19T04:58:42.976+0000 7f121e5f3640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi064 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:43.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:43.421 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:43 smithi103 ceph-mon[34446]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' 2024-09-19T04:58:43.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi064 2024-09-19T04:58:43.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:43 smithi174 ceph-mon[34488]: from='osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144]' entity='osd.0' 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] boot 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:58:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:44 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:44.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T04:58:44.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T04:58:44.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:44.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:44.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:44.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] boot 2024-09-19T04:58:44.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:58:44.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:44 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:44.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T04:58:44.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T04:58:44.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:58:44.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:44.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:44.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: osd.0 [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] boot 2024-09-19T04:58:44.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:58:44.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:44 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:58:46.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:46.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:58:46.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/1904732849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]: dispatch 2024-09-19T04:58:46.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]: dispatch 2024-09-19T04:58:46.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]': finished 2024-09-19T04:58:46.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:58:46.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:58:46.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:46 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/4218908123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/1904732849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]: dispatch 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]: dispatch 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]': finished 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:58:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:58:46.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:46 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/4218908123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:46.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:46.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:58:46.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/1904732849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]: dispatch 2024-09-19T04:58:46.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]: dispatch 2024-09-19T04:58:46.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f802a0ea-cb17-4a5e-85c1-ab9764fd2e94"}]': finished 2024-09-19T04:58:46.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:58:46.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:58:46.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:46 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/4218908123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:48.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:48 smithi103 ceph-mon[34446]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:48.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:48 smithi174 ceph-mon[34488]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:48.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:48 smithi064 ceph-mon[30960]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:50.357 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:50 smithi103 ceph-mon[34446]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:50.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:50 smithi174 ceph-mon[34488]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:50.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:50 smithi064 ceph-mon[30960]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:52.450 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:52 smithi103 ceph-mon[34446]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:52.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:52 smithi174 ceph-mon[34488]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:52.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:52 smithi064 ceph-mon[30960]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:54.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:54 smithi103 ceph-mon[34446]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:54.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:54 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:58:54.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:54 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:54.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:54 smithi174 ceph-mon[34488]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:54.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:54 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:58:54.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:54 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:54.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:54 smithi064 ceph-mon[30960]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:54.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:54 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:58:54.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:54 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:55.392 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:55 smithi103 ceph-mon[34446]: Deploying daemon osd.1 on smithi103 2024-09-19T04:58:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:55 smithi174 ceph-mon[34488]: Deploying daemon osd.1 on smithi103 2024-09-19T04:58:55.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:55 smithi064 ceph-mon[30960]: Deploying daemon osd.1 on smithi103 2024-09-19T04:58:56.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:56 smithi103 ceph-mon[34446]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:56.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:56 smithi174 ceph-mon[34488]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:56.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:56 smithi064 ceph-mon[30960]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:58.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:58:58 smithi103 ceph-mon[34446]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:58:58 smithi174 ceph-mon[34488]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:58:58.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:58:58 smithi064 ceph-mon[30960]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:00.450 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:00 smithi103 ceph-mon[34446]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:00.450 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:00 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:00.450 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:00 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:00.450 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:00 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:00.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:00 smithi174 ceph-mon[34488]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:00.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:00 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:00.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:00 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:00 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:00.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:00 smithi064 ceph-mon[30960]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:00.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:00 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:00.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:00 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:00.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:00 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:02 smithi103 ceph-mon[34446]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:02.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:02 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:02 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:02 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:02.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:02 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:02.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:02 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:02 smithi174 ceph-mon[34488]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:02 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:02 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:02 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:02.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:02 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:02.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:02 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:02 smithi064 ceph-mon[30960]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:02 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:02 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:02 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:02.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:02 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:02.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:02 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:03.985 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 1 on host 'smithi103' 2024-09-19T04:59:03.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.984+0000 7f7c44ff9640 1 -- 172.21.15.103:0/1947766320 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7c48062630 con 0x7f7c300a31b0 2024-09-19T04:59:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7c300a31b0 msgr2=0x7f7c300a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7c300a31b0 0x7f7c300a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006010 tx=0x7f7c38000bc0 comp rx=0 tx=0).stop 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 msgr2=0x7f7c48074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 0x7f7c48074b20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009fd0 tx=0x7f7c4000def0 comp rx=0 tx=0).stop 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 shutdown_connections 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7c300a31b0 0x7f7c300a5670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c481037c0 0x7f7c48074b20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c48102e00 0x7f7c480745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 --2- 172.21.15.103:0/1947766320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c48101c00 0x7f7c48075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.986+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 >> 172.21.15.103:0/1947766320 conn(0x7f7c480fd390 msgr2=0x7f7c48106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.987+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 shutdown_connections 2024-09-19T04:59:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:03.987+0000 7f7c4e238640 1 -- 172.21.15.103:0/1947766320 wait complete. 2024-09-19T04:59:04.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:04 smithi103 ceph-mon[34446]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:04.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:04.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:04.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:04.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:04 smithi174 ceph-mon[34488]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:04.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:04.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:04.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:04.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:04 smithi064 ceph-mon[30960]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:04.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:04.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:04.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:04.775 DEBUG:teuthology.orchestra.run.smithi103:osd.1> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T04:59:04.777 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-09-19T04:59:04.777 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:59:05.027 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:59:06.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:06 smithi174 ceph-mon[34488]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:06 smithi103 ceph-mon[34446]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:06.469 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:06 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1[38910]: 2024-09-19T04:59:06.235+0000 7f97b6c2a640 -1 osd.1 0 log_to_monitors true 2024-09-19T04:59:06.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:06 smithi064 ceph-mon[30960]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:07.281 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T04:59:07.306 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-09-19T04:59:07.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:07 smithi103 ceph-mon[34446]: from='osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:59:07.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:07 smithi103 ceph-mon[34446]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:59:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:07 smithi174 ceph-mon[34488]: from='osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:59:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:07 smithi174 ceph-mon[34488]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:59:07.547 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:59:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:07 smithi064 ceph-mon[30960]: from='osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:59:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:07 smithi064 ceph-mon[30960]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:59:08.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 -- 172.21.15.174:0/104543351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 msgr2=0x7f158c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 --2- 172.21.15.174:0/104543351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c102150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1580009900 tx=0x7f158002a410 comp rx=0 tx=0).stop 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 -- 172.21.15.174:0/104543351 shutdown_connections 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 --2- 172.21.15.174:0/104543351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c102690 0x7f158c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 --2- 172.21.15.174:0/104543351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c102150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 --2- 172.21.15.174:0/104543351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f158c100ad0 0x7f158c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 -- 172.21.15.174:0/104543351 >> 172.21.15.174:0/104543351 conn(0x7f158c0fc280 msgr2=0x7f158c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 -- 172.21.15.174:0/104543351 shutdown_connections 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 -- 172.21.15.174:0/104543351 wait complete. 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.005+0000 7f15915ac640 1 Processor -- start 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 -- start start 2024-09-19T04:59:08.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c100ad0 0x7f158c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f158c102690 0x7f158c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f158c10c5e0 con 0x7f158c100ad0 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f158c10c460 con 0x7f158c102690 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f158c10c760 con 0x7f158c101cd0 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:54110/0 (socket says 172.21.15.174:54110) 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 -- 172.21.15.174:0/4041717819 learned_addr learned my addr 172.21.15.174:0/4041717819 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158affd640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c100ad0 0x7f158c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:08.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 -- 172.21.15.174:0/4041717819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f158c102690 msgr2=0x7f158c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f158c102690 0x7f158c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 -- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c100ad0 msgr2=0x7f158c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c100ad0 0x7f158c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 -- 172.21.15.174:0/4041717819 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c19fa80 con 0x7f158c101cd0 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158affd640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c100ad0 0x7f158c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f158a7fc640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c19cb50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1580004770 tx=0x7f15800042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158003f030 con 0x7f158c101cd0 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c19fcb0 con 0x7f158c101cd0 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1580003cb0 con 0x7f158c101cd0 2024-09-19T04:59:08.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.006+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580035070 con 0x7f158c101cd0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a0190 con 0x7f158c101cd0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.008+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c108010 con 0x7f158c101cd0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.009+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f1580042030 con 0x7f158c101cd0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.009+0000 7f1573fff640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f15740a31b0 0x7f15740a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.009+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7f15800ee900 con 0x7f158c101cd0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.009+0000 7f158affd640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f15740a31b0 0x7f15740a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.010+0000 7f158affd640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f15740a31b0 0x7f15740a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f158c101b30 tx=0x7f157c009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:08.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.014+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15800bd010 con 0x7f158c101cd0 2024-09-19T04:59:08.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:08.188+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f158c062630 con 0x7f15740a31b0 2024-09-19T04:59:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:59:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:08.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:59:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T04:59:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:08.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:09.468 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1[38910]: 2024-09-19T04:59:08.969+0000 7f97b33c1640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T04:59:09.468 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1[38910]: 2024-09-19T04:59:08.977+0000 7f97aa9bc640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi103 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:09 smithi103 ceph-mon[34446]: from='osd.1 ' entity='osd.1' 2024-09-19T04:59:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi103 2024-09-19T04:59:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:09 smithi174 ceph-mon[34488]: from='osd.1 ' entity='osd.1' 2024-09-19T04:59:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi103 2024-09-19T04:59:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T04:59:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T04:59:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:59:09.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:09.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:09.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:09.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:09.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:09.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:09 smithi064 ceph-mon[30960]: from='osd.1 ' entity='osd.1' 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] boot 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:59:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:10 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T04:59:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T04:59:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] boot 2024-09-19T04:59:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:59:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:10 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T04:59:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T04:59:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:59:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: osd.1 [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] boot 2024-09-19T04:59:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:59:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:10 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:59:11.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:11 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/4277417789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]: dispatch 2024-09-19T04:59:11.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:11 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]: dispatch 2024-09-19T04:59:11.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:11 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]': finished 2024-09-19T04:59:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:11 smithi103 ceph-mon[34446]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T04:59:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:11 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:11 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/2994800174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:11.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:11 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/4277417789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]: dispatch 2024-09-19T04:59:11.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:11 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]: dispatch 2024-09-19T04:59:11.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:11 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]': finished 2024-09-19T04:59:11.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:11 smithi174 ceph-mon[34488]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T04:59:11.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:11 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:11.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:11 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/2994800174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:11 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/4277417789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]: dispatch 2024-09-19T04:59:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:11 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]: dispatch 2024-09-19T04:59:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:11 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0be6c1fd-a08a-4e73-904a-fee6cb38adee"}]': finished 2024-09-19T04:59:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:11 smithi064 ceph-mon[30960]: osdmap e14: 3 total, 2 up, 3 in 2024-09-19T04:59:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:11 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:11 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/2994800174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:12.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:12 smithi103 ceph-mon[34446]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:12.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:12 smithi174 ceph-mon[34488]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:12.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:12 smithi064 ceph-mon[30960]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:14.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:14 smithi103 ceph-mon[34446]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:14 smithi174 ceph-mon[34488]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:14 smithi064 ceph-mon[30960]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:16.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:16 smithi174 ceph-mon[34488]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:16.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:16 smithi103 ceph-mon[34446]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:16 smithi064 ceph-mon[30960]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:18.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:18 smithi174 ceph-mon[34488]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:18.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:18 smithi064 ceph-mon[30960]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:18.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:18 smithi103 ceph-mon[34446]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:19.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:59:19.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:19.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:59:19.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:59:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:20.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:20 smithi174 ceph-mon[34488]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:20.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:20 smithi174 ceph-mon[34488]: Deploying daemon osd.2 on smithi174 2024-09-19T04:59:20.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:20 smithi064 ceph-mon[30960]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:20.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:20 smithi064 ceph-mon[30960]: Deploying daemon osd.2 on smithi174 2024-09-19T04:59:20.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:20 smithi103 ceph-mon[34446]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:20.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:20 smithi103 ceph-mon[34446]: Deploying daemon osd.2 on smithi174 2024-09-19T04:59:22.351 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:22 smithi174 ceph-mon[34488]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:22.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:22 smithi064 ceph-mon[30960]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:22.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:22 smithi103 ceph-mon[34446]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:24 smithi174 ceph-mon[34488]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:24.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:24 smithi064 ceph-mon[30960]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:24.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:24 smithi103 ceph-mon[34446]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:25.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:25.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:25.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:25.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:25.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:25.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:26.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:26 smithi174 ceph-mon[34488]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:26.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:26 smithi064 ceph-mon[30960]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:26.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:26 smithi103 ceph-mon[34446]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:28 smithi174 ceph-mon[34488]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:28 smithi064 ceph-mon[30960]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:28.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:28.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:28 smithi103 ceph-mon[34446]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:28.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:28.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:28.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:28.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.005 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.004+0000 7f1573fff640 1 -- 172.21.15.174:0/4041717819 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f158c062630 con 0x7f15740a31b0 2024-09-19T04:59:29.005 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-09-19T04:59:29.007 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f15740a31b0 msgr2=0x7f15740a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:29.007 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f15740a31b0 0x7f15740a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f158c101b30 tx=0x7f157c009540 comp rx=0 tx=0).stop 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 msgr2=0x7f158c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c19cb50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1580004770 tx=0x7f15800042f0 comp rx=0 tx=0).stop 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 shutdown_connections 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f15740a31b0 0x7f15740a5670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f158c102690 0x7f158c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f158c101cd0 0x7f158c19cb50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 --2- 172.21.15.174:0/4041717819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c100ad0 0x7f158c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 >> 172.21.15.174:0/4041717819 conn(0x7f158c0fc280 msgr2=0x7f158c1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 shutdown_connections 2024-09-19T04:59:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T04:59:29.007+0000 7f15915ac640 1 -- 172.21.15.174:0/4041717819 wait complete. 2024-09-19T04:59:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:29 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:29 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:29 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.619 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:29 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:29.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:29 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:29 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.632 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T04:59:29.635 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/vg_nvme/lv_3... 2024-09-19T04:59:29.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:59:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:29.899 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:59:30.376 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:30 smithi064 ceph-mon[30960]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:30.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:30 smithi174 ceph-mon[34488]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:30.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:30 smithi103 ceph-mon[34446]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:31.778 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2[39445]: 2024-09-19T04:59:31.501+0000 7f4c125bc640 -1 osd.2 0 log_to_monitors true 2024-09-19T04:59:32.281 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T04:59:32.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-09-19T04:59:32.330 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:32 smithi064 ceph-mon[30960]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:32.330 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:32 smithi064 ceph-mon[30960]: from='osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:59:32.330 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:32 smithi064 ceph-mon[30960]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:59:32.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:32 smithi174 ceph-mon[34488]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:32.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:32 smithi174 ceph-mon[34488]: from='osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:59:32.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:32 smithi174 ceph-mon[34488]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:59:32.593 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:59:32.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:32 smithi103 ceph-mon[34446]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:32.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:32 smithi103 ceph-mon[34446]: from='osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:59:32.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:32 smithi103 ceph-mon[34446]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 -- 172.21.15.64:0/4149780207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc101cf0 msgr2=0x7f78bc102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 --2- 172.21.15.64:0/4149780207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc101cf0 0x7f78bc102170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009900 tx=0x7f78b802a410 comp rx=0 tx=0).stop 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 -- 172.21.15.64:0/4149780207 shutdown_connections 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 --2- 172.21.15.64:0/4149780207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78bc1026b0 0x7f78bc108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 --2- 172.21.15.64:0/4149780207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc101cf0 0x7f78bc102170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 --2- 172.21.15.64:0/4149780207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78bc100af0 0x7f78bc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 -- 172.21.15.64:0/4149780207 >> 172.21.15.64:0/4149780207 conn(0x7f78bc0fc280 msgr2=0x7f78bc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 -- 172.21.15.64:0/4149780207 shutdown_connections 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 -- 172.21.15.64:0/4149780207 wait complete. 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.068+0000 7f78c39b4640 1 Processor -- start 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 -- start start 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 0x7f78bc197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78bc101cf0 0x7f78bc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78bc1026b0 0x7f78bc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10c620 con 0x7f78bc100af0 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10c4a0 con 0x7f78bc101cf0 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c39b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10c7a0 con 0x7f78bc1026b0 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 0x7f78bc197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 0x7f78bc197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33960/0 (socket says 172.21.15.64:33960) 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 -- 172.21.15.64:0/2323557370 learned_addr learned my addr 172.21.15.64:0/2323557370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1f51640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78bc1026b0 0x7f78bc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c0f4f640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78bc101cf0 0x7f78bc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 -- 172.21.15.64:0/2323557370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78bc1026b0 msgr2=0x7f78bc19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78bc1026b0 0x7f78bc19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 -- 172.21.15.64:0/2323557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78bc101cf0 msgr2=0x7f78bc198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78bc101cf0 0x7f78bc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 -- 172.21.15.64:0/2323557370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc19fa20 con 0x7f78bc100af0 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.069+0000 7f78c1750640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 0x7f78bc197df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9c0 tx=0x7f78ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.070+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00cd80 con 0x7f78bc100af0 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.070+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78ac0035f0 con 0x7f78bc100af0 2024-09-19T04:59:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.070+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac010690 con 0x7f78bc100af0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.070+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc19fd10 con 0x7f78bc100af0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.070+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a01a0 con 0x7f78bc100af0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.071+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc1a03d0 con 0x7f78bc100af0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.073+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f78ac0037a0 con 0x7f78bc100af0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.073+0000 7f78b27fc640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a80a3260 0x7f78a80a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.074+0000 7f78c0f4f640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a80a3260 0x7f78a80a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.074+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2445+0+0 (secure 0 0 0) 0x7f78ac0ce610 con 0x7f78bc100af0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.074+0000 7f78c0f4f640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a80a3260 0x7f78a80a5720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78b802a930 tx=0x7f78b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.078+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f78ac09cb00 con 0x7f78bc100af0 2024-09-19T04:59:33.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:33.266+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f78bc06af50 con 0x7f78a80a3260 2024-09-19T04:59:33.527 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2[39445]: 2024-09-19T04:59:33.263+0000 7f4c0dd1c640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T04:59:33.527 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2[39445]: 2024-09-19T04:59:33.270+0000 7f4c0a356640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:33.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:59:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:59:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T04:59:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T04:59:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:33.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:33.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:33.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:33.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:34.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi174 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:34.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:34.562 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi174 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:34.563 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:34.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi174 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T04:59:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T04:59:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] boot 2024-09-19T04:59:35.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T04:59:35.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:35.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:35 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:59:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T04:59:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T04:59:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:35.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] boot 2024-09-19T04:59:35.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T04:59:35.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:35.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:35 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: osd.2 [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] boot 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: osdmap e17: 3 total, 3 up, 3 in 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:59:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:35 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:59:36.400 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:59:36.400 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:36.400 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47313]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:36.400 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47313]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:36.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/728462245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62258a68-87a3-4c2e-bc80-49a435c5df82"}]: dispatch 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/728462245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62258a68-87a3-4c2e-bc80-49a435c5df82"}]': finished 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:36.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:36 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2384760570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:36.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:36.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/728462245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62258a68-87a3-4c2e-bc80-49a435c5df82"}]: dispatch 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/728462245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62258a68-87a3-4c2e-bc80-49a435c5df82"}]': finished 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:36.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:36 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2384760570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:36.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:59:36.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/728462245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62258a68-87a3-4c2e-bc80-49a435c5df82"}]: dispatch 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/728462245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62258a68-87a3-4c2e-bc80-49a435c5df82"}]': finished 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: osdmap e19: 4 total, 3 up, 4 in 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:36 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2384760570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:36.900 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47342]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:59:36.900 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47342]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:36.900 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47342]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:36.901 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 04:59:36 smithi064 sudo[47342]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:37.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:37.293+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f78ac098a70 con 0x7f78bc100af0 2024-09-19T04:59:37.302 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42914]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:59:37.302 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42914]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:37.302 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:37.302 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42914]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:37.718 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:59:37.718 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:37.718 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42917]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:37.719 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 04:59:37 smithi103 sudo[42917]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:37 smithi103 ceph-mon[34446]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:37 smithi103 ceph-mon[34446]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T04:59:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:37 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:37 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:37 smithi103 ceph-mon[34446]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:59:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:37 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:37.835 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:59:37.835 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:37.835 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43237]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:37.835 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43237]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:37.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:37 smithi174 ceph-mon[34488]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:37.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:37 smithi174 ceph-mon[34488]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T04:59:37.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:37 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:37.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:37 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:37.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:37 smithi174 ceph-mon[34488]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:59:37.836 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:37 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:37.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:37 smithi064 ceph-mon[30960]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:37.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:37 smithi064 ceph-mon[30960]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T04:59:37.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:37 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:37.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:37 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:37.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:37 smithi064 ceph-mon[30960]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:59:37.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:37 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:38.277 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43240]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:59:38.277 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:38.277 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43240]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:38.277 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 04:59:37 smithi174 sudo[43240]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:38.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:59:38.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:38.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47745]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:38.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47745]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:39.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47936]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:59:39.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47936]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:39.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47936]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:39.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:38 smithi064 sudo[47936]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:39.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43243]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:59:39.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:39.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43243]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:39.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43243]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43246]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:59:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43246]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:39.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 sudo[43246]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:39.756 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:59:39.756 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:39.756 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:39.756 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42920]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:40.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:40.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:40.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42923]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42923]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 sudo[42923]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:40.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:40.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:40.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:40.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:40.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:40.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:40.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:40 smithi064 ceph-mon[30960]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:41.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:41.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:41.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:41.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:41.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:41.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:41.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:41.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:40 smithi174 ceph-mon[34488]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:59:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:40 smithi103 ceph-mon[34446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:59:42.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:41 smithi174 ceph-mon[34488]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:41 smithi064 ceph-mon[30960]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:42.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:41 smithi103 ceph-mon[34446]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:44.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:43 smithi174 ceph-mon[34488]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:44.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:43 smithi064 ceph-mon[30960]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:44.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:43 smithi103 ceph-mon[34446]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:45.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:44 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:59:45.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:44 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:45.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:44 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:59:45.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:44 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:45.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:44 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:59:45.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:44 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:46.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:45 smithi174 ceph-mon[34488]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:46.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:45 smithi174 ceph-mon[34488]: Deploying daemon osd.3 on smithi064 2024-09-19T04:59:46.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:45 smithi064 ceph-mon[30960]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:46.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:45 smithi064 ceph-mon[30960]: Deploying daemon osd.3 on smithi064 2024-09-19T04:59:46.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:45 smithi103 ceph-mon[34446]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:46.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:45 smithi103 ceph-mon[34446]: Deploying daemon osd.3 on smithi064 2024-09-19T04:59:47.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:47 smithi064 ceph-mon[30960]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:48.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:47 smithi103 ceph-mon[34446]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:48.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:47 smithi174 ceph-mon[34488]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:50.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:49 smithi064 ceph-mon[30960]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:50.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:49 smithi103 ceph-mon[34446]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:50.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:49 smithi174 ceph-mon[34488]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:51 smithi064 ceph-mon[30960]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:51 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:51 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:52.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:51 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:52.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:51 smithi103 ceph-mon[34446]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:52.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:51 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:52.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:51 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:52.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:51 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:52.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:51 smithi174 ceph-mon[34488]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:52.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:51 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:52.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:51 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:52.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:51 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:54.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:53 smithi064 ceph-mon[30960]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:54.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.166+0000 7f78b27fc640 1 -- 172.21.15.64:0/2323557370 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78bc06af50 con 0x7f78a80a3260 2024-09-19T04:59:54.169 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2024-09-19T04:59:54.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a80a3260 msgr2=0x7f78a80a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a80a3260 0x7f78a80a5720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78b802a930 tx=0x7f78b8000bc0 comp rx=0 tx=0).stop 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 msgr2=0x7f78bc197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 0x7f78bc197df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9c0 tx=0x7f78ac00ee80 comp rx=0 tx=0).stop 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 shutdown_connections 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a80a3260 0x7f78a80a5720 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78bc1026b0 0x7f78bc19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78bc101cf0 0x7f78bc198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 --2- 172.21.15.64:0/2323557370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78bc100af0 0x7f78bc197df0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 >> 172.21.15.64:0/2323557370 conn(0x7f78bc0fc280 msgr2=0x7f78bc06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:54.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 shutdown_connections 2024-09-19T04:59:54.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T04:59:54.168+0000 7f78c39b4640 1 -- 172.21.15.64:0/2323557370 wait complete. 2024-09-19T04:59:54.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:53 smithi103 ceph-mon[34446]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:54.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:53 smithi174 ceph-mon[34488]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:54.764 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T04:59:54.767 INFO:tasks.cephadm:Deploying osd.4 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-19T04:59:54.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:59:55.024 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:59:55.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.439 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.439 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.439 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.439 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:56.258 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:56 smithi064 ceph-mon[30960]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:56.318 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:56 smithi103 ceph-mon[34446]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:56.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:56 smithi174 ceph-mon[34488]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:57.395 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T04:59:57.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-19T04:59:57.650 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 04:59:57 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T04:59:57.311+0000 7f15395a0640 -1 osd.3 0 log_to_monitors true 2024-09-19T04:59:57.671 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:59:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 -- 172.21.15.103:0/3014903500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 msgr2=0x7fecd0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 --2- 172.21.15.103:0/3014903500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd0109d10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009ca0 tx=0x7fecc802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:59:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 -- 172.21.15.103:0/3014903500 shutdown_connections 2024-09-19T04:59:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 --2- 172.21.15.103:0/3014903500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd0109d10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 --2- 172.21.15.103:0/3014903500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd0104fd0 0x7fecd01073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 --2- 172.21.15.103:0/3014903500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd01026a0 0x7fecd0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 -- 172.21.15.103:0/3014903500 >> 172.21.15.103:0/3014903500 conn(0x7fecd00fc280 msgr2=0x7fecd00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 -- 172.21.15.103:0/3014903500 shutdown_connections 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 -- 172.21.15.103:0/3014903500 wait complete. 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 Processor -- start 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.129+0000 7fecd6c12640 1 -- start start 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd6c12640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd01026a0 0x7fecd0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd6c12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd0104fd0 0x7fecd01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd6c12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd6c12640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fecd010d730 con 0x7fecd0104fd0 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd6c12640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecd010d5b0 con 0x7fecd0107900 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd6c12640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecd010d8b0 con 0x7fecd01026a0 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39374/0 (socket says 172.21.15.103:39374) 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 -- 172.21.15.103:0/2084780201 learned_addr learned my addr 172.21.15.103:0/2084780201 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7feccffff640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd0104fd0 0x7fecd01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 -- 172.21.15.103:0/2084780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd01026a0 msgr2=0x7fecd0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd01026a0 0x7fecd0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 -- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd0104fd0 msgr2=0x7fecd01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd0104fd0 0x7fecd01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 -- 172.21.15.103:0/2084780201 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd019fab0 con 0x7fecd0107900 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7fecd51af640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd019f440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fecc8031970 tx=0x7fecc8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.130+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8004040 con 0x7fecd0107900 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.131+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc8004690 con 0x7fecd0107900 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.131+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8030c60 con 0x7fecd0107900 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.131+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd019fd40 con 0x7fecd0107900 2024-09-19T04:59:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.131+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd01a01d0 con 0x7fecd0107900 2024-09-19T04:59:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.132+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd00fdd70 con 0x7fecd0107900 2024-09-19T04:59:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.133+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fecc80041e0 con 0x7fecd0107900 2024-09-19T04:59:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.134+0000 7feccdffb640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecc00a3270 0x7fecc00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.134+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3352+0+0 (secure 0 0 0) 0x7fecc80efc90 con 0x7fecd0107900 2024-09-19T04:59:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.134+0000 7fecd49ae640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecc00a3270 0x7fecc00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:58.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.134+0000 7fecd49ae640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecc00a3270 0x7fecc00a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecc400a8f0 tx=0x7fecc4005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:58.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.141+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecc80bdcf0 con 0x7fecd0107900 2024-09-19T04:59:58.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T04:59:58.322+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fecd0062630 con 0x7fecc00a3270 2024-09-19T04:59:58.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:58.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:58.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:58 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:59:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:58 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:59:58.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:59:58.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:58.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:58 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi064 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:59.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:59.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:59.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 04:59:59 smithi103 ceph-mon[34446]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T04:59:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi064 2024-09-19T04:59:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 04:59:59 smithi174 ceph-mon[34488]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi064 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:59.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:59.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-mon[30960]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:00:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:00:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:00:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' 2024-09-19T05:00:00.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: overall HEALTH_OK 2024-09-19T05:00:00.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] boot 2024-09-19T05:00:00.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: osdmap e23: 4 total, 4 up, 4 in 2024-09-19T05:00:00.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:00 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.400 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T04:59:59.985+0000 7f1534d00640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T05:00:00.400 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 04:59:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T04:59:59.993+0000 7f152d332640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:00:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:00:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: overall HEALTH_OK 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] boot 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: osdmap e23: 4 total, 4 up, 4 in 2024-09-19T05:00:00.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:00 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: from='osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608]' entity='osd.3' 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: overall HEALTH_OK 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: osd.3 [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] boot 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: osdmap e23: 4 total, 4 up, 4 in 2024-09-19T05:00:00.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:00 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:00:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:00:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:00:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/996860400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]: dispatch 2024-09-19T05:00:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]: dispatch 2024-09-19T05:00:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]': finished 2024-09-19T05:00:01.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: osdmap e24: 5 total, 4 up, 5 in 2024-09-19T05:00:01.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:01.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:00:01.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:00:01.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/996860400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]: dispatch 2024-09-19T05:00:01.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]: dispatch 2024-09-19T05:00:01.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]': finished 2024-09-19T05:00:01.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: osdmap e24: 5 total, 4 up, 5 in 2024-09-19T05:00:01.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:01.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:00:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:00:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/996860400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]: dispatch 2024-09-19T05:00:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]: dispatch 2024-09-19T05:00:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d7e9b07-2d48-4029-8396-fdbae9a82494"}]': finished 2024-09-19T05:00:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: osdmap e24: 5 total, 4 up, 5 in 2024-09-19T05:00:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:02 smithi174 ceph-mon[34488]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:02 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/341481932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:02 smithi174 ceph-mon[34488]: osdmap e25: 5 total, 4 up, 5 in 2024-09-19T05:00:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:02 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:02 smithi064 ceph-mon[30960]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:02 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/341481932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:02 smithi064 ceph-mon[30960]: osdmap e25: 5 total, 4 up, 5 in 2024-09-19T05:00:02.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:02 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:02.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:02 smithi103 ceph-mon[34446]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:02.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:02 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/341481932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:02.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:02 smithi103 ceph-mon[34446]: osdmap e25: 5 total, 4 up, 5 in 2024-09-19T05:00:02.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:02 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:03.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:03 smithi103 ceph-mon[34446]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:03 smithi103 ceph-mon[34446]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T05:00:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:04.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:03 smithi174 ceph-mon[34488]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:04.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:03 smithi174 ceph-mon[34488]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T05:00:04.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:04.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:03 smithi064 ceph-mon[30960]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:04.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:03 smithi064 ceph-mon[30960]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T05:00:04.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:04.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:04 smithi103 ceph-mon[34446]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T05:00:04.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:05.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:04 smithi174 ceph-mon[34488]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T05:00:05.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:05.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:04 smithi064 ceph-mon[30960]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T05:00:05.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:05.995 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:05 smithi103 ceph-mon[34446]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:06.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:05 smithi174 ceph-mon[34488]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:06.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:05 smithi064 ceph-mon[30960]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:07.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:07 smithi103 ceph-mon[34446]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T05:00:08.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:07 smithi174 ceph-mon[34488]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T05:00:08.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:07 smithi064 ceph-mon[30960]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T05:00:08.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:00:08.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:09.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:00:09.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:09.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:00:09.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:10.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:09 smithi174 ceph-mon[34488]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:00:10.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:09 smithi174 ceph-mon[34488]: Deploying daemon osd.4 on smithi103 2024-09-19T05:00:10.048 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:09 smithi103 ceph-mon[34446]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:00:10.048 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:09 smithi103 ceph-mon[34446]: Deploying daemon osd.4 on smithi103 2024-09-19T05:00:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:09 smithi064 ceph-mon[30960]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:00:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:09 smithi064 ceph-mon[30960]: Deploying daemon osd.4 on smithi103 2024-09-19T05:00:11.946 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:11 smithi103 ceph-mon[34446]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:00:12.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:11 smithi174 ceph-mon[34488]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:00:12.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:11 smithi064 ceph-mon[30960]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:00:13.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:13 smithi103 ceph-mon[34446]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T05:00:14.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:13 smithi174 ceph-mon[34488]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T05:00:14.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:13 smithi064 ceph-mon[30960]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-19T05:00:15.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:15 smithi103 ceph-mon[34446]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:00:15.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:15.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:15.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:16.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:15 smithi174 ceph-mon[34488]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:00:16.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:16.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:16.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:16.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:15 smithi064 ceph-mon[30960]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:00:16.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:16.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:16.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:17 smithi174 ceph-mon[34488]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:00:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:18.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.074 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:17 smithi103 ceph-mon[34446]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:00:18.075 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.075 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.075 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:18.075 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:18.075 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:17 smithi064 ceph-mon[30960]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:00:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:18.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.726+0000 7feccdffb640 1 -- 172.21.15.103:0/2084780201 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fecd0062630 con 0x7fecc00a3270 2024-09-19T05:00:18.728 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 4 on host 'smithi103' 2024-09-19T05:00:18.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecc00a3270 msgr2=0x7fecc00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:18.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecc00a3270 0x7fecc00a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecc400a8f0 tx=0x7fecc4005bf0 comp rx=0 tx=0).stop 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 msgr2=0x7fecd019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd019f440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fecc8031970 tx=0x7fecc8003ba0 comp rx=0 tx=0).stop 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 shutdown_connections 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fecc00a3270 0x7fecc00a5730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecd0107900 0x7fecd019f440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd0104fd0 0x7fecd01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 --2- 172.21.15.103:0/2084780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd01026a0 0x7fecd0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.728+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 >> 172.21.15.103:0/2084780201 conn(0x7fecd00fc280 msgr2=0x7fecd0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.729+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 shutdown_connections 2024-09-19T05:00:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:00:18.729+0000 7fecd6c12640 1 -- 172.21.15.103:0/2084780201 wait complete. 2024-09-19T05:00:18.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:18 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:18.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:18 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:18.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:18 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:19.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:18 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:19.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:18 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:19.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:18 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:19.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:19.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:19.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:19.324 DEBUG:teuthology.orchestra.run.smithi103:osd.4> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T05:00:19.327 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-09-19T05:00:19.327 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:00:19.580 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:00:19.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:19 smithi064 ceph-mon[30960]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:19.908 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:19 smithi103 ceph-mon[34446]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:19.966 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:19 smithi174 ceph-mon[34488]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:21.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:00:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:00:21.132+0000 7fd36e490640 -1 osd.4 0 log_to_monitors true 2024-09-19T05:00:21.793 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:21 smithi103 ceph-mon[34446]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:21.793 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:21 smithi103 ceph-mon[34446]: from='osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:00:21.794 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:21 smithi103 ceph-mon[34446]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:00:21.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:21 smithi174 ceph-mon[34488]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:21.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:21 smithi174 ceph-mon[34488]: from='osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:00:21.901 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:21 smithi174 ceph-mon[34488]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:00:21.904 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T05:00:21.937 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-09-19T05:00:22.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:21 smithi064 ceph-mon[30960]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:22.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:21 smithi064 ceph-mon[30960]: from='osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:00:22.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:21 smithi064 ceph-mon[30960]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:00:22.191 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:00:22.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 -- 172.21.15.174:0/3079024978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4101cf0 msgr2=0x7fefb4102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:22.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/3079024978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4101cf0 0x7fefb4102170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009900 tx=0x7fef9c02a410 comp rx=0 tx=0).stop 2024-09-19T05:00:22.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 -- 172.21.15.174:0/3079024978 shutdown_connections 2024-09-19T05:00:22.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/3079024978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb41026b0 0x7fefb4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:22.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/3079024978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4101cf0 0x7fefb4102170 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:22.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/3079024978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb4100af0 0x7fefb4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:22.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 -- 172.21.15.174:0/3079024978 >> 172.21.15.174:0/3079024978 conn(0x7fefb40fc2a0 msgr2=0x7fefb40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:22.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 -- 172.21.15.174:0/3079024978 shutdown_connections 2024-09-19T05:00:22.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.636+0000 7fefb9d7b640 1 -- 172.21.15.174:0/3079024978 wait complete. 2024-09-19T05:00:22.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 Processor -- start 2024-09-19T05:00:22.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 -- start start 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 0x7fefb410b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb4101cf0 0x7fefb4109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb41026b0 0x7fefb410a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d750 con 0x7fefb4101cf0 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d5d0 con 0x7fefb41026b0 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb9d7b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fefb410d8d0 con 0x7fefb4100af0 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 0x7fefb410b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 0x7fefb410b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:35624/0 (socket says 172.21.15.174:35624) 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 -- 172.21.15.174:0/2082566479 learned_addr learned my addr 172.21.15.174:0/2082566479 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb3fff640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb41026b0 0x7fefb410a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:22.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb2ffd640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb4101cf0 0x7fefb4109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 -- 172.21.15.174:0/2082566479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb41026b0 msgr2=0x7fefb410a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb41026b0 0x7fefb410a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 -- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb4101cf0 msgr2=0x7fefb4109ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.637+0000 7fefb37fe640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb4101cf0 0x7fefb4109ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb37fe640 1 -- 172.21.15.174:0/2082566479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb410a8c0 con 0x7fefb4100af0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb2ffd640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb4101cf0 0x7fefb4109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb37fe640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 0x7fefb410b470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fefa400ea80 tx=0x7fefa400ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa400ce40 con 0x7fefb4100af0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa4011920 con 0x7fefb4100af0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb41a1fe0 con 0x7fefb4100af0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa401aaa0 con 0x7fefb4100af0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.638+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a23d0 con 0x7fefb4100af0 2024-09-19T05:00:22.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.639+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb4107fe0 con 0x7fefb4100af0 2024-09-19T05:00:22.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.640+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fefa4011ad0 con 0x7fefb4100af0 2024-09-19T05:00:22.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.641+0000 7fefb0ff9640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fefa00a3240 0x7fefa00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:22.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.641+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3805+0+0 (secure 0 0 0) 0x7fefa40ced20 con 0x7fefb4100af0 2024-09-19T05:00:22.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.641+0000 7fefb2ffd640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fefa00a3240 0x7fefa00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:22.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.642+0000 7fefb2ffd640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fefa00a3240 0x7fefa00a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefb410b1f0 tx=0x7fef9c032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:22.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.645+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fefa4016030 con 0x7fefb4100af0 2024-09-19T05:00:22.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:22.820+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fefb4062630 con 0x7fefa00a3240 2024-09-19T05:00:23.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:00:23.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:00:23.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi103 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:23.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:22 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:00:23.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:00:23.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi103 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:23.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:22 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi103 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T05:00:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:23.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:23.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:22 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:23.968 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:00:23.543+0000 7fd369bf0640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T05:00:23.968 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:00:23.551+0000 7fd362222640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:23.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:23 smithi103 ceph-mon[34446]: from='osd.4 ' entity='osd.4' 2024-09-19T05:00:24.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:23 smithi174 ceph-mon[34488]: from='osd.4 ' entity='osd.4' 2024-09-19T05:00:24.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:24.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:00:24.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:00:24.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:24.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:24.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:24.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:24.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:24.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:24.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:23 smithi064 ceph-mon[30960]: from='osd.4 ' entity='osd.4' 2024-09-19T05:00:25.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:24 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:00:25.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:24 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:00:25.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:24 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:25.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:24 smithi174 ceph-mon[34488]: osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] boot 2024-09-19T05:00:25.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:24 smithi174 ceph-mon[34488]: osdmap e30: 5 total, 5 up, 5 in 2024-09-19T05:00:25.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:24 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:24 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:00:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:24 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:00:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:24 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:24 smithi064 ceph-mon[30960]: osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] boot 2024-09-19T05:00:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:24 smithi064 ceph-mon[30960]: osdmap e30: 5 total, 5 up, 5 in 2024-09-19T05:00:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:24 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:25.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:24 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:00:25.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:24 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:00:25.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:24 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:25.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:24 smithi103 ceph-mon[34446]: osd.4 [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] boot 2024-09-19T05:00:25.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:24 smithi103 ceph-mon[34446]: osdmap e30: 5 total, 5 up, 5 in 2024-09-19T05:00:25.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:24 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:00:26.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:26.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/3276732991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]: dispatch 2024-09-19T05:00:26.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]: dispatch 2024-09-19T05:00:26.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]': finished 2024-09-19T05:00:26.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: osdmap e31: 6 total, 5 up, 6 in 2024-09-19T05:00:26.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:26.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:25 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/3314696329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:26.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:26.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/3276732991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]: dispatch 2024-09-19T05:00:26.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]: dispatch 2024-09-19T05:00:26.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]': finished 2024-09-19T05:00:26.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: osdmap e31: 6 total, 5 up, 6 in 2024-09-19T05:00:26.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:26.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:25 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/3314696329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:26.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:00:26.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/3276732991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]: dispatch 2024-09-19T05:00:26.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]: dispatch 2024-09-19T05:00:26.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93251354-cc4c-4dcf-9536-69770ad07286"}]': finished 2024-09-19T05:00:26.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: osdmap e31: 6 total, 5 up, 6 in 2024-09-19T05:00:26.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:26.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:25 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/3314696329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:28.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:27 smithi064 ceph-mon[30960]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:28.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:27 smithi103 ceph-mon[34446]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:28.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:27 smithi174 ceph-mon[34488]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:30.079 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:29 smithi174 ceph-mon[34488]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:30.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:29 smithi064 ceph-mon[30960]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:30.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:29 smithi103 ceph-mon[34446]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:32.025 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:31 smithi174 ceph-mon[34488]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:32.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:31 smithi064 ceph-mon[30960]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:32.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:31 smithi103 ceph-mon[34446]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:34.100 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:33 smithi174 ceph-mon[34488]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:34.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:00:34.101 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:33 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:34.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:33 smithi064 ceph-mon[30960]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:34.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:00:34.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:33 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:34.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:33 smithi103 ceph-mon[34446]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:34.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:00:34.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:33 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:35.054 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:34 smithi174 ceph-mon[34488]: Deploying daemon osd.5 on smithi174 2024-09-19T05:00:35.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:34 smithi064 ceph-mon[30960]: Deploying daemon osd.5 on smithi174 2024-09-19T05:00:35.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:34 smithi103 ceph-mon[34446]: Deploying daemon osd.5 on smithi174 2024-09-19T05:00:36.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:35 smithi174 ceph-mon[34488]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:36.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:35 smithi064 ceph-mon[30960]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:36.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:35 smithi103 ceph-mon[34446]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:38.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:37 smithi064 ceph-mon[30960]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:38.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:37 smithi103 ceph-mon[34446]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:38.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:37 smithi174 ceph-mon[34488]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:40.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:39 smithi174 ceph-mon[34488]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:40.086 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:40.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:40.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:39 smithi064 ceph-mon[30960]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:40.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:40.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:39 smithi103 ceph-mon[34446]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:40.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:40.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:41 smithi064 ceph-mon[30960]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:42.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:42.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:41 smithi103 ceph-mon[34446]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:42.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:42.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:42.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:41 smithi174 ceph-mon[34488]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:42.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:42.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:42.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:42.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:43.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.360+0000 7fefb0ff9640 1 -- 172.21.15.174:0/2082566479 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fefb4062630 con 0x7fefa00a3240 2024-09-19T05:00:43.361 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-09-19T05:00:43.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fefa00a3240 msgr2=0x7fefa00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:43.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fefa00a3240 0x7fefa00a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefb410b1f0 tx=0x7fef9c032d20 comp rx=0 tx=0).stop 2024-09-19T05:00:43.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 msgr2=0x7fefb410b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 0x7fefb410b470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fefa400ea80 tx=0x7fefa400ef40 comp rx=0 tx=0).stop 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 shutdown_connections 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fefa00a3240 0x7fefa00a5700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb41026b0 0x7fefb410a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb4101cf0 0x7fefb4109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 --2- 172.21.15.174:0/2082566479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb4100af0 0x7fefb410b470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 >> 172.21.15.174:0/2082566479 conn(0x7fefb40fc2a0 msgr2=0x7fefb41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 shutdown_connections 2024-09-19T05:00:43.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:00:43.363+0000 7fefb9d7b640 1 -- 172.21.15.174:0/2082566479 wait complete. 2024-09-19T05:00:44.024 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:43 smithi174 ceph-mon[34488]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:44.024 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:44.025 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:44.025 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:44.050 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T05:00:44.053 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2024-09-19T05:00:44.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:00:44.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:43 smithi064 ceph-mon[30960]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:44.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:44.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:44.082 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:44.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:43 smithi103 ceph-mon[34446]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:44.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:44.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:44.319 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:00:45.778 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:00:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:00:45.527+0000 7f72b7d24640 -1 osd.5 0 log_to_monitors true 2024-09-19T05:00:46.059 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:45 smithi064 ceph-mon[30960]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:46.059 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:45 smithi064 ceph-mon[30960]: from='osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:00:46.059 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:45 smithi064 ceph-mon[30960]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:00:46.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:45 smithi174 ceph-mon[34488]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:46.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:45 smithi174 ceph-mon[34488]: from='osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:00:46.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:45 smithi174 ceph-mon[34488]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:00:46.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:45 smithi103 ceph-mon[34446]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:46.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:45 smithi103 ceph-mon[34446]: from='osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:00:46.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:45 smithi103 ceph-mon[34446]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:00:46.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:00:46.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-09-19T05:00:47.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:00:47.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:47.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:46 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.129 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:00:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:00:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:00:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T05:00:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:00:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:46 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:00:47.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: osdmap e32: 6 total, 5 up, 6 in 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:46 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:00:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 -- 172.21.15.64:0/2365934357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0508100b10 msgr2=0x7f0508100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 --2- 172.21.15.64:0/2365934357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0508100b10 0x7f0508100f10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0504009900 tx=0x7f050402a410 comp rx=0 tx=0).stop 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 -- 172.21.15.64:0/2365934357 shutdown_connections 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 --2- 172.21.15.64:0/2365934357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05081026d0 0x7f0508108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 --2- 172.21.15.64:0/2365934357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508101d10 0x7f0508102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 --2- 172.21.15.64:0/2365934357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0508100b10 0x7f0508100f10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 -- 172.21.15.64:0/2365934357 >> 172.21.15.64:0/2365934357 conn(0x7f05080fc2a0 msgr2=0x7f05080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 -- 172.21.15.64:0/2365934357 shutdown_connections 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.555+0000 7f050fe0c640 1 -- 172.21.15.64:0/2365934357 wait complete. 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 Processor -- start 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 -- start start 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 0x7f0508197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0508101d10 0x7f0508198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05081026d0 0x7f050819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f050810d7b0 con 0x7f05081026d0 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f050810d630 con 0x7f0508101d10 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050fe0c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f050810d930 con 0x7f0508100b10 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050dba8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 0x7f0508197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050dba8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 0x7f0508197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:49452/0 (socket says 172.21.15.64:49452) 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050dba8640 1 -- 172.21.15.64:0/1721477068 learned_addr learned my addr 172.21.15.64:0/1721477068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050e3a9640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05081026d0 0x7f050819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.556+0000 7f050d3a7640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0508101d10 0x7f0508198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050dba8640 1 -- 172.21.15.64:0/1721477068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0508101d10 msgr2=0x7f0508198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050dba8640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0508101d10 0x7f0508198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050dba8640 1 -- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05081026d0 msgr2=0x7f050819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050dba8640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05081026d0 0x7f050819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050dba8640 1 -- 172.21.15.64:0/1721477068 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050819faf0 con 0x7f0508100b10 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050dba8640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 0x7f0508197ec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05040314e0 tx=0x7f05040042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050403f030 con 0x7f0508100b10 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0504003cb0 con 0x7f0508100b10 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050819fd80 con 0x7f0508100b10 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504035070 con 0x7f0508100b10 2024-09-19T05:00:47.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.557+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a01b0 con 0x7f0508100b10 2024-09-19T05:00:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.558+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0508108000 con 0x7f0508100b10 2024-09-19T05:00:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.559+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0504042030 con 0x7f0508100b10 2024-09-19T05:00:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.560+0000 7f04feffd640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04f40a3270 0x7f04f40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.560+0000 7f050d3a7640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04f40a3270 0x7f04f40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.560+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4253+0+0 (secure 0 0 0) 0x7f05040ef0b0 con 0x7f0508100b10 2024-09-19T05:00:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.561+0000 7f050d3a7640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04f40a3270 0x7f04f40a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05081993e0 tx=0x7f04f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.563+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05040bc6c0 con 0x7f0508100b10 2024-09-19T05:00:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:00:47.748+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0508062630 con 0x7f04f40a3270 2024-09-19T05:00:48.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:00:47.598+0000 7f72b44bb640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T05:00:48.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:00:47.605+0000 7f72abab6640 -1 osd.5 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:00:48.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:48.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi174 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='osd.5 ' entity='osd.5' 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:48.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:47 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi174 2024-09-19T05:00:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T05:00:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:00:48.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T05:00:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='osd.5 ' entity='osd.5' 2024-09-19T05:00:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:48.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:47 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:48.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi174 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='osd.5 ' entity='osd.5' 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:48.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:47 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:00:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:00:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] boot 2024-09-19T05:00:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: osdmap e34: 6 total, 6 up, 6 in 2024-09-19T05:00:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:48 smithi064 ceph-mon[30960]: osdmap e35: 6 total, 6 up, 6 in 2024-09-19T05:00:49.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:00:49.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:00:49.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:49.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] boot 2024-09-19T05:00:49.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: osdmap e34: 6 total, 6 up, 6 in 2024-09-19T05:00:49.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:49.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:48 smithi103 ceph-mon[34446]: osdmap e35: 6 total, 6 up, 6 in 2024-09-19T05:00:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:00:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:00:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: osd.5 [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] boot 2024-09-19T05:00:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: osdmap e34: 6 total, 6 up, 6 in 2024-09-19T05:00:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:00:49.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:48 smithi174 ceph-mon[34488]: osdmap e35: 6 total, 6 up, 6 in 2024-09-19T05:00:50.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:49 smithi064 ceph-mon[30960]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:50.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:49 smithi064 ceph-mon[30960]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T05:00:50.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:49 smithi103 ceph-mon[34446]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:50.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:49 smithi103 ceph-mon[34446]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T05:00:50.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:49 smithi174 ceph-mon[34488]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:00:50.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:49 smithi174 ceph-mon[34488]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T05:00:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:50 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3376372112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "346e4af1-6d30-4b90-94c9-e660333548ca"}]: dispatch 2024-09-19T05:00:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:50 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3376372112' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "346e4af1-6d30-4b90-94c9-e660333548ca"}]': finished 2024-09-19T05:00:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:50 smithi064 ceph-mon[30960]: osdmap e37: 7 total, 6 up, 7 in 2024-09-19T05:00:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:00:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:50 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3874362936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:51.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:50 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3376372112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "346e4af1-6d30-4b90-94c9-e660333548ca"}]: dispatch 2024-09-19T05:00:51.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:50 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3376372112' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "346e4af1-6d30-4b90-94c9-e660333548ca"}]': finished 2024-09-19T05:00:51.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:50 smithi103 ceph-mon[34446]: osdmap e37: 7 total, 6 up, 7 in 2024-09-19T05:00:51.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:00:51.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:50 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3874362936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:51.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:50 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3376372112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "346e4af1-6d30-4b90-94c9-e660333548ca"}]: dispatch 2024-09-19T05:00:51.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:50 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3376372112' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "346e4af1-6d30-4b90-94c9-e660333548ca"}]': finished 2024-09-19T05:00:51.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:50 smithi174 ceph-mon[34488]: osdmap e37: 7 total, 6 up, 7 in 2024-09-19T05:00:51.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:00:51.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:50 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3874362936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:52.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:52 smithi064 ceph-mon[30960]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 164 KiB/s, 0 objects/s recovering 2024-09-19T05:00:52.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:52 smithi064 ceph-mon[30960]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T05:00:52.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:00:52.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:52 smithi103 ceph-mon[34446]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 164 KiB/s, 0 objects/s recovering 2024-09-19T05:00:52.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:52 smithi103 ceph-mon[34446]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T05:00:52.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:00:52.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:52 smithi174 ceph-mon[34488]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 164 KiB/s, 0 objects/s recovering 2024-09-19T05:00:52.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:52 smithi174 ceph-mon[34488]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T05:00:52.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:00:54.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:54 smithi064 ceph-mon[30960]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T05:00:54.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:54 smithi103 ceph-mon[34446]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T05:00:54.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:54 smithi174 ceph-mon[34488]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T05:00:56.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:56 smithi064 ceph-mon[30960]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T05:00:56.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:56 smithi103 ceph-mon[34446]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T05:00:56.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:56 smithi174 ceph-mon[34488]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T05:00:58.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:58 smithi064 ceph-mon[30960]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T05:00:58.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:58 smithi103 ceph-mon[34446]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T05:00:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:58 smithi174 ceph-mon[34488]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T05:00:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:00:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:00:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:59.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:00:59.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:00:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:00:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:00:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:00.370 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:00 smithi103 ceph-mon[34446]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T05:01:00.370 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:00 smithi103 ceph-mon[34446]: Deploying daemon osd.6 on smithi064 2024-09-19T05:01:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:00 smithi064 ceph-mon[30960]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T05:01:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:00 smithi064 ceph-mon[30960]: Deploying daemon osd.6 on smithi064 2024-09-19T05:01:00.428 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:00 smithi174 ceph-mon[34488]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T05:01:00.428 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:00 smithi174 ceph-mon[34488]: Deploying daemon osd.6 on smithi064 2024-09-19T05:01:02.323 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:02 smithi064 ceph-mon[30960]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:02.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:02 smithi103 ceph-mon[34446]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:02 smithi174 ceph-mon[34488]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:04.326 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:04 smithi064 ceph-mon[30960]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:04.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:04 smithi103 ceph-mon[34446]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:04.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:04 smithi174 ceph-mon[34488]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:05.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:05.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:05.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:05.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:05.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:06.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:06 smithi064 ceph-mon[30960]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:06 smithi103 ceph-mon[34446]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:06.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:06 smithi174 ceph-mon[34488]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:08.236 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:08 smithi064 ceph-mon[30960]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:08.236 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.236 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.237 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:08.237 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:08.237 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:08 smithi103 ceph-mon[34446]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:08 smithi174 ceph-mon[34488]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:08.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.567+0000 7f04feffd640 1 -- 172.21.15.64:0/1721477068 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0508062630 con 0x7f04f40a3270 2024-09-19T05:01:08.569 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2024-09-19T05:01:08.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04f40a3270 msgr2=0x7f04f40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:08.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04f40a3270 0x7f04f40a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05081993e0 tx=0x7f04f800b410 comp rx=0 tx=0).stop 2024-09-19T05:01:08.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 msgr2=0x7f0508197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:08.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 0x7f0508197ec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05040314e0 tx=0x7f05040042e0 comp rx=0 tx=0).stop 2024-09-19T05:01:08.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 shutdown_connections 2024-09-19T05:01:08.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04f40a3270 0x7f04f40a5730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:08.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05081026d0 0x7f050819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0508101d10 0x7f0508198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 --2- 172.21.15.64:0/1721477068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0508100b10 0x7f0508197ec0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 >> 172.21.15.64:0/1721477068 conn(0x7f05080fc2a0 msgr2=0x7f05081051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 shutdown_connections 2024-09-19T05:01:08.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:01:08.569+0000 7f050fe0c640 1 -- 172.21.15.64:0/1721477068 wait complete. 2024-09-19T05:01:09.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:09.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:09.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:09.176 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T05:01:09.179 INFO:tasks.cephadm:Deploying osd.7 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-19T05:01:09.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:01:09.449 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:01:09.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:09.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:09.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:09.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:09.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:09.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:10.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:10 smithi103 ceph-mon[34446]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:10.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:10 smithi064 ceph-mon[30960]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:10 smithi174 ceph-mon[34488]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:11.434 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:01:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:01:11.175+0000 7f6d7a541640 -1 osd.6 0 log_to_monitors true 2024-09-19T05:01:12.162 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T05:01:12.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-19T05:01:12.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:12.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:01:12.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:12.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:12 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:01:12.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:12.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:12.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:12 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:12.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:12 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:12.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 -- 172.21.15.103:0/3515868537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0101490 msgr2=0x7fe9a0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:12.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/3515868537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0101490 0x7fe9a0101910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe988009900 tx=0x7fe98802a410 comp rx=0 tx=0).stop 2024-09-19T05:01:12.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 -- 172.21.15.103:0/3515868537 shutdown_connections 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/3515868537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a0101e50 0x7fe9a01086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/3515868537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0101490 0x7fe9a0101910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/3515868537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9a0100290 0x7fe9a0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 -- 172.21.15.103:0/3515868537 >> 172.21.15.103:0/3515868537 conn(0x7fe9a00fba40 msgr2=0x7fe9a00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 -- 172.21.15.103:0/3515868537 shutdown_connections 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 -- 172.21.15.103:0/3515868537 wait complete. 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.843+0000 7fe9a5b27640 1 Processor -- start 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 -- start start 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 0x7fe9a0109940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a0101490 0x7fe9a019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9a0101e50 0x7fe9a019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:12.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a010c750 con 0x7fe9a0101490 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a010c5d0 con 0x7fe9a0100290 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe9a5b27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a010c8d0 con 0x7fe9a0101e50 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 0x7fe9a0109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 0x7fe9a0109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60810/0 (socket says 172.21.15.103:60810) 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 -- 172.21.15.103:0/823788815 learned_addr learned my addr 172.21.15.103:0/823788815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99effd640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a0101490 0x7fe9a019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99ffff640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9a0101e50 0x7fe9a019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 -- 172.21.15.103:0/823788815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9a0101e50 msgr2=0x7fe9a019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9a0101e50 0x7fe9a019f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 -- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a0101490 msgr2=0x7fe9a019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a0101490 0x7fe9a019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:12.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.844+0000 7fe99f7fe640 1 -- 172.21.15.103:0/823788815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a019fc80 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.845+0000 7fe99f7fe640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 0x7fe9a0109940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe99000b9c0 tx=0x7fe99000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.845+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990019070 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.845+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a019ff70 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.845+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe99000cba0 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.845+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900125a0 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.845+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a01a03f0 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.846+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a0107820 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.847+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe99001c070 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.848+0000 7fe99cff9640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe98c0a3240 0x7fe98c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.848+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4669+0+0 (secure 0 0 0) 0x7fe9900ce0a0 con 0x7fe9a0100290 2024-09-19T05:01:12.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.848+0000 7fe99effd640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe98c0a3240 0x7fe98c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:12.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.848+0000 7fe99effd640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe98c0a3240 0x7fe98c0a5700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe9880314e0 tx=0x7fe988005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:12.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:12.852+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe99009b5b0 con 0x7fe9a0100290 2024-09-19T05:01:13.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:13.028+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe9a0062630 con 0x7fe98c0a3240 2024-09-19T05:01:13.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:01:13.149+0000 7f6d75ca1640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T05:01:13.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:01:13.161+0000 7f6d722db640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi064 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:13.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi064 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:13.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi064 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:14.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:14 smithi103 ceph-mon[34446]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:14.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:14 smithi103 ceph-mon[34446]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:14.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:14 smithi103 ceph-mon[34446]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:01:14.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:14 smithi103 ceph-mon[34446]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T05:01:14.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:14.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:14 smithi174 ceph-mon[34488]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:14 smithi174 ceph-mon[34488]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:14 smithi174 ceph-mon[34488]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:01:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:14 smithi174 ceph-mon[34488]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T05:01:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:14 smithi064 ceph-mon[30960]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:14 smithi064 ceph-mon[30960]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:14 smithi064 ceph-mon[30960]: from='osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:01:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:14 smithi064 ceph-mon[30960]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T05:01:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:14.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:15 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:01:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:15 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:01:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:15 smithi103 ceph-mon[34446]: osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] boot 2024-09-19T05:01:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:15 smithi103 ceph-mon[34446]: osdmap e41: 7 total, 7 up, 7 in 2024-09-19T05:01:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:15 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:01:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:15 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:01:15.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:15 smithi174 ceph-mon[34488]: osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] boot 2024-09-19T05:01:15.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:15 smithi174 ceph-mon[34488]: osdmap e41: 7 total, 7 up, 7 in 2024-09-19T05:01:15.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:15.551 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:15 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:01:15.551 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:15 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:01:15.551 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:15 smithi064 ceph-mon[30960]: osd.6 [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] boot 2024-09-19T05:01:15.551 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:15 smithi064 ceph-mon[30960]: osdmap e41: 7 total, 7 up, 7 in 2024-09-19T05:01:15.551 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:01:16.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/2025961427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]: dispatch 2024-09-19T05:01:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]: dispatch 2024-09-19T05:01:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]': finished 2024-09-19T05:01:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: osdmap e42: 8 total, 7 up, 8 in 2024-09-19T05:01:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:16 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/3059800805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/2025961427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]: dispatch 2024-09-19T05:01:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]: dispatch 2024-09-19T05:01:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]': finished 2024-09-19T05:01:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: osdmap e42: 8 total, 7 up, 8 in 2024-09-19T05:01:16.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:16.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:16 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/3059800805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/2025961427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]: dispatch 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]: dispatch 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29311352-ffd9-4b21-9876-4bb7c8a522b3"}]': finished 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: osdmap e42: 8 total, 7 up, 8 in 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:16.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:16 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/3059800805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:18.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:18 smithi103 ceph-mon[34446]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:18.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:18 smithi174 ceph-mon[34488]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:18.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:18 smithi064 ceph-mon[30960]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:20.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:20 smithi103 ceph-mon[34446]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:20.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:20 smithi174 ceph-mon[34488]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:20.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:20 smithi064 ceph-mon[30960]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:22.198 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:22 smithi103 ceph-mon[34446]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:22.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:22 smithi174 ceph-mon[34488]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:22.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:22 smithi064 ceph-mon[30960]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:24.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:24 smithi103 ceph-mon[34446]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:24 smithi174 ceph-mon[34488]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:24.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:24 smithi064 ceph-mon[30960]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:25.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:01:25.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:25 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:25.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:25 smithi103 ceph-mon[34446]: Deploying daemon osd.7 on smithi103 2024-09-19T05:01:25.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:01:25.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:25 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:25.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:25 smithi174 ceph-mon[34488]: Deploying daemon osd.7 on smithi103 2024-09-19T05:01:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:01:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:25 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:25 smithi064 ceph-mon[30960]: Deploying daemon osd.7 on smithi103 2024-09-19T05:01:26.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:26 smithi103 ceph-mon[34446]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:26.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:26 smithi174 ceph-mon[34488]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:26.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:26 smithi064 ceph-mon[30960]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:28.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:28 smithi103 ceph-mon[34446]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:28 smithi174 ceph-mon[34488]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:28.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:28 smithi064 ceph-mon[30960]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:30.448 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:30 smithi103 ceph-mon[34446]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:30.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:30 smithi174 ceph-mon[34488]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:30.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:30 smithi064 ceph-mon[30960]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:31.462 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:31.462 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:31.462 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:31.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:31.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:31.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:31.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:31.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:32.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:32 smithi174 ceph-mon[34488]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:32.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:32 smithi103 ceph-mon[34446]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:32.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:32 smithi064 ceph-mon[30960]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.953+0000 7fe99cff9640 1 -- 172.21.15.103:0/823788815 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe9a0062630 con 0x7fe98c0a3240 2024-09-19T05:01:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 7 on host 'smithi103' 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe98c0a3240 msgr2=0x7fe98c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe98c0a3240 0x7fe98c0a5700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe9880314e0 tx=0x7fe988005e80 comp rx=0 tx=0).stop 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 msgr2=0x7fe9a0109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 0x7fe9a0109940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe99000b9c0 tx=0x7fe99000be80 comp rx=0 tx=0).stop 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 shutdown_connections 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe98c0a3240 0x7fe98c0a5700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9a0101e50 0x7fe9a019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a0101490 0x7fe9a019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 --2- 172.21.15.103:0/823788815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9a0100290 0x7fe9a0109940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 >> 172.21.15.103:0/823788815 conn(0x7fe9a00fba40 msgr2=0x7fe9a01049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 shutdown_connections 2024-09-19T05:01:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:01:33.955+0000 7fe9a5b27640 1 -- 172.21.15.103:0/823788815 wait complete. 2024-09-19T05:01:34.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:34.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:34.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:34.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:34.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:34.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:34.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:34.618 DEBUG:teuthology.orchestra.run.smithi103:osd.7> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T05:01:34.620 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-09-19T05:01:34.621 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:01:34.876 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:01:36.292 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:36 smithi174 ceph-mon[34488]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:36.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:36 smithi064 ceph-mon[30960]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:36.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:36 smithi103 ceph-mon[34446]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:36.969 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:01:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:01:36.568+0000 7fb820f76640 -1 osd.7 0 log_to_monitors true 2024-09-19T05:01:37.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:37 smithi174 ceph-mon[34488]: from='osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:01:37.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:37 smithi174 ceph-mon[34488]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:01:37.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:37 smithi103 ceph-mon[34446]: from='osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:01:37.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:37 smithi103 ceph-mon[34446]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:01:37.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:37 smithi064 ceph-mon[30960]: from='osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:01:37.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:37 smithi064 ceph-mon[30960]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:01:37.426 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T05:01:37.452 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-09-19T05:01:37.704 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:01:38.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 -- 172.21.15.174:0/3425187359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2241037b0 msgr2=0x7fb22410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 --2- 172.21.15.174:0/3425187359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2241037b0 0x7fb22410a040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb214009900 tx=0x7fb21402a410 comp rx=0 tx=0).stop 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 -- 172.21.15.174:0/3425187359 shutdown_connections 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 --2- 172.21.15.174:0/3425187359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2241037b0 0x7fb22410a040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 --2- 172.21.15.174:0/3425187359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb224102df0 0x7fb224103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 --2- 172.21.15.174:0/3425187359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb224101bf0 0x7fb224101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 -- 172.21.15.174:0/3425187359 >> 172.21.15.174:0/3425187359 conn(0x7fb2240fd3c0 msgr2=0x7fb2240ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 -- 172.21.15.174:0/3425187359 shutdown_connections 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.193+0000 7fb22884d640 1 -- 172.21.15.174:0/3425187359 wait complete. 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 Processor -- start 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 -- start start 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb224101bf0 0x7fb224197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 0x7fb2241981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2241037b0 0x7fb22419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d700 con 0x7fb224101bf0 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d580 con 0x7fb2241037b0 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb22884d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d880 con 0x7fb224102df0 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb221d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 0x7fb2241981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb221d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 0x7fb2241981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:58556/0 (socket says 172.21.15.174:58556) 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.194+0000 7fb221d9b640 1 -- 172.21.15.174:0/3573179580 learned_addr learned my addr 172.21.15.174:0/3573179580 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T05:01:38.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb22259c640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb224101bf0 0x7fb224197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb222d9d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2241037b0 0x7fb22419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb221d9b640 1 -- 172.21.15.174:0/3573179580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2241037b0 msgr2=0x7fb22419f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb221d9b640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2241037b0 0x7fb22419f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb221d9b640 1 -- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb224101bf0 msgr2=0x7fb224197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb221d9b640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb224101bf0 0x7fb224197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb221d9b640 1 -- 172.21.15.174:0/3573179580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22419f8b0 con 0x7fb224102df0 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb221d9b640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 0x7fb2241981d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c920 tx=0x7fb20c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007c70 con 0x7fb224102df0 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22419fba0 con 0x7fb224102df0 2024-09-19T05:01:38.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb20c007e10 con 0x7fb224102df0 2024-09-19T05:01:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c018820 con 0x7fb224102df0 2024-09-19T05:01:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.195+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2241a00e0 con 0x7fb224102df0 2024-09-19T05:01:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.196+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb224108f20 con 0x7fb224102df0 2024-09-19T05:01:38.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.197+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb20c0189c0 con 0x7fb224102df0 2024-09-19T05:01:38.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.198+0000 7fb21b7fe640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2100a3240 0x7fb2100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.198+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5117+0+0 (secure 0 0 0) 0x7fb20c0cf1d0 con 0x7fb224102df0 2024-09-19T05:01:38.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.199+0000 7fb22259c640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2100a3240 0x7fb2100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.199+0000 7fb22259c640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2100a3240 0x7fb2100a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005c40 tx=0x7fb21c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:38.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.201+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb20c014030 con 0x7fb224102df0 2024-09-19T05:01:38.380 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:38.378+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb224062630 con 0x7fb2100a3240 2024-09-19T05:01:38.380 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:38.380 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:01:38.380 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:01:38.380 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: osdmap e43: 8 total, 7 up, 8 in 2024-09-19T05:01:38.380 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:38.381 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: osdmap e43: 8 total, 7 up, 8 in 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:38.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.468 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:01:38.049+0000 7fb81c6d6640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T05:01:38.469 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:01:38.062+0000 7fb818d10640 -1 osd.7 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: osdmap e43: 8 total, 7 up, 8 in 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:01:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:39.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi103 2024-09-19T05:01:39.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] boot 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: osdmap e45: 8 total, 8 up, 8 in 2024-09-19T05:01:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi103 2024-09-19T05:01:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T05:01:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:01:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] boot 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: osdmap e45: 8 total, 8 up, 8 in 2024-09-19T05:01:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi103 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: osd.7 [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] boot 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: osdmap e45: 8 total, 8 up, 8 in 2024-09-19T05:01:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:01:40.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:40 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:01:40.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:40 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:01:40.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:40 smithi174 ceph-mon[34488]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:40.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:40 smithi174 ceph-mon[34488]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:40.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:40 smithi174 ceph-mon[34488]: osdmap e46: 8 total, 8 up, 8 in 2024-09-19T05:01:40.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:40 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:01:40.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:40 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:01:40.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:40 smithi064 ceph-mon[30960]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:40.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:40 smithi064 ceph-mon[30960]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:40.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:40 smithi064 ceph-mon[30960]: osdmap e46: 8 total, 8 up, 8 in 2024-09-19T05:01:40.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:40 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:01:40.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:40 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:01:40.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:40 smithi103 ceph-mon[34446]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:01:40.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:40 smithi103 ceph-mon[34446]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:40.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:40 smithi103 ceph-mon[34446]: osdmap e46: 8 total, 8 up, 8 in 2024-09-19T05:01:41.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:41 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/1596857997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]: dispatch 2024-09-19T05:01:41.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:41 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]: dispatch 2024-09-19T05:01:41.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:41 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]': finished 2024-09-19T05:01:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:41 smithi103 ceph-mon[34446]: osdmap e47: 9 total, 8 up, 9 in 2024-09-19T05:01:41.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:41.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:41 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/1596857997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]: dispatch 2024-09-19T05:01:41.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:41 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]: dispatch 2024-09-19T05:01:41.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:41 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]': finished 2024-09-19T05:01:41.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:41 smithi174 ceph-mon[34488]: osdmap e47: 9 total, 8 up, 9 in 2024-09-19T05:01:41.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:41 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/1596857997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]: dispatch 2024-09-19T05:01:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:41 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]: dispatch 2024-09-19T05:01:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:41 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae077168-4b5f-44a0-a269-03f13dba90b1"}]': finished 2024-09-19T05:01:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:41 smithi064 ceph-mon[30960]: osdmap e47: 9 total, 8 up, 9 in 2024-09-19T05:01:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:42.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:42 smithi064 ceph-mon[30960]: pgmap v167: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:42.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:42 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/3946326680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:42 smithi064 ceph-mon[30960]: osdmap e48: 9 total, 8 up, 9 in 2024-09-19T05:01:42.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:42.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:42 smithi103 ceph-mon[34446]: pgmap v167: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:42.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:42 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/3946326680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:42.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:42 smithi103 ceph-mon[34446]: osdmap e48: 9 total, 8 up, 9 in 2024-09-19T05:01:42.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:42.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:42 smithi174 ceph-mon[34488]: pgmap v167: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:42.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:42 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/3946326680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:42 smithi174 ceph-mon[34488]: osdmap e48: 9 total, 8 up, 9 in 2024-09-19T05:01:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:43.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:43 smithi064 ceph-mon[30960]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:43.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:43 smithi064 ceph-mon[30960]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T05:01:43.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:43.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:43 smithi103 ceph-mon[34446]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:43.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:43 smithi103 ceph-mon[34446]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T05:01:43.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:44.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:43 smithi174 ceph-mon[34488]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:44.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:43 smithi174 ceph-mon[34488]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T05:01:44.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:01:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:45 smithi174 ceph-mon[34488]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:45.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:45 smithi064 ceph-mon[30960]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:45.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:45 smithi103 ceph-mon[34446]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:47.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:47 smithi064 ceph-mon[30960]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:47.963 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:47 smithi174 ceph-mon[34488]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:47.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:47 smithi103 ceph-mon[34446]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:49.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:49 smithi174 ceph-mon[34488]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:49.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:49 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:01:49.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:49 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:49 smithi064 ceph-mon[30960]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:49 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:01:49.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:49 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:49.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:49 smithi103 ceph-mon[34446]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:49.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:49 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:01:49.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:49 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:50.881 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:50 smithi174 ceph-mon[34488]: Deploying daemon osd.8 on smithi174 2024-09-19T05:01:50.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:50 smithi064 ceph-mon[30960]: Deploying daemon osd.8 on smithi174 2024-09-19T05:01:50.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:50 smithi103 ceph-mon[34446]: Deploying daemon osd.8 on smithi174 2024-09-19T05:01:51.881 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:51 smithi174 ceph-mon[34488]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:51.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:51 smithi064 ceph-mon[30960]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:51.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:51 smithi103 ceph-mon[34446]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:53.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:53 smithi064 ceph-mon[30960]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:53.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:53 smithi103 ceph-mon[34446]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:54.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:53 smithi174 ceph-mon[34488]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:55.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:55 smithi064 ceph-mon[30960]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:55.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:55.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:55.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:55.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:55 smithi103 ceph-mon[34446]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:55.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:55 smithi174 ceph-mon[34488]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:55.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:55.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:55.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:57.728 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:57 smithi174 ceph-mon[34488]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:57 smithi064 ceph-mon[30960]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:57.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:57 smithi103 ceph-mon[34446]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:01:59.067 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-09-19T05:01:59.067 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.066+0000 7fb21b7fe640 1 -- 172.21.15.174:0/3573179580 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb224062630 con 0x7fb2100a3240 2024-09-19T05:01:59.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2100a3240 msgr2=0x7fb2100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:59.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2100a3240 0x7fb2100a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005c40 tx=0x7fb21c005bd0 comp rx=0 tx=0).stop 2024-09-19T05:01:59.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 msgr2=0x7fb2241981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:59.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 0x7fb2241981d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c920 tx=0x7fb20c00cde0 comp rx=0 tx=0).stop 2024-09-19T05:01:59.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 shutdown_connections 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2100a3240 0x7fb2100a5700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2241037b0 0x7fb22419f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb224102df0 0x7fb2241981d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 --2- 172.21.15.174:0/3573179580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb224101bf0 0x7fb224197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.068+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 >> 172.21.15.174:0/3573179580 conn(0x7fb2240fd3c0 msgr2=0x7fb2241060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.069+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 shutdown_connections 2024-09-19T05:01:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:01:59.069+0000 7fb22884d640 1 -- 172.21.15.174:0/3573179580 wait complete. 2024-09-19T05:01:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:01:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:59.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:59.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:59.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:01:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:01:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:01:59.786 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T05:01:59.788 INFO:tasks.cephadm:Deploying osd.9 on smithi064 with /dev/vg_nvme/lv_1... 2024-09-19T05:01:59.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:02:00.055 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:00.370 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:00 smithi103 ceph-mon[34446]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:00 smithi064 ceph-mon[30960]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:00.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:00 smithi174 ceph-mon[34488]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:01.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:02:01 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:02:01.365+0000 7fc65180b640 -1 osd.8 0 log_to_monitors true 2024-09-19T05:02:02.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:02 smithi064 ceph-mon[30960]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:02.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:02 smithi064 ceph-mon[30960]: from='osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:02:02.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:02 smithi064 ceph-mon[30960]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:02:02.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:02 smithi174 ceph-mon[34488]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:02.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:02 smithi174 ceph-mon[34488]: from='osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:02:02.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:02 smithi174 ceph-mon[34488]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:02:02.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:02 smithi103 ceph-mon[34446]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:02.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:02 smithi103 ceph-mon[34446]: from='osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:02:02.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:02 smithi103 ceph-mon[34446]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:02:02.647 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:02:02.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-09-19T05:02:02.931 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:03.315 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:02:03.315 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:03.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 -- 172.21.15.64:0/430094093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c074c10 msgr2=0x7fee0c075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 --2- 172.21.15.64:0/430094093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c074c10 0x7fee0c075070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009860 tx=0x7fedfc02a3d0 comp rx=0 tx=0).stop 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 -- 172.21.15.64:0/430094093 shutdown_connections 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 --2- 172.21.15.64:0/430094093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c109a80 0x7fee0c10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 --2- 172.21.15.64:0/430094093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c074c10 0x7fee0c075070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 --2- 172.21.15.64:0/430094093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c0761e0 0x7fee0c074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.341+0000 7fee140e9640 1 -- 172.21.15.64:0/430094093 >> 172.21.15.64:0/430094093 conn(0x7fee0c0fd390 msgr2=0x7fee0c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:03.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.342+0000 7fee140e9640 1 -- 172.21.15.64:0/430094093 shutdown_connections 2024-09-19T05:02:03.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.342+0000 7fee140e9640 1 -- 172.21.15.64:0/430094093 wait complete. 2024-09-19T05:02:03.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.342+0000 7fee140e9640 1 Processor -- start 2024-09-19T05:02:03.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.342+0000 7fee140e9640 1 -- start start 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee140e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c074c10 0x7fee0c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee140e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 0x7fee0c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee140e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c109a80 0x7fee0c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee140e9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c10f910 con 0x7fee0c0761e0 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee140e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c10f790 con 0x7fee0c074c10 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee140e9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c10fa90 con 0x7fee0c109a80 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 0x7fee0c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 0x7fee0c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49670/0 (socket says 172.21.15.64:49670) 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 -- 172.21.15.64:0/4112426220 learned_addr learned my addr 172.21.15.64:0/4112426220 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee12686640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c109a80 0x7fee0c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:03.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11e85640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c074c10 0x7fee0c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 -- 172.21.15.64:0/4112426220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c109a80 msgr2=0x7fee0c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c109a80 0x7fee0c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 -- 172.21.15.64:0/4112426220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c074c10 msgr2=0x7fee0c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c074c10 0x7fee0c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 -- 172.21.15.64:0/4112426220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c1a1bf0 con 0x7fee0c0761e0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee11684640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 0x7fee0c19a500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009980 tx=0x7fedfc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.343+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc040070 con 0x7fee0c0761e0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.344+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c1a1e80 con 0x7fee0c0761e0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.344+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedfc030de0 con 0x7fee0c0761e0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.344+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc045460 con 0x7fee0c0761e0 2024-09-19T05:02:03.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.344+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1a22d0 con 0x7fee0c0761e0 2024-09-19T05:02:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.345+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c108840 con 0x7fee0c0761e0 2024-09-19T05:02:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.346+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fedfc039040 con 0x7fee0c0761e0 2024-09-19T05:02:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.347+0000 7fee02ffd640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf80a3270 0x7fedf80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.347+0000 7fee11e85640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf80a3270 0x7fedf80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:03.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.347+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5549+0+0 (secure 0 0 0) 0x7fedfc0eff90 con 0x7fee0c0761e0 2024-09-19T05:02:03.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.347+0000 7fee11e85640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf80a3270 0x7fedf80a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee08005fd0 tx=0x7fee08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:03.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.351+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fedfc0bd780 con 0x7fee0c0761e0 2024-09-19T05:02:03.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:03.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:03.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:03.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:03.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:03.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:03.536+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fee0c062630 con 0x7fedf80a3270 2024-09-19T05:02:04.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:04.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi174 2024-09-19T05:02:04.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T05:02:04.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:02:04.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T05:02:04.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:04.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:04.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:04.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:04.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:04.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:04 smithi064 ceph-mon[30960]: from='osd.8 ' entity='osd.8' 2024-09-19T05:02:04.277 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:02:03.958+0000 7fc64dfa2640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T05:02:04.277 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:02:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:02:03.973+0000 7fc64559d640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi174 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:04 smithi174 ceph-mon[34488]: from='osd.8 ' entity='osd.8' 2024-09-19T05:02:04.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi174 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:04.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:04 smithi103 ceph-mon[34446]: from='osd.8 ' entity='osd.8' 2024-09-19T05:02:05.315 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:02:05.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:02:05.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:05.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:05.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] boot 2024-09-19T05:02:05.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: osdmap e52: 9 total, 9 up, 9 in 2024-09-19T05:02:05.316 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:05.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:02:05.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:02:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] boot 2024-09-19T05:02:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: osdmap e52: 9 total, 9 up, 9 in 2024-09-19T05:02:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:02:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:02:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: osd.8 [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] boot 2024-09-19T05:02:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: osdmap e52: 9 total, 9 up, 9 in 2024-09-19T05:02:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:02:06.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:06 smithi064 ceph-mon[30960]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:06.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:06 smithi064 ceph-mon[30960]: osdmap e53: 9 total, 9 up, 9 in 2024-09-19T05:02:06.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:06 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2092513136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1767cc43-897a-408c-9e10-ecabe9b39731"}]: dispatch 2024-09-19T05:02:06.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:06 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2092513136' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1767cc43-897a-408c-9e10-ecabe9b39731"}]': finished 2024-09-19T05:02:06.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:06 smithi064 ceph-mon[30960]: osdmap e54: 10 total, 9 up, 10 in 2024-09-19T05:02:06.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:06 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:06 smithi103 ceph-mon[34446]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:06 smithi103 ceph-mon[34446]: osdmap e53: 9 total, 9 up, 9 in 2024-09-19T05:02:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:06 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2092513136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1767cc43-897a-408c-9e10-ecabe9b39731"}]: dispatch 2024-09-19T05:02:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:06 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2092513136' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1767cc43-897a-408c-9e10-ecabe9b39731"}]': finished 2024-09-19T05:02:06.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:06 smithi103 ceph-mon[34446]: osdmap e54: 10 total, 9 up, 10 in 2024-09-19T05:02:06.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:06 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:06.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:06 smithi174 ceph-mon[34488]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:02:06.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:06 smithi174 ceph-mon[34488]: osdmap e53: 9 total, 9 up, 9 in 2024-09-19T05:02:06.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:06 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2092513136' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1767cc43-897a-408c-9e10-ecabe9b39731"}]: dispatch 2024-09-19T05:02:06.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:06 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2092513136' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1767cc43-897a-408c-9e10-ecabe9b39731"}]': finished 2024-09-19T05:02:06.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:06 smithi174 ceph-mon[34488]: osdmap e54: 10 total, 9 up, 10 in 2024-09-19T05:02:06.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:06 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:07 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3137996734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:07 smithi064 ceph-mon[30960]: osdmap e55: 10 total, 9 up, 10 in 2024-09-19T05:02:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:07 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:07.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:07 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3137996734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:07.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:07 smithi103 ceph-mon[34446]: osdmap e55: 10 total, 9 up, 10 in 2024-09-19T05:02:07.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:07 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:07 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3137996734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:07 smithi174 ceph-mon[34488]: osdmap e55: 10 total, 9 up, 10 in 2024-09-19T05:02:07.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:07 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:08 smithi103 ceph-mon[34446]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:08 smithi103 ceph-mon[34446]: osdmap e56: 10 total, 9 up, 10 in 2024-09-19T05:02:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:08 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:08 smithi174 ceph-mon[34488]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:08 smithi174 ceph-mon[34488]: osdmap e56: 10 total, 9 up, 10 in 2024-09-19T05:02:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:08 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:08 smithi064 ceph-mon[30960]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:08 smithi064 ceph-mon[30960]: osdmap e56: 10 total, 9 up, 10 in 2024-09-19T05:02:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:08 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:10.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:10 smithi103 ceph-mon[34446]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:10 smithi174 ceph-mon[34488]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:10.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:10 smithi064 ceph-mon[30960]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:12.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:12 smithi174 ceph-mon[34488]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T05:02:12.457 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:12 smithi064 ceph-mon[30960]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T05:02:12.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:12 smithi103 ceph-mon[34446]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T05:02:14.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:14 smithi103 ceph-mon[34446]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T05:02:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:14 smithi174 ceph-mon[34488]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T05:02:14.543 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:14 smithi064 ceph-mon[30960]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T05:02:15.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:02:15.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:15 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:02:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:15 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:02:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:15 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:16.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:16 smithi064 ceph-mon[30960]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:02:16.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:16 smithi064 ceph-mon[30960]: Deploying daemon osd.9 on smithi064 2024-09-19T05:02:16.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:16 smithi103 ceph-mon[34446]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:02:16.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:16 smithi103 ceph-mon[34446]: Deploying daemon osd.9 on smithi064 2024-09-19T05:02:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:16 smithi174 ceph-mon[34488]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:02:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:16 smithi174 ceph-mon[34488]: Deploying daemon osd.9 on smithi064 2024-09-19T05:02:18.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:18 smithi064 ceph-mon[30960]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:02:18.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:18 smithi103 ceph-mon[34446]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:02:18.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:18 smithi174 ceph-mon[34488]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:02:20.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:20 smithi064 ceph-mon[30960]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:02:20.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:20 smithi103 ceph-mon[34446]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:02:20.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:20 smithi174 ceph-mon[34488]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:02:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:21 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:21 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:21 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:22.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:22 smithi174 ceph-mon[34488]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:02:22.589 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:22 smithi064 ceph-mon[30960]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:02:22.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:22 smithi103 ceph-mon[34446]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:02:23.479 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 9 on host 'smithi064' 2024-09-19T05:02:23.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.477+0000 7fee02ffd640 1 -- 172.21.15.64:0/4112426220 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee0c062630 con 0x7fedf80a3270 2024-09-19T05:02:23.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf80a3270 msgr2=0x7fedf80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:23.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf80a3270 0x7fedf80a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee08005fd0 tx=0x7fee08009260 comp rx=0 tx=0).stop 2024-09-19T05:02:23.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 msgr2=0x7fee0c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 0x7fee0c19a500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009980 tx=0x7fedfc005270 comp rx=0 tx=0).stop 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 shutdown_connections 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf80a3270 0x7fedf80a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c109a80 0x7fee0c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c0761e0 0x7fee0c19a500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 --2- 172.21.15.64:0/4112426220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c074c10 0x7fee0c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.479+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 >> 172.21.15.64:0/4112426220 conn(0x7fee0c0fd390 msgr2=0x7fee0c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.480+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 shutdown_connections 2024-09-19T05:02:23.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:02:23.480+0000 7fee140e9640 1 -- 172.21.15.64:0/4112426220 wait complete. 2024-09-19T05:02:24.036 DEBUG:teuthology.orchestra.run.smithi064:osd.9> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T05:02:24.039 INFO:tasks.cephadm:Deploying osd.10 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-19T05:02:24.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:02:24.300 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:02:24.420 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:24 smithi103 ceph-mon[34446]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:24.421 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:24 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.421 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:24 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.421 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:24 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.421 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:24 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:24 smithi174 ceph-mon[34488]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:24 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:24 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:24 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:24 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.616 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:24 smithi064 ceph-mon[30960]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:24.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:24 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:24 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:24 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:24.617 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:24 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:26.494 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:26 smithi103 ceph-mon[34446]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:26.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:26 smithi174 ceph-mon[34488]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:26.550 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:26 smithi064 ceph-mon[30960]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:26.901 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:02:26 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:02:26.547+0000 7fdb27bfc640 -1 osd.9 0 log_to_monitors true 2024-09-19T05:02:26.915 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T05:02:26.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-19T05:02:27.198 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:02:27.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:27 smithi174 ceph-mon[34488]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:02:27.606 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:27 smithi103 ceph-mon[34446]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:02:27.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:27 smithi064 ceph-mon[30960]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:02:27.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 -- 172.21.15.103:0/1413243052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90541078c0 msgr2=0x7f9054109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:27.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 --2- 172.21.15.103:0/1413243052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90541078c0 0x7f9054109d10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f904c009ca0 tx=0x7f904c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 -- 172.21.15.103:0/1413243052 shutdown_connections 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 --2- 172.21.15.103:0/1413243052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90541078c0 0x7f9054109d10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 --2- 172.21.15.103:0/1413243052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9054104f90 0x7f9054107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 --2- 172.21.15.103:0/1413243052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9054102660 0x7f9054104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 -- 172.21.15.103:0/1413243052 >> 172.21.15.103:0/1413243052 conn(0x7f90540fc280 msgr2=0x7f90540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 -- 172.21.15.103:0/1413243052 shutdown_connections 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 -- 172.21.15.103:0/1413243052 wait complete. 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 Processor -- start 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.690+0000 7f905a5cd640 1 -- start start 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f905a5cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9054102660 0x7f905419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f905a5cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 0x7f905419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:27.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f905a5cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90541078c0 0x7f90541a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f905a5cd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d7b0 con 0x7f9054102660 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f905a5cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d630 con 0x7f9054104f90 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f905a5cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f905410d930 con 0x7f90541078c0 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 0x7f905419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 0x7f905419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33504/0 (socket says 172.21.15.103:33504) 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 -- 172.21.15.103:0/2437697714 learned_addr learned my addr 172.21.15.103:0/2437697714 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f9053fff640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9054102660 0x7f905419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f9058b6a640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90541078c0 0x7f90541a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 -- 172.21.15.103:0/2437697714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90541078c0 msgr2=0x7f90541a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90541078c0 0x7f90541a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 -- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9054102660 msgr2=0x7f905419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:27.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9054102660 0x7f905419a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 -- 172.21.15.103:0/2437697714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90541a1c90 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90537fe640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 0x7f905419a5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f904800c970 tx=0x7f904800cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.691+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048007bb0 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.692+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90541a1f80 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.692+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9048007d50 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.692+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048018800 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.692+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541a23e0 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.693+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90541093b0 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.694+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f904800f450 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.694+0000 7f90517fa640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f903c0a3270 0x7f903c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.694+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5965+0+0 (secure 0 0 0) 0x7f90480cf0b0 con 0x7f9054104f90 2024-09-19T05:02:27.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.694+0000 7f9053fff640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f903c0a3270 0x7f903c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:27.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.695+0000 7f9053fff640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f903c0a3270 0x7f903c0a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9044009990 tx=0x7f9044009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:27.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.698+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9048014030 con 0x7f9054104f90 2024-09-19T05:02:27.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:27.874+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9054062630 con 0x7f903c0a3270 2024-09-19T05:02:28.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:28.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:02:28.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T05:02:28.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T05:02:28.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:28.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:28.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.537 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:28.537 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.537 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:28.537 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:28.537 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:28.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:28.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:28.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:28.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:29.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:02:29.101+0000 7fdb2335c640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T05:02:29.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:02:29.121+0000 7fdb1b98e640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi064 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:29.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' 2024-09-19T05:02:29.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:29 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.650 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi064 2024-09-19T05:02:29.651 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-19T05:02:29.651 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:29.651 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:02:29.651 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T05:02:29.651 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.652 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.652 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:29.652 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' 2024-09-19T05:02:29.652 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:29 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi064 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: from='osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919]' entity='osd.9' 2024-09-19T05:02:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:29 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:30.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:02:30.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:02:30.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] boot 2024-09-19T05:02:30.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: osdmap e59: 10 total, 10 up, 10 in 2024-09-19T05:02:30.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:30.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/59517741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]: dispatch 2024-09-19T05:02:30.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]: dispatch 2024-09-19T05:02:30.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]': finished 2024-09-19T05:02:30.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: osdmap e60: 11 total, 10 up, 11 in 2024-09-19T05:02:30.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:30 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:30.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:02:30.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:02:30.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] boot 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: osdmap e59: 10 total, 10 up, 10 in 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/59517741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]: dispatch 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]: dispatch 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]': finished 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: osdmap e60: 11 total, 10 up, 11 in 2024-09-19T05:02:30.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:30 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: osd.9 [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] boot 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: osdmap e59: 10 total, 10 up, 10 in 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/59517741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]: dispatch 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]: dispatch 2024-09-19T05:02:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b507cd6f-ed66-4b6e-9560-fca660e09350"}]': finished 2024-09-19T05:02:30.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: osdmap e60: 11 total, 10 up, 11 in 2024-09-19T05:02:30.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:30 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:31 smithi064 ceph-mon[30960]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:31 smithi064 ceph-mon[30960]: from='client.? 172.21.15.103:0/3950324748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:31 smithi103 ceph-mon[34446]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:31 smithi103 ceph-mon[34446]: from='client.? 172.21.15.103:0/3950324748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:31 smithi174 ceph-mon[34488]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:31 smithi174 ceph-mon[34488]: from='client.? 172.21.15.103:0/3950324748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:32.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:32 smithi103 ceph-mon[34446]: osdmap e61: 11 total, 10 up, 11 in 2024-09-19T05:02:32.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:32 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:32.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:32 smithi064 ceph-mon[30960]: osdmap e61: 11 total, 10 up, 11 in 2024-09-19T05:02:32.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:32 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:32.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:32 smithi174 ceph-mon[34488]: osdmap e61: 11 total, 10 up, 11 in 2024-09-19T05:02:32.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:32 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:33 smithi064 ceph-mon[30960]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:33.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:33 smithi103 ceph-mon[34446]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:33.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:33 smithi174 ceph-mon[34488]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:35.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:35 smithi103 ceph-mon[34446]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:35 smithi064 ceph-mon[30960]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:35 smithi174 ceph-mon[34488]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:37.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:37 smithi064 ceph-mon[30960]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:37.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:37 smithi103 ceph-mon[34446]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:37.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:37 smithi174 ceph-mon[34488]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:39 smithi064 ceph-mon[30960]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:02:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:39 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:39 smithi064 ceph-mon[30960]: Deploying daemon osd.10 on smithi103 2024-09-19T05:02:39.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:39 smithi103 ceph-mon[34446]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:39.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:02:39.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:39 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:39.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:39 smithi103 ceph-mon[34446]: Deploying daemon osd.10 on smithi103 2024-09-19T05:02:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:39 smithi174 ceph-mon[34488]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:02:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:39 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:39.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:39 smithi174 ceph-mon[34488]: Deploying daemon osd.10 on smithi103 2024-09-19T05:02:41.544 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:41 smithi103 ceph-mon[34446]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:41.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:41 smithi064 ceph-mon[30960]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:41 smithi174 ceph-mon[34488]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:43 smithi064 ceph-mon[30960]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:43.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:43 smithi103 ceph-mon[34446]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:43 smithi174 ceph-mon[34488]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:45.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:45 smithi064 ceph-mon[30960]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:45.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:45.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:45.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:45 smithi103 ceph-mon[34446]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:45 smithi174 ceph-mon[34488]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:47.570 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:47 smithi103 ceph-mon[34446]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:47.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:47 smithi064 ceph-mon[30960]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:47.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:47 smithi174 ceph-mon[34488]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:48.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.556+0000 7f90517fa640 1 -- 172.21.15.103:0/2437697714 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9054062630 con 0x7f903c0a3270 2024-09-19T05:02:48.558 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 10 on host 'smithi103' 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f903c0a3270 msgr2=0x7f903c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f903c0a3270 0x7f903c0a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9044009990 tx=0x7f9044009110 comp rx=0 tx=0).stop 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 msgr2=0x7f905419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 0x7f905419a5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f904800c970 tx=0x7f904800cd70 comp rx=0 tx=0).stop 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 shutdown_connections 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f903c0a3270 0x7f903c0a5730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90541078c0 0x7f90541a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9054104f90 0x7f905419a5a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 --2- 172.21.15.103:0/2437697714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9054102660 0x7f905419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 >> 172.21.15.103:0/2437697714 conn(0x7f90540fc280 msgr2=0x7f9054101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:48.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 shutdown_connections 2024-09-19T05:02:48.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T05:02:48.558+0000 7f905a5cd640 1 -- 172.21.15.103:0/2437697714 wait complete. 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:48.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:49.222 DEBUG:teuthology.orchestra.run.smithi103:osd.10> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T05:02:49.225 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-09-19T05:02:49.225 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:02:49.482 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:02:49.920 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:49 smithi174 ceph-mon[34488]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:49.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:49 smithi103 ceph-mon[34446]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:50.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:49 smithi064 ceph-mon[30960]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:51.700 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:02:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:02:51.238+0000 7f9b0bb9f640 -1 osd.10 0 log_to_monitors true 2024-09-19T05:02:51.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:51 smithi174 ceph-mon[34488]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:51.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:51 smithi174 ceph-mon[34488]: from='osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:02:51.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:51 smithi174 ceph-mon[34488]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:02:51.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:51 smithi103 ceph-mon[34446]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:51.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:51 smithi103 ceph-mon[34446]: from='osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:02:51.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:51 smithi103 ceph-mon[34446]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:02:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:51 smithi064 ceph-mon[30960]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:51 smithi064 ceph-mon[30960]: from='osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:02:52.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:51 smithi064 ceph-mon[30960]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:02:52.160 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T05:02:52.188 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-09-19T05:02:52.441 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:02:52.737 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:02:52.737 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: osdmap e62: 11 total, 10 up, 11 in 2024-09-19T05:02:52.737 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:52.737 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:02:52.737 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi103 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:52 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 -- 172.21.15.174:0/759677831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff300107900 msgr2=0x7ff300109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 --2- 172.21.15.174:0/759677831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff300107900 0x7ff300109d50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ca0 tx=0x7ff2f802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 -- 172.21.15.174:0/759677831 shutdown_connections 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 --2- 172.21.15.174:0/759677831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff300107900 0x7ff300109d50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 --2- 172.21.15.174:0/759677831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff300104fd0 0x7ff3001073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 --2- 172.21.15.174:0/759677831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3001026a0 0x7ff300104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 -- 172.21.15.174:0/759677831 >> 172.21.15.174:0/759677831 conn(0x7ff3000fc2a0 msgr2=0x7ff3000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 -- 172.21.15.174:0/759677831 shutdown_connections 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.917+0000 7ff305b8e640 1 -- 172.21.15.174:0/759677831 wait complete. 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 Processor -- start 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 -- start start 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 0x7ff300197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff300104fd0 0x7ff3001983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff300107900 0x7ff30019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff30010d770 con 0x7ff300107900 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff30010d5f0 con 0x7ff300104fd0 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff305b8e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff30010d8f0 con 0x7ff3001026a0 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 0x7ff300197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 0x7ff300197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50400/0 (socket says 172.21.15.174:50400) 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff2ff7fe640 1 -- 172.21.15.174:0/3749438502 learned_addr learned my addr 172.21.15.174:0/3749438502 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff2feffd640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff300104fd0 0x7ff3001983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.918+0000 7ff2fffff640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff300107900 0x7ff30019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2ff7fe640 1 -- 172.21.15.174:0/3749438502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff300104fd0 msgr2=0x7ff3001983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2ff7fe640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff300104fd0 0x7ff3001983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2ff7fe640 1 -- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff300107900 msgr2=0x7ff30019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2ff7fe640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff300107900 0x7ff30019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2ff7fe640 1 -- 172.21.15.174:0/3749438502 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30019fab0 con 0x7ff3001026a0 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2fffff640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff300107900 0x7ff30019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2ff7fe640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 0x7ff300197e80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400d950 tx=0x7ff2f400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4017070 con 0x7ff3001026a0 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f4009c30 con 0x7ff3001026a0 2024-09-19T05:02:52.937 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30019fda0 con 0x7ff3001026a0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f620 con 0x7ff3001026a0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.919+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001a0200 con 0x7ff3001026a0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.920+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3000fdd90 con 0x7ff3001026a0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.921+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff2f40109a0 con 0x7ff3001026a0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.922+0000 7ff2fcff9640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff2e80a3270 0x7ff2e80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.922+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6414+0+0 (secure 0 0 0) 0x7ff2f40ceda0 con 0x7ff3001026a0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.922+0000 7ff2feffd640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff2e80a3270 0x7ff2e80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.922+0000 7ff2feffd640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff2e80a3270 0x7ff2e80a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3001993a0 tx=0x7ff2f000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:52.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:52.926+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2f409c230 con 0x7ff3001026a0 2024-09-19T05:02:52.968 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:02:52.705+0000 7f9b072ff640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T05:02:52.968 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:02:52.718+0000 7f9b03939640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: osdmap e62: 11 total, 10 up, 11 in 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi103 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:52.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T05:02:52.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:52.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:52.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:52.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:52 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:53.103 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:02:53.101+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff300062630 con 0x7ff2e80a3270 2024-09-19T05:02:53.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:02:53.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: osdmap e62: 11 total, 10 up, 11 in 2024-09-19T05:02:53.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi103 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:53.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:02:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T05:02:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:53.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:52 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:02:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:02:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T05:02:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:53.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:53.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:54.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:02:54.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T05:02:54.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T05:02:54.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:54.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:54.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:54.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:02:54.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:02:54.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:54 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:02:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:54 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:02:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:54 smithi174 ceph-mon[34488]: osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] boot 2024-09-19T05:02:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:54 smithi174 ceph-mon[34488]: osdmap e64: 11 total, 11 up, 11 in 2024-09-19T05:02:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:54 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:54.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:54 smithi174 ceph-mon[34488]: osdmap e65: 11 total, 11 up, 11 in 2024-09-19T05:02:54.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:54 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:02:54.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:54 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:02:54.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:54 smithi103 ceph-mon[34446]: osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] boot 2024-09-19T05:02:54.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:54 smithi103 ceph-mon[34446]: osdmap e64: 11 total, 11 up, 11 in 2024-09-19T05:02:54.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:54 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:54.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:54 smithi103 ceph-mon[34446]: osdmap e65: 11 total, 11 up, 11 in 2024-09-19T05:02:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:54 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:02:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:54 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:02:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:54 smithi064 ceph-mon[30960]: osd.10 [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] boot 2024-09-19T05:02:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:54 smithi064 ceph-mon[30960]: osdmap e64: 11 total, 11 up, 11 in 2024-09-19T05:02:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:54 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:02:55.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:54 smithi064 ceph-mon[30960]: osdmap e65: 11 total, 11 up, 11 in 2024-09-19T05:02:55.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:55 smithi103 ceph-mon[34446]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:02:55.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:55 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/2911798553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]: dispatch 2024-09-19T05:02:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:55 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]: dispatch 2024-09-19T05:02:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:55 smithi103 ceph-mon[34446]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]': finished 2024-09-19T05:02:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:55 smithi103 ceph-mon[34446]: osdmap e66: 12 total, 11 up, 12 in 2024-09-19T05:02:55.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:55 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:56.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:55 smithi174 ceph-mon[34488]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:02:56.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:55 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/2911798553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]: dispatch 2024-09-19T05:02:56.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:55 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]: dispatch 2024-09-19T05:02:56.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:55 smithi174 ceph-mon[34488]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]': finished 2024-09-19T05:02:56.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:55 smithi174 ceph-mon[34488]: osdmap e66: 12 total, 11 up, 12 in 2024-09-19T05:02:56.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:55 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:56.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:55 smithi064 ceph-mon[30960]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:02:56.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:55 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/2911798553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]: dispatch 2024-09-19T05:02:56.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:55 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]: dispatch 2024-09-19T05:02:56.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:55 smithi064 ceph-mon[30960]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3169c4e9-d118-46eb-9b6d-b37e181ffa88"}]': finished 2024-09-19T05:02:56.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:55 smithi064 ceph-mon[30960]: osdmap e66: 12 total, 11 up, 12 in 2024-09-19T05:02:56.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:55 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:57.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:56 smithi174 ceph-mon[34488]: from='client.? 172.21.15.174:0/963325588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:57.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:56 smithi174 ceph-mon[34488]: osdmap e67: 12 total, 11 up, 12 in 2024-09-19T05:02:57.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:57.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:56 smithi064 ceph-mon[30960]: from='client.? 172.21.15.174:0/963325588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:57.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:56 smithi064 ceph-mon[30960]: osdmap e67: 12 total, 11 up, 12 in 2024-09-19T05:02:57.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:57.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:56 smithi103 ceph-mon[34446]: from='client.? 172.21.15.174:0/963325588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:02:57.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:56 smithi103 ceph-mon[34446]: osdmap e67: 12 total, 11 up, 12 in 2024-09-19T05:02:57.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:58.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:57 smithi174 ceph-mon[34488]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:02:58.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:57 smithi174 ceph-mon[34488]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T05:02:58.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:57 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:58.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:57 smithi064 ceph-mon[30960]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:02:58.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:57 smithi064 ceph-mon[30960]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T05:02:58.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:57 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:02:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:57 smithi103 ceph-mon[34446]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:02:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:57 smithi103 ceph-mon[34446]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T05:02:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:57 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:00.021 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:02:59 smithi103 ceph-mon[34446]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:02:59 smithi174 ceph-mon[34488]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:00.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:02:59 smithi064 ceph-mon[30960]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:02.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:01 smithi174 ceph-mon[34488]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T05:03:02.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:01 smithi064 ceph-mon[30960]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T05:03:02.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:01 smithi103 ceph-mon[34446]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-19T05:03:04.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:03 smithi174 ceph-mon[34488]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-19T05:03:04.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:03 smithi064 ceph-mon[30960]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-19T05:03:04.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:03 smithi103 ceph-mon[34446]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-19T05:03:05.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:03:05.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:05.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:04 smithi174 ceph-mon[34488]: Deploying daemon osd.11 on smithi174 2024-09-19T05:03:05.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:03:05.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:05.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:04 smithi064 ceph-mon[30960]: Deploying daemon osd.11 on smithi174 2024-09-19T05:03:05.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:03:05.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:05.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:04 smithi103 ceph-mon[34446]: Deploying daemon osd.11 on smithi174 2024-09-19T05:03:05.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:05 smithi174 ceph-mon[34488]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:03:06.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:05 smithi064 ceph-mon[30960]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:03:06.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:05 smithi103 ceph-mon[34446]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:03:08.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:07 smithi174 ceph-mon[34488]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:03:08.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:07 smithi064 ceph-mon[30960]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:03:08.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:07 smithi103 ceph-mon[34446]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:03:10.073 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:09 smithi174 ceph-mon[34488]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:03:10.073 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:10.073 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:10.073 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:09 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:09 smithi064 ceph-mon[30960]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:03:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:10.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:10.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:09 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:10.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:09 smithi103 ceph-mon[34446]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:03:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:09 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:12.060 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:11 smithi174 ceph-mon[34488]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:03:12.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:11 smithi064 ceph-mon[30960]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:03:12.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:11 smithi103 ceph-mon[34446]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:03:13.086 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.084+0000 7ff2fcff9640 1 -- 172.21.15.174:0/3749438502 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff300062630 con 0x7ff2e80a3270 2024-09-19T05:03:13.086 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-09-19T05:03:13.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff2e80a3270 msgr2=0x7ff2e80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff2e80a3270 0x7ff2e80a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3001993a0 tx=0x7ff2f000b390 comp rx=0 tx=0).stop 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 msgr2=0x7ff300197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 0x7ff300197e80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400d950 tx=0x7ff2f400de10 comp rx=0 tx=0).stop 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 shutdown_connections 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff2e80a3270 0x7ff2e80a5730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff300107900 0x7ff30019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff300104fd0 0x7ff3001983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 --2- 172.21.15.174:0/3749438502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3001026a0 0x7ff300197e80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 >> 172.21.15.174:0/3749438502 conn(0x7ff3000fc2a0 msgr2=0x7ff300103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 shutdown_connections 2024-09-19T05:03:13.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:13.087+0000 7ff305b8e640 1 -- 172.21.15.174:0/3749438502 wait complete. 2024-09-19T05:03:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:13 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:13.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:13.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:13.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:13.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:13 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.694 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T05:03:13.696 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T05:03:13.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:03:13.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:13 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:13.967 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/573751130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 msgr2=0x7f9e08100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/573751130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e08100890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009900 tx=0x7f9df802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/573751130 shutdown_connections 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/573751130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e08101770 0x7f9e08101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/573751130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e08100dd0 0x7f9e08101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/573751130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e08100890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/573751130 >> 172.21.15.64:0/573751130 conn(0x7f9e080fc2a0 msgr2=0x7f9e080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/573751130 shutdown_connections 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.413+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/573751130 wait complete. 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 Processor -- start 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 -- start start 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e0819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e08100dd0 0x7f9e0819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e08101770 0x7f9e081a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0810f910 con 0x7f9e08100490 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0810f790 con 0x7f9e08101770 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e0d2c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0810fa90 con 0x7f9e08100dd0 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.414+0000 7f9e06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e0819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e0819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46120/0 (socket says 172.21.15.64:46120) 2024-09-19T05:03:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 -- 172.21.15.64:0/1953742964 learned_addr learned my addr 172.21.15.64:0/1953742964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e077fe640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e08101770 0x7f9e081a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e067fc640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e08100dd0 0x7f9e0819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 -- 172.21.15.64:0/1953742964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e08100dd0 msgr2=0x7f9e0819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e08100dd0 0x7f9e0819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 -- 172.21.15.64:0/1953742964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e08101770 msgr2=0x7f9e081a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e08101770 0x7f9e081a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 -- 172.21.15.64:0/1953742964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e081a1c30 con 0x7f9e08100490 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e06ffd640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e0819a000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9df8004770 tx=0x7f9df80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df803f030 con 0x7f9e08100490 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.415+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e081a1ec0 con 0x7f9e08100490 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.416+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081a2310 con 0x7f9e08100490 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.416+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df8003cb0 con 0x7f9e08100490 2024-09-19T05:03:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.416+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8049060 con 0x7f9e08100490 2024-09-19T05:03:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.417+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df4005290 con 0x7f9e08100490 2024-09-19T05:03:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.419+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9df8042030 con 0x7f9e08100490 2024-09-19T05:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.419+0000 7f9deffff640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9df00a3270 0x7f9df00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.420+0000 7f9e067fc640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9df00a3270 0x7f9df00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.420+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6809+0+0 (secure 0 0 0) 0x7f9df80efa20 con 0x7f9e08100490 2024-09-19T05:03:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.420+0000 7f9e067fc640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9df00a3270 0x7f9df00a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e0819b520 tx=0x7f9dfc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.423+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9df80bcd20 con 0x7f9e08100490 2024-09-19T05:03:14.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.600+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9df4005850 con 0x7f9e08100490 2024-09-19T05:03:14.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.600+0000 7f9deffff640 1 -- 172.21.15.64:0/1953742964 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f9df80bc620 con 0x7f9e08100490 2024-09-19T05:03:14.602 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.602+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9df00a3270 msgr2=0x7f9df00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.602+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9df00a3270 0x7f9df00a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e0819b520 tx=0x7f9dfc00a2d0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.602+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 msgr2=0x7f9e0819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.602+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e0819a000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9df8004770 tx=0x7f9df80042f0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 shutdown_connections 2024-09-19T05:03:14.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9df00a3270 0x7f9df00a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e08101770 0x7f9e081a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e08100dd0 0x7f9e0819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 --2- 172.21.15.64:0/1953742964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e08100490 0x7f9e0819a000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 >> 172.21.15.64:0/1953742964 conn(0x7f9e080fc2a0 msgr2=0x7f9e080fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 shutdown_connections 2024-09-19T05:03:14.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:14.603+0000 7f9e0d2c0640 1 -- 172.21.15.64:0/1953742964 wait complete. 2024-09-19T05:03:15.530 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726722173,"num_in_osds":12,"osd_in_since":1726722175,"num_remapped_pgs":0} 2024-09-19T05:03:15.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:15 smithi064 ceph-mon[30960]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:15.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:15 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1953742964' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:15.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:15 smithi103 ceph-mon[34446]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:15.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:15 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1953742964' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:15.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:15 smithi174 ceph-mon[34488]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:15.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:15 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1953742964' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:16.278 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:03:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:03:15.958+0000 7fb610550640 -1 osd.11 0 log_to_monitors true 2024-09-19T05:03:16.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:03:16.562 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:16 smithi064 ceph-mon[30960]: from='osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:03:16.562 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:16 smithi064 ceph-mon[30960]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:03:16.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:16 smithi174 ceph-mon[34488]: from='osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:03:16.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:16 smithi174 ceph-mon[34488]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:03:16.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:16 smithi103 ceph-mon[34446]: from='osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:03:16.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:16 smithi103 ceph-mon[34446]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:03:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.362+0000 7f1f88bcf640 1 -- 172.21.15.64:0/291770670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 msgr2=0x7f1f84102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.362+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/291770670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f84102190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009900 tx=0x7f1f7802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 -- 172.21.15.64:0/291770670 shutdown_connections 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/291770670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f841026d0 0x7f1f84108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/291770670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f84102190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/291770670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f84100b10 0x7f1f84100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 -- 172.21.15.64:0/291770670 >> 172.21.15.64:0/291770670 conn(0x7f1f840fc2a0 msgr2=0x7f1f840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 -- 172.21.15.64:0/291770670 shutdown_connections 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 -- 172.21.15.64:0/291770670 wait complete. 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 Processor -- start 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.363+0000 7f1f88bcf640 1 -- start start 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f88bcf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f84100b10 0x7f1f8406b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f88bcf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f8419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f88bcf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f841026d0 0x7f1f8419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f88bcf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410c6a0 con 0x7f1f84101d10 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f88bcf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410c520 con 0x7f1f841026d0 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f88bcf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410c820 con 0x7f1f84100b10 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f8419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f8419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46138/0 (socket says 172.21.15.64:46138) 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 -- 172.21.15.64:0/766233761 learned_addr learned my addr 172.21.15.64:0/766233761 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f8259c640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f84100b10 0x7f1f8406b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f82d9d640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f841026d0 0x7f1f8419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 -- 172.21.15.64:0/766233761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f84100b10 msgr2=0x7f1f8406b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f84100b10 0x7f1f8406b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 -- 172.21.15.64:0/766233761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f841026d0 msgr2=0x7f1f8419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f841026d0 0x7f1f8419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 -- 172.21.15.64:0/766233761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8419fa50 con 0x7f1f84101d10 2024-09-19T05:03:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.364+0000 7f1f81d9b640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f8419cb40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005940 tx=0x7f1f78004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.365+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7803f030 con 0x7f1f84101d10 2024-09-19T05:03:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.365+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8419fce0 con 0x7f1f84101d10 2024-09-19T05:03:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.365+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f78003cb0 con 0x7f1f84101d10 2024-09-19T05:03:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.365+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78035070 con 0x7f1f84101d10 2024-09-19T05:03:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.365+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f841a01c0 con 0x7f1f84101d10 2024-09-19T05:03:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.366+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f84107ee0 con 0x7f1f84101d10 2024-09-19T05:03:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.368+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1f78042030 con 0x7f1f84101d10 2024-09-19T05:03:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.368+0000 7f1f737fe640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1f6c0a3240 0x7f1f6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.368+0000 7f1f8259c640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1f6c0a3240 0x7f1f6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.369+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6831+0+0 (secure 0 0 0) 0x7f1f780f0010 con 0x7f1f84101d10 2024-09-19T05:03:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.369+0000 7f1f8259c640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1f6c0a3240 0x7f1f6c0a5700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f8406b870 tx=0x7f1f74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:17.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.372+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f780bd300 con 0x7f1f84101d10 2024-09-19T05:03:17.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.552+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1f8402da30 con 0x7f1f84101d10 2024-09-19T05:03:17.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.552+0000 7f1f737fe640 1 -- 172.21.15.64:0/766233761 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f1f780bcc00 con 0x7f1f84101d10 2024-09-19T05:03:17.554 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:17.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:17.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:03:17.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:03:17.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: Detected new or changed devices on smithi174 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:17.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:17.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:17 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1f6c0a3240 msgr2=0x7f1f6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:17.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1f6c0a3240 0x7f1f6c0a5700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f8406b870 tx=0x7f1f74005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:17.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 msgr2=0x7f1f8419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f8419cb40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005940 tx=0x7f1f78004430 comp rx=0 tx=0).stop 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 shutdown_connections 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1f6c0a3240 0x7f1f6c0a5700 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f841026d0 0x7f1f8419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f84101d10 0x7f1f8419cb40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 --2- 172.21.15.64:0/766233761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f84100b10 0x7f1f8406b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 >> 172.21.15.64:0/766233761 conn(0x7f1f840fc2a0 msgr2=0x7f1f841050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 shutdown_connections 2024-09-19T05:03:17.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:17.555+0000 7f1f88bcf640 1 -- 172.21.15.64:0/766233761 wait complete. 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: Detected new or changed devices on smithi174 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:17.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:17 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.777 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:03:17.397+0000 7fb60bcb0640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T05:03:17.777 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:03:17.412+0000 7fb6082ea640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: Detected new or changed devices on smithi174 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:03:17.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T05:03:17.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:17.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:17.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:17.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:17 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:18.553 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726722173,"num_in_osds":12,"osd_in_since":1726722175,"num_remapped_pgs":0} 2024-09-19T05:03:18.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:18 smithi103 ceph-mon[34446]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:03:18.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:18 smithi103 ceph-mon[34446]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T05:03:18.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:18 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:18.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:18 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:18.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:18 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/766233761' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:18.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:18 smithi174 ceph-mon[34488]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:03:18.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:18 smithi174 ceph-mon[34488]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T05:03:18.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:18 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:18.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:18 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:18.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:18 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/766233761' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:18.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:18 smithi064 ceph-mon[30960]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T05:03:18.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:18 smithi064 ceph-mon[30960]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T05:03:18.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:18.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:18 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:18.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:18 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/766233761' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:19.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:03:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: purged_snaps scrub starts 2024-09-19T05:03:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: purged_snaps scrub ok 2024-09-19T05:03:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:19.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:19.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] boot 2024-09-19T05:03:19.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: osdmap e71: 12 total, 12 up, 12 in 2024-09-19T05:03:19.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: purged_snaps scrub starts 2024-09-19T05:03:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: purged_snaps scrub ok 2024-09-19T05:03:19.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:19.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:19.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] boot 2024-09-19T05:03:19.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: osdmap e71: 12 total, 12 up, 12 in 2024-09-19T05:03:19.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:19.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: purged_snaps scrub starts 2024-09-19T05:03:19.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: purged_snaps scrub ok 2024-09-19T05:03:19.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:03:19.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:19.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: osd.11 [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] boot 2024-09-19T05:03:19.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: osdmap e71: 12 total, 12 up, 12 in 2024-09-19T05:03:19.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:03:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 -- 172.21.15.64:0/1683925912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 msgr2=0x7f2374103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 --2- 172.21.15.64:0/1683925912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f2374103280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2368009900 tx=0x7f236802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 -- 172.21.15.64:0/1683925912 shutdown_connections 2024-09-19T05:03:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 --2- 172.21.15.64:0/1683925912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23741037c0 0x7f237410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 --2- 172.21.15.64:0/1683925912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f2374103280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 --2- 172.21.15.64:0/1683925912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101c00 0x7f2374102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 -- 172.21.15.64:0/1683925912 >> 172.21.15.64:0/1683925912 conn(0x7f23740fd390 msgr2=0x7f23740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 -- 172.21.15.64:0/1683925912 shutdown_connections 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.211+0000 7f237b901640 1 -- 172.21.15.64:0/1683925912 wait complete. 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 Processor -- start 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 -- start start 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2374101c00 0x7f2374197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f23741982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23741037c0 0x7f237419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f237410d710 con 0x7f2374102e00 2024-09-19T05:03:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f237410d590 con 0x7f2374101c00 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237b901640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f237410d890 con 0x7f23741037c0 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f2378e9c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f23741982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f2378e9c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f23741982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46164/0 (socket says 172.21.15.64:46164) 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f2378e9c640 1 -- 172.21.15.64:0/48338146 learned_addr learned my addr 172.21.15.64:0/48338146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.212+0000 7f237969d640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2374101c00 0x7f2374197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2379e9e640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23741037c0 0x7f237419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2378e9c640 1 -- 172.21.15.64:0/48338146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23741037c0 msgr2=0x7f237419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2378e9c640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23741037c0 0x7f237419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2378e9c640 1 -- 172.21.15.64:0/48338146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2374101c00 msgr2=0x7f2374197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2378e9c640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2374101c00 0x7f2374197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2378e9c640 1 -- 172.21.15.64:0/48338146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237419f9d0 con 0x7f2374102e00 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f2378e9c640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f23741982f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2368004770 tx=0x7f23680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236803f030 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2368003cb0 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237419fc60 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368035070 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.213+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23741a0190 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.214+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23741090a0 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.216+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2368042030 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.216+0000 7f23667fc640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23600a32c0 0x7f23600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.216+0000 7f237969d640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23600a32c0 0x7f23600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.217+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f23680eff00 con 0x7f2374102e00 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.217+0000 7f237969d640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23600a32c0 0x7f23600a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2374199290 tx=0x7f237000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.219+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23680bca70 con 0x7f2374102e00 2024-09-19T05:03:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.396+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f237402d990 con 0x7f2374102e00 2024-09-19T05:03:20.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.396+0000 7f23667fc640 1 -- 172.21.15.64:0/48338146 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f23680c0d70 con 0x7f2374102e00 2024-09-19T05:03:20.398 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23600a32c0 msgr2=0x7f23600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23600a32c0 0x7f23600a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2374199290 tx=0x7f237000b390 comp rx=0 tx=0).stop 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 msgr2=0x7f23741982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f23741982f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2368004770 tx=0x7f23680042f0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 shutdown_connections 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23600a32c0 0x7f23600a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23741037c0 0x7f237419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2374102e00 0x7f23741982f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 --2- 172.21.15.64:0/48338146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2374101c00 0x7f2374197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.398+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 >> 172.21.15.64:0/48338146 conn(0x7f23740fd390 msgr2=0x7f2374106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.399+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 shutdown_connections 2024-09-19T05:03:20.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:20.399+0000 7f237b901640 1 -- 172.21.15.64:0/48338146 wait complete. 2024-09-19T05:03:20.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:20 smithi064 ceph-mon[30960]: osdmap e72: 12 total, 12 up, 12 in 2024-09-19T05:03:20.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:20 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/48338146' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:20.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:20 smithi103 ceph-mon[34446]: osdmap e72: 12 total, 12 up, 12 in 2024-09-19T05:03:20.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:20 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/48338146' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:20.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:20 smithi174 ceph-mon[34488]: osdmap e72: 12 total, 12 up, 12 in 2024-09-19T05:03:20.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:20 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/48338146' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:03:21.046 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726722198,"num_in_osds":12,"osd_in_since":1726722175,"num_remapped_pgs":0} 2024-09-19T05:03:21.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:03:21.298 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:21.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:21 smithi103 ceph-mon[34446]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:21.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:21 smithi064 ceph-mon[30960]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 -- 172.21.15.64:0/617113236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb568102d30 msgr2=0x7fb568103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 --2- 172.21.15.64:0/617113236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb568102d30 0x7fb568103110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb558009930 tx=0x7fb55802a420 comp rx=0 tx=0).stop 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 -- 172.21.15.64:0/617113236 shutdown_connections 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 --2- 172.21.15.64:0/617113236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5681082a0 0x7fb56810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 --2- 172.21.15.64:0/617113236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5681036e0 0x7fb568107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 --2- 172.21.15.64:0/617113236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb568102d30 0x7fb568103110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 -- 172.21.15.64:0/617113236 >> 172.21.15.64:0/617113236 conn(0x7fb5680fc7d0 msgr2=0x7fb5680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 -- 172.21.15.64:0/617113236 shutdown_connections 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.735+0000 7fb56dc35640 1 -- 172.21.15.64:0/617113236 wait complete. 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 Processor -- start 2024-09-19T05:03:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 -- start start 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb568102d30 0x7fb568073a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5681036e0 0x7fb56806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 0x7fb56806d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb568111fb0 con 0x7fb5681082a0 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb568111e30 con 0x7fb568102d30 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb56dc35640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb568112130 con 0x7fb5681036e0 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb567fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 0x7fb56806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb5677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb568102d30 0x7fb568073a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb567fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 0x7fb56806d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46176/0 (socket says 172.21.15.64:46176) 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb567fff640 1 -- 172.21.15.64:0/4166386863 learned_addr learned my addr 172.21.15.64:0/4166386863 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.736+0000 7fb566ffd640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5681036e0 0x7fb56806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb567fff640 1 -- 172.21.15.64:0/4166386863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5681036e0 msgr2=0x7fb56806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb567fff640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5681036e0 0x7fb56806caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb567fff640 1 -- 172.21.15.64:0/4166386863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb568102d30 msgr2=0x7fb568073a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb567fff640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb568102d30 0x7fb568073a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb567fff640 1 -- 172.21.15.64:0/4166386863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56806d670 con 0x7fb5681082a0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb567fff640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 0x7fb56806d030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb56000cad0 tx=0x7fb56000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56000ae20 con 0x7fb5681082a0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb560005020 con 0x7fb5681082a0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb568071590 con 0x7fb5681082a0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56000f6b0 con 0x7fb5681082a0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.737+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb568071a80 con 0x7fb5681082a0 2024-09-19T05:03:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.739+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56806d900 con 0x7fb5681082a0 2024-09-19T05:03:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.740+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb5600051d0 con 0x7fb5681082a0 2024-09-19T05:03:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.740+0000 7fb564ff9640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5500a3240 0x7fb5500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.741+0000 7fb5677fe640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5500a3240 0x7fb5500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.741+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb5600cf830 con 0x7fb5681082a0 2024-09-19T05:03:21.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.741+0000 7fb5677fe640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5500a3240 0x7fb5500a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb558031440 tx=0x7fb558005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.744+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb560014030 con 0x7fb5681082a0 2024-09-19T05:03:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:21 smithi174 ceph-mon[34488]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:21.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.922+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb56806e020 con 0x7fb5681082a0 2024-09-19T05:03:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.923+0000 7fb564ff9640 1 -- 172.21.15.64:0/4166386863 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18482 (secure 0 0 0) 0x7fb56009c300 con 0x7fb5681082a0 2024-09-19T05:03:21.925 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:21.925 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":72,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:03:19.400057+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5500a3240 msgr2=0x7fb5500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5500a3240 0x7fb5500a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb558031440 tx=0x7fb558005a20 comp rx=0 tx=0).stop 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 msgr2=0x7fb56806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 0x7fb56806d030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb56000cad0 tx=0x7fb56000cf90 comp rx=0 tx=0).stop 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 shutdown_connections 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5500a3240 0x7fb5500a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5681082a0 0x7fb56806d030 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5681036e0 0x7fb56806caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 --2- 172.21.15.64:0/4166386863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb568102d30 0x7fb568073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 >> 172.21.15.64:0/4166386863 conn(0x7fb5680fc7d0 msgr2=0x7fb568105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 shutdown_connections 2024-09-19T05:03:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:21.925+0000 7fb56dc35640 1 -- 172.21.15.64:0/4166386863 wait complete. 2024-09-19T05:03:22.563 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:59:34.802774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:03:22.563 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T05:03:22.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:22 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4166386863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:22.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:22 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4166386863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:22.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:22 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4166386863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 -- 172.21.15.64:0/2303437837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c100a80 msgr2=0x7fd36c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 --2- 172.21.15.64:0/2303437837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c100a80 0x7fd36c100ee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009930 tx=0x7fd35c02a420 comp rx=0 tx=0).stop 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 -- 172.21.15.64:0/2303437837 shutdown_connections 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 --2- 172.21.15.64:0/2303437837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd36c101420 0x7fd36c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 --2- 172.21.15.64:0/2303437837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c100a80 0x7fd36c100ee0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 --2- 172.21.15.64:0/2303437837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd36c106a80 0x7fd36c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 -- 172.21.15.64:0/2303437837 >> 172.21.15.64:0/2303437837 conn(0x7fd36c0fc7a0 msgr2=0x7fd36c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 -- 172.21.15.64:0/2303437837 shutdown_connections 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.244+0000 7fd372d1c640 1 -- 172.21.15.64:0/2303437837 wait complete. 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 Processor -- start 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 -- start start 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd36c100a80 0x7fd36c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd36c101420 0x7fd36c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 0x7fd36c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c110d30 con 0x7fd36c106a80 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c110bb0 con 0x7fd36c101420 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd372d1c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c110eb0 con 0x7fd36c100a80 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd3712b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 0x7fd36c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd3712b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 0x7fd36c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46194/0 (socket says 172.21.15.64:46194) 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd3712b9640 1 -- 172.21.15.64:0/3540203242 learned_addr learned my addr 172.21.15.64:0/3540203242 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.245+0000 7fd363fff640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd36c101420 0x7fd36c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd370ab8640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd36c100a80 0x7fd36c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd3712b9640 1 -- 172.21.15.64:0/3540203242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd36c100a80 msgr2=0x7fd36c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd3712b9640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd36c100a80 0x7fd36c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd3712b9640 1 -- 172.21.15.64:0/3540203242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd36c101420 msgr2=0x7fd36c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd3712b9640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd36c101420 0x7fd36c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd3712b9640 1 -- 172.21.15.64:0/3540203242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c19d310 con 0x7fd36c106a80 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd3712b9640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 0x7fd36c19cc30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd36400eaa0 tx=0x7fd36400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36400cdb0 con 0x7fd36c106a80 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c19d600 con 0x7fd36c106a80 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd364005020 con 0x7fd36c106a80 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3640116d0 con 0x7fd36c106a80 2024-09-19T05:03:23.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.246+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1a4ee0 con 0x7fd36c106a80 2024-09-19T05:03:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.247+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36c100ee0 con 0x7fd36c106a80 2024-09-19T05:03:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.248+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd3640051d0 con 0x7fd36c106a80 2024-09-19T05:03:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.249+0000 7fd361ffb640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd3540a3240 0x7fd3540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.249+0000 7fd370ab8640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd3540a3240 0x7fd3540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.249+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd3640cf670 con 0x7fd36c106a80 2024-09-19T05:03:23.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.250+0000 7fd370ab8640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd3540a3240 0x7fd3540a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd368005fd0 tx=0x7fd368005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:23.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.253+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd364016030 con 0x7fd36c106a80 2024-09-19T05:03:23.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.429+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd36c109a90 con 0x7fd36c106a80 2024-09-19T05:03:23.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.430+0000 7fd361ffb640 1 -- 172.21.15.64:0/3540203242 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fd36409c830 con 0x7fd36c106a80 2024-09-19T05:03:23.432 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-19T05:03:23.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd3540a3240 msgr2=0x7fd3540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd3540a3240 0x7fd3540a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd368005fd0 tx=0x7fd368005850 comp rx=0 tx=0).stop 2024-09-19T05:03:23.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 msgr2=0x7fd36c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 0x7fd36c19cc30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd36400eaa0 tx=0x7fd36400ef60 comp rx=0 tx=0).stop 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 shutdown_connections 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd3540a3240 0x7fd3540a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd36c106a80 0x7fd36c19cc30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd36c101420 0x7fd36c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 --2- 172.21.15.64:0/3540203242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd36c100a80 0x7fd36c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 >> 172.21.15.64:0/3540203242 conn(0x7fd36c0fc7a0 msgr2=0x7fd36c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 shutdown_connections 2024-09-19T05:03:23.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:23.432+0000 7fd372d1c640 1 -- 172.21.15.64:0/3540203242 wait complete. 2024-09-19T05:03:23.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:23 smithi064 ceph-mon[30960]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:23.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:23 smithi103 ceph-mon[34446]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:23 smithi174 ceph-mon[34488]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.a on smithi064 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.d on smithi064 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.g on smithi064 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.j on smithi064 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.b on smithi103 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.e on smithi103 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.h on smithi103 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.k on smithi103 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-09-19T05:03:24.028 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-09-19T05:03:24.029 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-09-19T05:03:24.029 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-09-19T05:03:24.285 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:03:24.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:24 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3540203242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:03:24.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:24 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3540203242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:03:24.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:24 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3540203242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:03:25.053 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 -- 172.21.15.174:0/3057015664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8100ad0 msgr2=0x7f6ea8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.053 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3057015664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8100ad0 0x7f6ea8100ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009900 tx=0x7f6e9802a440 comp rx=0 tx=0).stop 2024-09-19T05:03:25.053 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 -- 172.21.15.174:0/3057015664 shutdown_connections 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3057015664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8102690 0x7f6ea8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3057015664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 0x7f6ea8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3057015664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8100ad0 0x7f6ea8100ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 -- 172.21.15.174:0/3057015664 >> 172.21.15.174:0/3057015664 conn(0x7f6ea80fc280 msgr2=0x7f6ea80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 -- 172.21.15.174:0/3057015664 shutdown_connections 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.038+0000 7f6eae04e640 1 -- 172.21.15.174:0/3057015664 wait complete. 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 Processor -- start 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 -- start start 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8100ad0 0x7f6ea806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 0x7f6ea819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 0x7f6ea819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810c770 con 0x7f6ea8101cd0 2024-09-19T05:03:25.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810c5f0 con 0x7f6ea8100ad0 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6eae04e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810c8f0 con 0x7f6ea8102690 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 0x7f6ea819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 0x7f6ea819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38188/0 (socket says 172.21.15.174:38188) 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6ea7fff640 1 -- 172.21.15.174:0/3141223522 learned_addr learned my addr 172.21.15.174:0/3141223522 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6ea77fe640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8100ad0 0x7f6ea806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6ea7fff640 1 -- 172.21.15.174:0/3141223522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8100ad0 msgr2=0x7f6ea806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6e9effd640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 0x7f6ea819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.039+0000 7f6ea7fff640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8100ad0 0x7f6ea806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea7fff640 1 -- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 msgr2=0x7f6ea819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea7fff640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 0x7f6ea819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea7fff640 1 -- 172.21.15.174:0/3141223522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea819fc60 con 0x7f6ea8102690 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6e9effd640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 0x7f6ea819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:03:25.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea7fff640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 0x7f6ea819f350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000b990 tx=0x7f6ea000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0019070 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea000cba0 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea819fef0 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00075d0 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.040+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a0420 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.041+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea8107fc0 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.042+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6ea001c070 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.043+0000 7f6ea57fa640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6e940a3240 0x7f6e940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.043+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6ea00cef30 con 0x7f6ea8102690 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.043+0000 7f6ea77fe640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6e940a3240 0x7f6e940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.044+0000 7f6ea77fe640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6e940a3240 0x7f6e940a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6e98006010 tx=0x7f6e98000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.046+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ea009c0f0 con 0x7f6ea8102690 2024-09-19T05:03:25.228 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.226+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7f6ea80627f0 con 0x7f6e940a3240 2024-09-19T05:03:25.235 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.234+0000 7f6ea57fa640 1 -- 172.21.15.174:0/3141223522 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6ea80627f0 con 0x7f6e940a3240 2024-09-19T05:03:25.235 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6e940a3240 msgr2=0x7f6e940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6e940a3240 0x7f6e940a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6e98006010 tx=0x7f6e98000bc0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 msgr2=0x7f6ea819f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 0x7f6ea819f350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000b990 tx=0x7f6ea000be50 comp rx=0 tx=0).stop 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 shutdown_connections 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6e940a3240 0x7f6e940a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102690 0x7f6ea819f350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8101cd0 0x7f6ea819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 --2- 172.21.15.174:0/3141223522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8100ad0 0x7f6ea806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 >> 172.21.15.174:0/3141223522 conn(0x7f6ea80fc280 msgr2=0x7f6ea8105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:25.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 shutdown_connections 2024-09-19T05:03:25.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T05:03:25.236+0000 7f6eae04e640 1 -- 172.21.15.174:0/3141223522 wait complete. 2024-09-19T05:03:25.933 DEBUG:teuthology.orchestra.run.smithi064:mds.a> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T05:03:25.936 DEBUG:teuthology.orchestra.run.smithi064:mds.d> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T05:03:25.982 DEBUG:teuthology.orchestra.run.smithi064:mds.g> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T05:03:26.031 DEBUG:teuthology.orchestra.run.smithi064:mds.j> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T05:03:26.077 DEBUG:teuthology.orchestra.run.smithi103:mds.b> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T05:03:26.080 DEBUG:teuthology.orchestra.run.smithi103:mds.e> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T05:03:26.127 DEBUG:teuthology.orchestra.run.smithi103:mds.h> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T05:03:26.171 DEBUG:teuthology.orchestra.run.smithi103:mds.k> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T05:03:26.216 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T05:03:26.219 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T05:03:26.262 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T05:03:26.306 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T05:03:26.348 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T05:03:26.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:26.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:26 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:03:26.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:26.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:26 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:26 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:26.602 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 -- 172.21.15.64:0/160951612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 msgr2=0x7f28c8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 --2- 172.21.15.64:0/160951612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c8107b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009930 tx=0x7f28bc02a420 comp rx=0 tx=0).stop 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 -- 172.21.15.64:0/160951612 shutdown_connections 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 --2- 172.21.15.64:0/160951612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28c8108290 0x7f28c810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 --2- 172.21.15.64:0/160951612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c8107b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 --2- 172.21.15.64:0/160951612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c8102d90 0x7f28c8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 -- 172.21.15.64:0/160951612 >> 172.21.15.64:0/160951612 conn(0x7f28c80fc7d0 msgr2=0x7f28c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:27.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 -- 172.21.15.64:0/160951612 shutdown_connections 2024-09-19T05:03:27.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 -- 172.21.15.64:0/160951612 wait complete. 2024-09-19T05:03:27.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 Processor -- start 2024-09-19T05:03:27.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.090+0000 7f28ce502640 1 -- start start 2024-09-19T05:03:27.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28ce502640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28c8102d90 0x7f28c8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:27.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28ce502640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28ce502640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c8108290 0x7f28c819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28ce502640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8111f00 con 0x7f28c8103740 2024-09-19T05:03:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28ce502640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8111d80 con 0x7f28c8102d90 2024-09-19T05:03:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28ce502640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8112080 con 0x7f28c8108290 2024-09-19T05:03:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:27.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36838/0 (socket says 172.21.15.64:36838) 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 -- 172.21.15.64:0/3471924582 learned_addr learned my addr 172.21.15.64:0/3471924582 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c7fff640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28c8102d90 0x7f28c8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 -- 172.21.15.64:0/3471924582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c8108290 msgr2=0x7f28c819cd30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c8108290 0x7f28c819cd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 -- 172.21.15.64:0/3471924582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28c8102d90 msgr2=0x7f28c8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28c8102d90 0x7f28c8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 -- 172.21.15.64:0/3471924582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c819d410 con 0x7f28c8103740 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.091+0000 7f28c77fe640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c81989a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f28bc031440 tx=0x7f28bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.092+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc03f030 con 0x7f28c8103740 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.092+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28bc003aa0 con 0x7f28c8103740 2024-09-19T05:03:27.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.092+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc035070 con 0x7f28c8103740 2024-09-19T05:03:27.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.092+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c819d6a0 con 0x7f28c8103740 2024-09-19T05:03:27.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.092+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c819db30 con 0x7f28c8103740 2024-09-19T05:03:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.093+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c8103170 con 0x7f28c8103740 2024-09-19T05:03:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.095+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f28bc042030 con 0x7f28c8103740 2024-09-19T05:03:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.095+0000 7f28c57fa640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28b00a32c0 0x7f28b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.095+0000 7f28c7fff640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28b00a32c0 0x7f28b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.095+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f28bc0efe60 con 0x7f28c8103740 2024-09-19T05:03:27.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.096+0000 7f28c7fff640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28b00a32c0 0x7f28b00a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28b800abe0 tx=0x7f28b800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:27.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.098+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f28bc049030 con 0x7f28c8103740 2024-09-19T05:03:27.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.310+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c810a2a0 con 0x7f28c8103740 2024-09-19T05:03:27.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:27 smithi064 ceph-mon[30960]: Deploying daemon mds.l on smithi174 2024-09-19T05:03:27.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.311+0000 7f28c57fa640 1 -- 172.21.15.64:0/3471924582 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f28bc0bd020 con 0x7f28c8103740 2024-09-19T05:03:27.314 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"btime":"2024-09-19T04:56:23:797082+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T05:03:27.314 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 1 2024-09-19T05:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.312+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28b00a32c0 msgr2=0x7f28b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.312+0000 7f28ce502640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28b00a32c0 0x7f28b00a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28b800abe0 tx=0x7f28b800a2d0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.312+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 msgr2=0x7f28c81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c81989a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f28bc031440 tx=0x7f28bc004270 comp rx=0 tx=0).stop 2024-09-19T05:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 shutdown_connections 2024-09-19T05:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28b00a32c0 0x7f28b00a5780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c8108290 0x7f28c819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28c8103740 0x7f28c81989a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 --2- 172.21.15.64:0/3471924582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28c8102d90 0x7f28c8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:27.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 >> 172.21.15.64:0/3471924582 conn(0x7f28c80fc7d0 msgr2=0x7f28c8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:27.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 shutdown_connections 2024-09-19T05:03:27.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:27.313+0000 7f28ce502640 1 -- 172.21.15.64:0/3471924582 wait complete. 2024-09-19T05:03:27.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:27 smithi174 ceph-mon[34488]: Deploying daemon mds.l on smithi174 2024-09-19T05:03:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:27 smithi103 ceph-mon[34446]: Deploying daemon mds.l on smithi174 2024-09-19T05:03:27.777 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:27 smithi174 systemd[1]: Starting Ceph mds.l for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:03:28.041 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:27 smithi174 podman[63949]: 2024-09-19 05:03:27.687223101 +0000 UTC m=+0.015866968 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:28.041 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:27 smithi174 podman[63949]: 2024-09-19 05:03:27.804754256 +0000 UTC m=+0.133398112 container create 2ebaa3649d6d5a0f0dd526ba6e792b51c98d442abae5adb94bf7172e46d4e263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3471924582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6832/252917517,v1:172.21.15.174:6833/252917517] up:boot 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: fsmap 1 up:standby 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:28.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:28.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:28 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3471924582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6832/252917517,v1:172.21.15.174:6833/252917517] up:boot 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: fsmap 1 up:standby 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:28.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:28.529 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:28 smithi174 podman[63949]: 2024-09-19 05:03:28.040708609 +0000 UTC m=+0.369352462 container init 2ebaa3649d6d5a0f0dd526ba6e792b51c98d442abae5adb94bf7172e46d4e263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T05:03:28.529 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:28 smithi174 podman[63949]: 2024-09-19 05:03:28.043602149 +0000 UTC m=+0.372246020 container start 2ebaa3649d6d5a0f0dd526ba6e792b51c98d442abae5adb94bf7172e46d4e263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:03:28.529 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-mds[63968]: -- 172.21.15.174:0/3710439779 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a82ff3c20 con 0x561a82fd0800 2024-09-19T05:03:28.529 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l[63964]: starting mds.l at 2024-09-19T05:03:28.529 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:28 smithi174 bash[63949]: 2ebaa3649d6d5a0f0dd526ba6e792b51c98d442abae5adb94bf7172e46d4e263 2024-09-19T05:03:28.529 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:03:28 smithi174 systemd[1]: Started Ceph mds.l for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:28.641 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:28.641 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3471924582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:28.641 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6832/252917517,v1:172.21.15.174:6833/252917517] up:boot 2024-09-19T05:03:28.641 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: fsmap 1 up:standby 2024-09-19T05:03:28.641 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:03:28.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:28.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:28.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:28.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:28 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:29 smithi174 ceph-mon[34488]: Deploying daemon mds.k on smithi103 2024-09-19T05:03:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:29 smithi064 ceph-mon[30960]: Deploying daemon mds.k on smithi103 2024-09-19T05:03:29.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:29 smithi103 ceph-mon[34446]: Deploying daemon mds.k on smithi103 2024-09-19T05:03:30.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:30.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:30 smithi064 ceph-mon[30960]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:30.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:30 smithi103 ceph-mon[34446]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:30.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:30 smithi174 ceph-mon[34488]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:30.673 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:03:30 smithi103 podman[63636]: 2024-09-19 05:03:30.649280606 +0000 UTC m=+0.358734994 container start dd2d8a6ada26d080841763691dce9679eb602a7f74408dcf2575f9b3957e13a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 -- 172.21.15.64:0/3080281892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 msgr2=0x7f8874112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 --2- 172.21.15.64:0/3080281892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f8874112650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8864009900 tx=0x7f886402a480 comp rx=0 tx=0).stop 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 -- 172.21.15.64:0/3080281892 shutdown_connections 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 --2- 172.21.15.64:0/3080281892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 0x7f8874114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 --2- 172.21.15.64:0/3080281892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f8874112650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 --2- 172.21.15.64:0/3080281892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874074590 0x7f8874074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 -- 172.21.15.64:0/3080281892 >> 172.21.15.64:0/3080281892 conn(0x7f88740fd8c0 msgr2=0x7f88740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 -- 172.21.15.64:0/3080281892 shutdown_connections 2024-09-19T05:03:30.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.739+0000 7f887b582640 1 -- 172.21.15.64:0/3080281892 wait complete. 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.740+0000 7f887b582640 1 Processor -- start 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.740+0000 7f887b582640 1 -- start start 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.740+0000 7f887b582640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874074590 0x7f88741a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887b582640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f88741a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887b582640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 0x7f887419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887b582640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8874118470 con 0x7f8874074eb0 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887b582640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88741182f0 con 0x7f8874112b90 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887b582640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88741185f0 con 0x7f8874074590 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f88741a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f88741a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36856/0 (socket says 172.21.15.64:36856) 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 -- 172.21.15.64:0/2982688749 learned_addr learned my addr 172.21.15.64:0/2982688749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8879b1f640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 0x7f887419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:30.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887931e640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874074590 0x7f88741a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 -- 172.21.15.64:0/2982688749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874074590 msgr2=0x7f88741a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874074590 0x7f88741a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 -- 172.21.15.64:0/2982688749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 msgr2=0x7f887419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 0x7f887419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 -- 172.21.15.64:0/2982688749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887419ef90 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8879b1f640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 0x7f887419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f8878b1d640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f88741a4be0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8864006010 tx=0x7f8864004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886403f030 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.741+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887419f220 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.742+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88640035f0 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.742+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864035070 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.742+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88741ab570 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.743+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8874074970 con 0x7f8874074eb0 2024-09-19T05:03:30.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.744+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8864003b10 con 0x7f8874074eb0 2024-09-19T05:03:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.744+0000 7f886a7fc640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f88600a3240 0x7f88600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.745+0000 7f887931e640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f88600a3240 0x7f88600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.745+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f88640f00a0 con 0x7f8874074eb0 2024-09-19T05:03:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.745+0000 7f887931e640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f88600a3240 0x7f88600a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f887419f9a0 tx=0x7f8870008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:30.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.749+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f88640bd260 con 0x7f8874074eb0 2024-09-19T05:03:30.968 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:03:30 smithi103 ceph-mds[63655]: -- 172.21.15.103:0/3313096971 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55669b61b860 con 0x55669b5f9400 2024-09-19T05:03:30.969 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:03:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k[63651]: starting mds.k at 2024-09-19T05:03:30.969 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:03:30 smithi103 bash[63636]: dd2d8a6ada26d080841763691dce9679eb602a7f74408dcf2575f9b3957e13a9 2024-09-19T05:03:30.969 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:03:30 smithi103 systemd[1]: Started Ceph mds.k for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.979+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8874105030 con 0x7f8874074eb0 2024-09-19T05:03:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.979+0000 7f886a7fc640 1 -- 172.21.15.64:0/2982688749 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f88640bcb60 con 0x7f8874074eb0 2024-09-19T05:03:30.982 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:30.982 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"btime":"2024-09-19T05:03:28:072250+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:03:30.982 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-09-19T05:03:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.981+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f88600a3240 msgr2=0x7f88600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.981+0000 7f887b582640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f88600a3240 0x7f88600a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f887419f9a0 tx=0x7f8870008040 comp rx=0 tx=0).stop 2024-09-19T05:03:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.981+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 msgr2=0x7f88741a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.981+0000 7f887b582640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f88741a4be0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8864006010 tx=0x7f8864004270 comp rx=0 tx=0).stop 2024-09-19T05:03:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.981+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 shutdown_connections 2024-09-19T05:03:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f88600a3240 0x7f88600a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874112b90 0x7f887419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874074eb0 0x7f88741a4be0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 --2- 172.21.15.64:0/2982688749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874074590 0x7f88741a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 >> 172.21.15.64:0/2982688749 conn(0x7f88740fd8c0 msgr2=0x7f88740ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 shutdown_connections 2024-09-19T05:03:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:30.982+0000 7f887b582640 1 -- 172.21.15.64:0/2982688749 wait complete. 2024-09-19T05:03:32.063 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:32.063 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.063 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.063 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.063 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: Deploying daemon mds.j on smithi064 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2982688749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6832/550094745,v1:172.21.15.103:6833/550094745] up:boot 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: fsmap 2 up:standby 2024-09-19T05:03:32.064 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:31 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:03:32.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:32.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: Deploying daemon mds.j on smithi064 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2982688749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6832/550094745,v1:172.21.15.103:6833/550094745] up:boot 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: fsmap 2 up:standby 2024-09-19T05:03:32.076 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:31 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:03:32.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: Deploying daemon mds.j on smithi064 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2982688749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6832/550094745,v1:172.21.15.103:6833/550094745] up:boot 2024-09-19T05:03:32.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: fsmap 2 up:standby 2024-09-19T05:03:32.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:31 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:03:33.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:33.965 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:34.087 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:33 smithi064 ceph-mon[30960]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:34.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:33 smithi103 ceph-mon[34446]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:34.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:33 smithi174 ceph-mon[34488]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 -- 172.21.15.64:0/2616030135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac81000d0 msgr2=0x7f0ac8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/2616030135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac81000d0 0x7f0ac8100530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009900 tx=0x7f0abc02a410 comp rx=0 tx=0).stop 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 -- 172.21.15.64:0/2616030135 shutdown_connections 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/2616030135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac8100a70 0x7f0ac810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/2616030135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac81000d0 0x7f0ac8100530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/2616030135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac8106ba0 0x7f0ac8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 -- 172.21.15.64:0/2616030135 >> 172.21.15.64:0/2616030135 conn(0x7f0ac8077210 msgr2=0x7f0ac80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 -- 172.21.15.64:0/2616030135 shutdown_connections 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.417+0000 7f0acd1e9640 1 -- 172.21.15.64:0/2616030135 wait complete. 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 Processor -- start 2024-09-19T05:03:34.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 -- start start 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac81000d0 0x7f0ac819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 0x7f0ac819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac8106ba0 0x7f0ac819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8111f50 con 0x7f0ac8100a70 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8111dd0 con 0x7f0ac8106ba0 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.418+0000 7f0acd1e9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac81120d0 con 0x7f0ac81000d0 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 0x7f0ac819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 0x7f0ac819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34052/0 (socket says 172.21.15.64:34052) 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 -- 172.21.15.64:0/886121312 learned_addr learned my addr 172.21.15.64:0/886121312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac6d9d640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac81000d0 0x7f0ac819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac759e640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac8106ba0 0x7f0ac819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 -- 172.21.15.64:0/886121312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac81000d0 msgr2=0x7f0ac819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac81000d0 0x7f0ac819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 -- 172.21.15.64:0/886121312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac8106ba0 msgr2=0x7f0ac819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac8106ba0 0x7f0ac819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 -- 172.21.15.64:0/886121312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac819f660 con 0x7f0ac8100a70 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac6d9d640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac81000d0 0x7f0ac819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0ac659c640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 0x7f0ac819ac90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0abc004770 tx=0x7f0abc003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc03f070 con 0x7f0ac8100a70 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac819f8f0 con 0x7f0ac8100a70 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0abc03bd20 con 0x7f0ac8100a70 2024-09-19T05:03:34.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc035030 con 0x7f0ac8100a70 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.419+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac81a7230 con 0x7f0ac8100a70 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.420+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac81033f0 con 0x7f0ac8100a70 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.422+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0abc042070 con 0x7f0ac8100a70 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.422+0000 7f0aaffff640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0ab00a3240 0x7f0ab00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.422+0000 7f0ac6d9d640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0ab00a3240 0x7f0ab00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.422+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0abc0f01b0 con 0x7f0ac8100a70 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.423+0000 7f0ac6d9d640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0ab00a3240 0x7f0ab00a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005fd0 tx=0x7f0ab8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:34.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.426+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0abc0bd2d0 con 0x7f0ac8100a70 2024-09-19T05:03:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.642+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac81a7400 con 0x7f0ac8100a70 2024-09-19T05:03:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.643+0000 7f0aaffff640 1 -- 172.21.15.64:0/886121312 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7f0abc0bcbd0 con 0x7f0ac8100a70 2024-09-19T05:03:34.646 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:34.646 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":4,"btime":"2024-09-19T05:03:33:798866+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:03:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 4 2024-09-19T05:03:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0ab00a3240 msgr2=0x7f0ab00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0ab00a3240 0x7f0ab00a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005fd0 tx=0x7f0ab8005850 comp rx=0 tx=0).stop 2024-09-19T05:03:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 msgr2=0x7f0ac819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 0x7f0ac819ac90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0abc004770 tx=0x7f0abc003ca0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 shutdown_connections 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0ab00a3240 0x7f0ab00a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac8106ba0 0x7f0ac819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ac8100a70 0x7f0ac819ac90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 --2- 172.21.15.64:0/886121312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac81000d0 0x7f0ac819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 >> 172.21.15.64:0/886121312 conn(0x7f0ac8077210 msgr2=0x7f0ac810e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 shutdown_connections 2024-09-19T05:03:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:34.645+0000 7f0acd1e9640 1 -- 172.21.15.64:0/886121312 wait complete. 2024-09-19T05:03:34.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.64:6834/726782789,v1:172.21.15.64:6835/726782789] up:boot 2024-09-19T05:03:34.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: fsmap 3 up:standby 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: Deploying daemon mds.d on smithi064 2024-09-19T05:03:34.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:34 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/886121312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:35.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.64:6834/726782789,v1:172.21.15.64:6835/726782789] up:boot 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: fsmap 3 up:standby 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: Deploying daemon mds.d on smithi064 2024-09-19T05:03:35.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:34 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/886121312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:35.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.64:6834/726782789,v1:172.21.15.64:6835/726782789] up:boot 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: fsmap 3 up:standby 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: Deploying daemon mds.d on smithi064 2024-09-19T05:03:35.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:34 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/886121312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:36.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:35 smithi174 ceph-mon[34488]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:36.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:35 smithi064 ceph-mon[30960]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:36.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:35 smithi103 ceph-mon[34446]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:36.884 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:36 smithi064 systemd[1]: Starting Ceph mds.d for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:03:37.144 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:36 smithi064 podman[69184]: 2024-09-19 05:03:36.785908328 +0000 UTC m=+0.016442741 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:37.144 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:36 smithi064 podman[69184]: 2024-09-19 05:03:36.908155231 +0000 UTC m=+0.138689640 container create 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T05:03:37.145 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:37 smithi064 podman[69184]: 2024-09-19 05:03:37.14384495 +0000 UTC m=+0.374379370 container init 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T05:03:37.400 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:37 smithi064 podman[69184]: 2024-09-19 05:03:37.146891251 +0000 UTC m=+0.377425662 container start 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T05:03:37.401 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:37 smithi064 ceph-mds[69204]: -- 172.21.15.64:0/2434406905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3012f3860 con 0x55d3012d1400 2024-09-19T05:03:37.401 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d[69200]: starting mds.d at 2024-09-19T05:03:37.401 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:37 smithi064 bash[69184]: 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 2024-09-19T05:03:37.401 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:03:37 smithi064 systemd[1]: Started Ceph mds.d for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:37.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:37.700 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:38.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 -- 172.21.15.64:0/20341160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34106a90 msgr2=0x7f2f34106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/20341160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34106a90 0x7f2f34106e70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009930 tx=0x7f2f2802a420 comp rx=0 tx=0).stop 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 -- 172.21.15.64:0/20341160 shutdown_connections 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/20341160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101430 0x7f2f3410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/20341160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 0x7f2f34100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/20341160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34106a90 0x7f2f34106e70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 -- 172.21.15.64:0/20341160 >> 172.21.15.64:0/20341160 conn(0x7f2f340fc7d0 msgr2=0x7f2f340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 -- 172.21.15.64:0/20341160 shutdown_connections 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.121+0000 7f2f3acbd640 1 -- 172.21.15.64:0/20341160 wait complete. 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 Processor -- start 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 -- start start 2024-09-19T05:03:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 0x7f2f34199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 0x7f2f34199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34106a90 0x7f2f341a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34110de0 con 0x7f2f34101430 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34110c60 con 0x7f2f34100a90 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3acbd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34110f60 con 0x7f2f34106a90 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 0x7f2f34199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 0x7f2f34199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34086/0 (socket says 172.21.15.64:34086) 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f33fff640 1 -- 172.21.15.64:0/428567952 learned_addr learned my addr 172.21.15.64:0/428567952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f38a59640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 0x7f2f34199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.122+0000 7f2f3925a640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34106a90 0x7f2f341a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f33fff640 1 -- 172.21.15.64:0/428567952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34106a90 msgr2=0x7f2f341a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f33fff640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34106a90 0x7f2f341a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f33fff640 1 -- 172.21.15.64:0/428567952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 msgr2=0x7f2f34199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f33fff640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 0x7f2f34199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f33fff640 1 -- 172.21.15.64:0/428567952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f341a48d0 con 0x7f2f34101430 2024-09-19T05:03:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f38a59640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 0x7f2f34199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f33fff640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 0x7f2f34199850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000b4a0 tx=0x7f2f2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20014070 con 0x7f2f34101430 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f20010ac0 con 0x7f2f34101430 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f341a4aa0 con 0x7f2f34101430 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20017c30 con 0x7f2f34101430 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.123+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f341a5030 con 0x7f2f34101430 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.124+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f34100f70 con 0x7f2f34101430 2024-09-19T05:03:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.125+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2f20017070 con 0x7f2f34101430 2024-09-19T05:03:38.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.126+0000 7f2f31ffb640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2f240a32c0 0x7f2f240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:38.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.126+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2f200ce9f0 con 0x7f2f34101430 2024-09-19T05:03:38.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.126+0000 7f2f38a59640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2f240a32c0 0x7f2f240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:38.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.127+0000 7f2f38a59640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2f240a32c0 0x7f2f240a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f28006010 tx=0x7f2f28000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:38.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.130+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f2009b560 con 0x7f2f34101430 2024-09-19T05:03:38.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.341+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f34109000 con 0x7f2f34101430 2024-09-19T05:03:38.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.342+0000 7f2f31ffb640 1 -- 172.21.15.64:0/428567952 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f2f2009f860 con 0x7f2f34101430 2024-09-19T05:03:38.343 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:38.343 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"btime":"2024-09-19T05:03:37:838314+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:03:38.344 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 5 2024-09-19T05:03:38.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2f240a32c0 msgr2=0x7f2f240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:38.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2f240a32c0 0x7f2f240a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f28006010 tx=0x7f2f28000bc0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 msgr2=0x7f2f34199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 0x7f2f34199850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000b4a0 tx=0x7f2f2000b960 comp rx=0 tx=0).stop 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 shutdown_connections 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2f240a32c0 0x7f2f240a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34106a90 0x7f2f341a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101430 0x7f2f34199850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 --2- 172.21.15.64:0/428567952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f34100a90 0x7f2f34199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:38.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 >> 172.21.15.64:0/428567952 conn(0x7f2f340fc7d0 msgr2=0x7f2f340fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:38.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 shutdown_connections 2024-09-19T05:03:38.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:38.344+0000 7f2f3acbd640 1 -- 172.21.15.64:0/428567952 wait complete. 2024-09-19T05:03:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: Deploying daemon mds.e on smithi103 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.64:6836/2636818153,v1:172.21.15.64:6837/2636818153] up:boot 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: fsmap 4 up:standby 2024-09-19T05:03:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:38 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:03:38.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: Deploying daemon mds.e on smithi103 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.64:6836/2636818153,v1:172.21.15.64:6837/2636818153] up:boot 2024-09-19T05:03:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: fsmap 4 up:standby 2024-09-19T05:03:38.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:38 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:03:38.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: Deploying daemon mds.e on smithi103 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.64:6836/2636818153,v1:172.21.15.64:6837/2636818153] up:boot 2024-09-19T05:03:38.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: fsmap 4 up:standby 2024-09-19T05:03:38.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:38 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:03:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:39 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/428567952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:39.461 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:39 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/428567952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:39.718 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:03:39 smithi103 podman[63950]: 2024-09-19 05:03:39.695399964 +0000 UTC m=+0.348836583 container start 24163245ae70915e17c429c710054084b1fb711bfde6f5a5b35c9062964f76ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T05:03:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:39 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/428567952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:40.218 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:03:39 smithi103 ceph-mds[63969]: -- 172.21.15.103:0/4072259635 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f668a5860 con 0x562f66883400 2024-09-19T05:03:40.219 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:03:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e[63965]: starting mds.e at 2024-09-19T05:03:40.219 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:03:39 smithi103 bash[63950]: 24163245ae70915e17c429c710054084b1fb711bfde6f5a5b35c9062964f76ae 2024-09-19T05:03:40.219 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:03:39 smithi103 systemd[1]: Started Ceph mds.e for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:40.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:40.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:40.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:40.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:40 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:40.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:40.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:40.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:40.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:40 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:40.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:40.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:40.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:40.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:40.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:40 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:41.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:41.373 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:41 smithi064 ceph-mon[30960]: Deploying daemon mds.f on smithi174 2024-09-19T05:03:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:41 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6834/642500144,v1:172.21.15.103:6835/642500144] up:boot 2024-09-19T05:03:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:41 smithi064 ceph-mon[30960]: fsmap 5 up:standby 2024-09-19T05:03:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:41 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:03:41.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:41 smithi174 ceph-mon[34488]: Deploying daemon mds.f on smithi174 2024-09-19T05:03:41.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:41 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6834/642500144,v1:172.21.15.103:6835/642500144] up:boot 2024-09-19T05:03:41.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:41 smithi174 ceph-mon[34488]: fsmap 5 up:standby 2024-09-19T05:03:41.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:41 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:03:41.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:41 smithi103 ceph-mon[34446]: Deploying daemon mds.f on smithi174 2024-09-19T05:03:41.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:41 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6834/642500144,v1:172.21.15.103:6835/642500144] up:boot 2024-09-19T05:03:41.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:41 smithi103 ceph-mon[34446]: fsmap 5 up:standby 2024-09-19T05:03:41.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:41 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:03:41.778 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:41 smithi174 systemd[1]: Starting Ceph mds.f for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:03:41.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.828+0000 7fb929ca3640 1 -- 172.21.15.64:0/1369367346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb924102d30 msgr2=0x7fb924103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.828+0000 7fb929ca3640 1 --2- 172.21.15.64:0/1369367346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb924102d30 0x7fb924103110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb914009930 tx=0x7fb91402a440 comp rx=0 tx=0).stop 2024-09-19T05:03:41.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 -- 172.21.15.64:0/1369367346 shutdown_connections 2024-09-19T05:03:41.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 --2- 172.21.15.64:0/1369367346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 0x7fb92410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 --2- 172.21.15.64:0/1369367346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9241036e0 0x7fb924107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 --2- 172.21.15.64:0/1369367346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb924102d30 0x7fb924103110 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 -- 172.21.15.64:0/1369367346 >> 172.21.15.64:0/1369367346 conn(0x7fb9240fc7d0 msgr2=0x7fb9240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 -- 172.21.15.64:0/1369367346 shutdown_connections 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 -- 172.21.15.64:0/1369367346 wait complete. 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 Processor -- start 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.829+0000 7fb929ca3640 1 -- start start 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb924102d30 0x7fb924100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 0x7fb924100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 0x7fb924101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111f90 con 0x7fb9241036e0 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb924111e10 con 0x7fb9241082a0 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb924112110 con 0x7fb924102d30 2024-09-19T05:03:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 0x7fb924100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 0x7fb924100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34118/0 (socket says 172.21.15.64:34118) 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 -- 172.21.15.64:0/261207721 learned_addr learned my addr 172.21.15.64:0/261207721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 -- 172.21.15.64:0/261207721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb924102d30 msgr2=0x7fb924100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb924102d30 0x7fb924100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 -- 172.21.15.64:0/261207721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 msgr2=0x7fb924101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb923fff640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 0x7fb924101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 0x7fb924101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 -- 172.21.15.64:0/261207721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9241a6a30 con 0x7fb9241036e0 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb923fff640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 0x7fb924101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb91affd640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 0x7fb924100ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006fd0 tx=0x7fb90c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c011830 con 0x7fb9241036e0 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb90c016070 con 0x7fb9241036e0 2024-09-19T05:03:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241a6d20 con 0x7fb9241036e0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.830+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c010df0 con 0x7fb9241036e0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.831+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a7200 con 0x7fb9241036e0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.832+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92406a850 con 0x7fb9241036e0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.833+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb90c0119d0 con 0x7fb9241036e0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.834+0000 7fb9217fa640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb9100a32c0 0x7fb9100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.834+0000 7fb9237fe640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb9100a32c0 0x7fb9100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.834+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb90c0ceb10 con 0x7fb9241036e0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.834+0000 7fb9237fe640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb9100a32c0 0x7fb9100a5780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9140096c0 tx=0x7fb914031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:41.838+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb90c09bcd0 con 0x7fb9241036e0 2024-09-19T05:03:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.048+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb92410a2b0 con 0x7fb9241036e0 2024-09-19T05:03:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.049+0000 7fb9217fa640 1 -- 172.21.15.64:0/261207721 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4732 (secure 0 0 0) 0x7fb90c09b5d0 con 0x7fb9241036e0 2024-09-19T05:03:42.050 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:42.050 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"btime":"2024-09-19T05:03:40:291217+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T05:03:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb9100a32c0 msgr2=0x7fb9100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb9100a32c0 0x7fb9100a5780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9140096c0 tx=0x7fb914031400 comp rx=0 tx=0).stop 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 msgr2=0x7fb924100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 0x7fb924100ed0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006fd0 tx=0x7fb90c00b8e0 comp rx=0 tx=0).stop 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 shutdown_connections 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb9100a32c0 0x7fb9100a5780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9241082a0 0x7fb924101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9241036e0 0x7fb924100ed0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 --2- 172.21.15.64:0/261207721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb924102d30 0x7fb924100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 >> 172.21.15.64:0/261207721 conn(0x7fb9240fc7d0 msgr2=0x7fb924105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 shutdown_connections 2024-09-19T05:03:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:42.051+0000 7fb929ca3640 1 -- 172.21.15.64:0/261207721 wait complete. 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:41 smithi174 podman[64247]: 2024-09-19 05:03:41.773840412 +0000 UTC m=+0.016025383 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:41 smithi174 podman[64247]: 2024-09-19 05:03:41.873903437 +0000 UTC m=+0.116088391 container create 15f2a2c6a0bd01100edcedded4e154344234c5c1d0a73b7d665dce0929eb6900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:42 smithi174 podman[64247]: 2024-09-19 05:03:42.09308825 +0000 UTC m=+0.335273221 container init 15f2a2c6a0bd01100edcedded4e154344234c5c1d0a73b7d665dce0929eb6900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:42 smithi174 podman[64247]: 2024-09-19 05:03:42.096014128 +0000 UTC m=+0.338199081 container start 15f2a2c6a0bd01100edcedded4e154344234c5c1d0a73b7d665dce0929eb6900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mds[64266]: -- 172.21.15.174:0/1574017402 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8def85860 con 0x55d8def63000 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f[64262]: starting mds.f at 2024-09-19T05:03:42.177 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:42 smithi174 bash[64247]: 15f2a2c6a0bd01100edcedded4e154344234c5c1d0a73b7d665dce0929eb6900 2024-09-19T05:03:42.527 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:03:42 smithi174 systemd[1]: Started Ceph mds.f for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:42.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/261207721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:42 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/261207721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:42.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:42.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:42 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:42.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/261207721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:42.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:42 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:43.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:43 smithi103 ceph-mon[34446]: Deploying daemon mds.c on smithi174 2024-09-19T05:03:43.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:43 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6834/2004435266,v1:172.21.15.174:6835/2004435266] up:boot 2024-09-19T05:03:43.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:43 smithi103 ceph-mon[34446]: fsmap 6 up:standby 2024-09-19T05:03:43.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:43 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:03:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:43 smithi174 ceph-mon[34488]: Deploying daemon mds.c on smithi174 2024-09-19T05:03:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:43 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6834/2004435266,v1:172.21.15.174:6835/2004435266] up:boot 2024-09-19T05:03:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:43 smithi174 ceph-mon[34488]: fsmap 6 up:standby 2024-09-19T05:03:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:43 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:03:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:43 smithi064 ceph-mon[30960]: Deploying daemon mds.c on smithi174 2024-09-19T05:03:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:43 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6834/2004435266,v1:172.21.15.174:6835/2004435266] up:boot 2024-09-19T05:03:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:43 smithi064 ceph-mon[30960]: fsmap 6 up:standby 2024-09-19T05:03:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:43 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:03:44.277 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 systemd[1]: Starting Ceph mds.c for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:03:44.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:44 smithi174 ceph-mon[34488]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:44.596 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 podman[64546]: 2024-09-19 05:03:44.284444647 +0000 UTC m=+0.107435624 container create 932430102e2c7cd8c59b4b5ead678d49e61bf554947af871a735856b615494e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T05:03:44.596 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 podman[64546]: 2024-09-19 05:03:44.193389993 +0000 UTC m=+0.016380985 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:44.596 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 podman[64546]: 2024-09-19 05:03:44.495268725 +0000 UTC m=+0.318259709 container init 932430102e2c7cd8c59b4b5ead678d49e61bf554947af871a735856b615494e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T05:03:44.596 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 podman[64546]: 2024-09-19 05:03:44.498143309 +0000 UTC m=+0.321134287 container start 932430102e2c7cd8c59b4b5ead678d49e61bf554947af871a735856b615494e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T05:03:44.596 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 ceph-mds[64566]: -- 172.21.15.174:0/3287129971 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2a46c7860 con 0x55b2a46a5000 2024-09-19T05:03:44.596 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c[64562]: starting mds.c at 2024-09-19T05:03:44.597 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 bash[64546]: 932430102e2c7cd8c59b4b5ead678d49e61bf554947af871a735856b615494e4 2024-09-19T05:03:44.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:44 smithi064 ceph-mon[30960]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:44.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:44 smithi103 ceph-mon[34446]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:44.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:45.027 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:03:44 smithi174 systemd[1]: Started Ceph mds.c for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 -- 172.21.15.64:0/3255780406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102ef0 msgr2=0x7f7b7410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/3255780406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102ef0 0x7f7b7410da40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009ca0 tx=0x7f7b6c02a950 comp rx=0 tx=0).stop 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 -- 172.21.15.64:0/3255780406 shutdown_connections 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/3255780406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102ef0 0x7f7b7410da40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/3255780406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b74102570 0x7f7b741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/3255780406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74100e20 0x7f7b74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 -- 172.21.15.64:0/3255780406 >> 172.21.15.64:0/3255780406 conn(0x7f7b740fc7d0 msgr2=0x7f7b740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 -- 172.21.15.64:0/3255780406 shutdown_connections 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 -- 172.21.15.64:0/3255780406 wait complete. 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 Processor -- start 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.572+0000 7f7b7b60b640 1 -- start start 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b7b60b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b74100e20 0x7f7b7419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b7b60b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 0x7f7b7419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b7b60b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74102ef0 0x7f7b7419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b7b60b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74110e50 con 0x7f7b74102570 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b7b60b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74110cd0 con 0x7f7b74102ef0 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b7b60b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74110fd0 con 0x7f7b74100e20 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 0x7f7b7419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 0x7f7b7419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58684/0 (socket says 172.21.15.64:58684) 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 -- 172.21.15.64:0/725476375 learned_addr learned my addr 172.21.15.64:0/725476375 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b793a7640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b74100e20 0x7f7b7419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b79ba8640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74102ef0 0x7f7b7419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 -- 172.21.15.64:0/725476375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b74100e20 msgr2=0x7f7b7419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b74100e20 0x7f7b7419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 -- 172.21.15.64:0/725476375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74102ef0 msgr2=0x7f7b7419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74102ef0 0x7f7b7419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 -- 172.21.15.64:0/725476375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7419f5a0 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b79ba8640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74102ef0 0x7f7b7419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.573+0000 7f7b78ba6640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 0x7f7b7419ab30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b4a0 tx=0x7f7b6400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.574+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64014070 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.574+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b64010ac0 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.574+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64017c30 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.574+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7419f890 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.574+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a7170 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.575+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b74101200 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.576+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7b64017070 con 0x7f7b74102570 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.577+0000 7f7b6a7fc640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b600a32c0 0x7f7b600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.577+0000 7f7b793a7640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b600a32c0 0x7f7b600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.577+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7b640ceab0 con 0x7f7b74102570 2024-09-19T05:03:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.577+0000 7f7b793a7640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b600a32c0 0x7f7b600a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000bbe0 tx=0x7f7b7000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.581+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b6409bc70 con 0x7f7b74102570 2024-09-19T05:03:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: Deploying daemon mds.h on smithi103 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] up:boot 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: fsmap 7 up:standby 2024-09-19T05:03:45.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:45 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:03:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.794+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b741a7430 con 0x7f7b74102570 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: Deploying daemon mds.h on smithi103 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] up:boot 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: fsmap 7 up:standby 2024-09-19T05:03:45.797 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:45 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:03:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.795+0000 7f7b6a7fc640 1 -- 172.21.15.64:0/725476375 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f7b640bb020 con 0x7f7b74102570 2024-09-19T05:03:45.798 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:45.798 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"btime":"2024-09-19T05:03:45:314908+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T05:03:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-09-19T05:03:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b600a32c0 msgr2=0x7f7b600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b600a32c0 0x7f7b600a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000bbe0 tx=0x7f7b7000b390 comp rx=0 tx=0).stop 2024-09-19T05:03:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 msgr2=0x7f7b7419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 0x7f7b7419ab30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b4a0 tx=0x7f7b6400b960 comp rx=0 tx=0).stop 2024-09-19T05:03:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 shutdown_connections 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b600a32c0 0x7f7b600a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b74102ef0 0x7f7b7419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b74102570 0x7f7b7419ab30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 --2- 172.21.15.64:0/725476375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b74100e20 0x7f7b7419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 >> 172.21.15.64:0/725476375 conn(0x7f7b740fc7d0 msgr2=0x7f7b7406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 shutdown_connections 2024-09-19T05:03:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:45.799+0000 7f7b7b60b640 1 -- 172.21.15.64:0/725476375 wait complete. 2024-09-19T05:03:46.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:46.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: Deploying daemon mds.h on smithi103 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] up:boot 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: fsmap 7 up:standby 2024-09-19T05:03:46.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:45 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:03:46.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:46 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/725476375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:47.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:46 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/725476375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:46 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/725476375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:47.468 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:03:47 smithi103 systemd[1]: Started Ceph mds.h for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:48.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:48.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:48.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: Deploying daemon mds.i on smithi174 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6836/3232384876,v1:172.21.15.103:6837/3232384876] up:boot 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: fsmap 8 up:standby 2024-09-19T05:03:48.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:48 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:03:48.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:48.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: Deploying daemon mds.i on smithi174 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6836/3232384876,v1:172.21.15.103:6837/3232384876] up:boot 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: fsmap 8 up:standby 2024-09-19T05:03:48.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:48 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:03:48.712 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:48.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:48.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: Deploying daemon mds.i on smithi174 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6836/3232384876,v1:172.21.15.103:6837/3232384876] up:boot 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: fsmap 8 up:standby 2024-09-19T05:03:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:48 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:03:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 -- 172.21.15.64:0/399669627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 msgr2=0x7fc9ac1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/399669627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1031b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009930 tx=0x7fc9a402a420 comp rx=0 tx=0).stop 2024-09-19T05:03:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 -- 172.21.15.64:0/399669627 shutdown_connections 2024-09-19T05:03:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/399669627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9ac108300 0x7fc9ac10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/399669627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9ac103780 0x7fc9ac107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/399669627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1031b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 -- 172.21.15.64:0/399669627 >> 172.21.15.64:0/399669627 conn(0x7fc9ac0fc7d0 msgr2=0x7fc9ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 -- 172.21.15.64:0/399669627 shutdown_connections 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.161+0000 7fc9b47a1640 1 -- 172.21.15.64:0/399669627 wait complete. 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 Processor -- start 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 -- start start 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9ac103780 0x7fc9ac1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9ac108300 0x7fc9ac19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac111fc0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac111e40 con 0x7fc9ac103780 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b47a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac112140 con 0x7fc9ac108300 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58700/0 (socket says 172.21.15.64:58700) 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 -- 172.21.15.64:0/3227404097 learned_addr learned my addr 172.21.15.64:0/3227404097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b2d3e640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9ac108300 0x7fc9ac19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b1d3c640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9ac103780 0x7fc9ac1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 -- 172.21.15.64:0/3227404097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9ac108300 msgr2=0x7fc9ac19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9ac108300 0x7fc9ac19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 -- 172.21.15.64:0/3227404097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9ac103780 msgr2=0x7fc9ac1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9ac103780 0x7fc9ac1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.162+0000 7fc9b253d640 1 -- 172.21.15.64:0/3227404097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac19ace0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.163+0000 7fc9b253d640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1a03f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4004770 tx=0x7fc9a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.163+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a403f070 con 0x7fc9ac102dd0 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.163+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a403b460 con 0x7fc9ac102dd0 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.163+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4035070 con 0x7fc9ac102dd0 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.163+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac19af70 con 0x7fc9ac102dd0 2024-09-19T05:03:49.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.163+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1a72c0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.164+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac1031b0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.165+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc9a403b610 con 0x7fc9ac102dd0 2024-09-19T05:03:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.166+0000 7fc9ab7fe640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9940a3240 0x7fc9940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.166+0000 7fc9b1d3c640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9940a3240 0x7fc9940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.166+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc9a40efb00 con 0x7fc9ac102dd0 2024-09-19T05:03:49.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.166+0000 7fc9b1d3c640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9940a3240 0x7fc9940a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0ff0c0 tx=0x7fc9a000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:49.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.169+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc9a40bccc0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.278 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 systemd[1]: Starting Ceph mds.i for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:03:49.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.382+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9ac10c2a0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.383+0000 7fc9ab7fe640 1 -- 172.21.15.64:0/3227404097 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7fc9a40bc5c0 con 0x7fc9ac102dd0 2024-09-19T05:03:49.384 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:49.384 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":9,"btime":"2024-09-19T05:03:47:653184+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T05:03:49.384 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 9 2024-09-19T05:03:49.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9940a3240 msgr2=0x7fc9940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9940a3240 0x7fc9940a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0ff0c0 tx=0x7fc9a000b590 comp rx=0 tx=0).stop 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 msgr2=0x7fc9ac1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1a03f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4004770 tx=0x7fc9a40042f0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 shutdown_connections 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9940a3240 0x7fc9940a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9ac108300 0x7fc9ac19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9ac103780 0x7fc9ac1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 --2- 172.21.15.64:0/3227404097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac102dd0 0x7fc9ac1a03f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 >> 172.21.15.64:0/3227404097 conn(0x7fc9ac0fc7d0 msgr2=0x7fc9ac100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 shutdown_connections 2024-09-19T05:03:49.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:49.386+0000 7fc9b47a1640 1 -- 172.21.15.64:0/3227404097 wait complete. 2024-09-19T05:03:49.607 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 podman[64848]: 2024-09-19 05:03:49.295640382 +0000 UTC m=+0.104185345 container create 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T05:03:49.607 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 podman[64848]: 2024-09-19 05:03:49.207336869 +0000 UTC m=+0.015881831 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:49.608 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 podman[64848]: 2024-09-19 05:03:49.506394643 +0000 UTC m=+0.314939605 container init 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T05:03:49.608 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 podman[64848]: 2024-09-19 05:03:49.509235544 +0000 UTC m=+0.317780505 container start 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T05:03:49.608 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 ceph-mds[64868]: -- 172.21.15.174:0/2028781602 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de3897d860 con 0x55de3895b000 2024-09-19T05:03:49.608 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i[64864]: starting mds.i at 2024-09-19T05:03:49.608 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 bash[64848]: 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc 2024-09-19T05:03:50.027 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:03:49 smithi174 systemd[1]: Started Ceph mds.i for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:50.494 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:50.494 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3227404097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:50.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:50.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:50.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:50 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:50.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3227404097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:50.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:50 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3227404097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:50.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:50 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:51.508 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:51 smithi064 ceph-mon[30960]: Deploying daemon mds.g on smithi064 2024-09-19T05:03:51.508 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:51 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] up:boot 2024-09-19T05:03:51.508 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:51 smithi064 ceph-mon[30960]: fsmap 9 up:standby 2024-09-19T05:03:51.508 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:51 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:03:51.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:51 smithi174 ceph-mon[34488]: Deploying daemon mds.g on smithi064 2024-09-19T05:03:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:51 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] up:boot 2024-09-19T05:03:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:51 smithi174 ceph-mon[34488]: fsmap 9 up:standby 2024-09-19T05:03:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:51 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:03:51.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:51 smithi103 ceph-mon[34446]: Deploying daemon mds.g on smithi064 2024-09-19T05:03:51.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:51 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] up:boot 2024-09-19T05:03:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:51 smithi103 ceph-mon[34446]: fsmap 9 up:standby 2024-09-19T05:03:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:51 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:03:51.900 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:51 smithi064 systemd[1]: Starting Ceph mds.g for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:03:52.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:52.164 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:51 smithi064 podman[69951]: 2024-09-19 05:03:51.815392376 +0000 UTC m=+0.016473806 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:52.164 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:51 smithi064 podman[69951]: 2024-09-19 05:03:51.932960081 +0000 UTC m=+0.134041550 container create 648d649d97a1249acc8fc758fb37d763b6084c992b8935db230c209587c64872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T05:03:52.164 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:52 smithi064 podman[69951]: 2024-09-19 05:03:52.160357119 +0000 UTC m=+0.361438555 container init 648d649d97a1249acc8fc758fb37d763b6084c992b8935db230c209587c64872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:03:52.164 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:52 smithi064 podman[69951]: 2024-09-19 05:03:52.163563102 +0000 UTC m=+0.364644534 container start 648d649d97a1249acc8fc758fb37d763b6084c992b8935db230c209587c64872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:03:52.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:52.472 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:52 smithi064 ceph-mon[30960]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:52.473 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:52 smithi064 ceph-mds[69988]: -- 172.21.15.64:0/55172970 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd436bb860 con 0x55dd43699400 2024-09-19T05:03:52.473 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g[69966]: starting mds.g at 2024-09-19T05:03:52.473 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:52 smithi064 bash[69951]: 648d649d97a1249acc8fc758fb37d763b6084c992b8935db230c209587c64872 2024-09-19T05:03:52.473 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:03:52 smithi064 systemd[1]: Started Ceph mds.g for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:52.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:52 smithi103 ceph-mon[34446]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:52.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:52 smithi174 ceph-mon[34488]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:52.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 -- 172.21.15.64:0/1221259552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe92010e810 msgr2=0x7fe920110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:52.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 --2- 172.21.15.64:0/1221259552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe92010e810 0x7fe920110c00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe918009ca0 tx=0x7fe91802a8b0 comp rx=0 tx=0).stop 2024-09-19T05:03:52.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 -- 172.21.15.64:0/1221259552 shutdown_connections 2024-09-19T05:03:52.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 --2- 172.21.15.64:0/1221259552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe92010e810 0x7fe920110c00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:52.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 --2- 172.21.15.64:0/1221259552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe920102fc0 0x7fe9201005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:52.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 --2- 172.21.15.64:0/1221259552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe920102b50 0x7fe920100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 -- 172.21.15.64:0/1221259552 >> 172.21.15.64:0/1221259552 conn(0x7fe920077210 msgr2=0x7fe9200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 -- 172.21.15.64:0/1221259552 shutdown_connections 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.871+0000 7fe9277c3640 1 -- 172.21.15.64:0/1221259552 wait complete. 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 Processor -- start 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 -- start start 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 0x7fe92019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe92010e810 0x7fe92019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe92019edd0 0x7fe92019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe920114060 con 0x7fe920102b50 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe920113ee0 con 0x7fe92010e810 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe9277c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201141e0 con 0x7fe92019edd0 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe92555f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 0x7fe92019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:52.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe92555f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 0x7fe92019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58730/0 (socket says 172.21.15.64:58730) 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe92555f640 1 -- 172.21.15.64:0/3686517997 learned_addr learned my addr 172.21.15.64:0/3686517997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.872+0000 7fe924d5e640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe92010e810 0x7fe92019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe925d60640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe92019edd0 0x7fe92019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe92555f640 1 -- 172.21.15.64:0/3686517997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe92019edd0 msgr2=0x7fe92019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe92555f640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe92019edd0 0x7fe92019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe92555f640 1 -- 172.21.15.64:0/3686517997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe92010e810 msgr2=0x7fe92019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe92555f640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe92010e810 0x7fe92019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe92555f640 1 -- 172.21.15.64:0/3686517997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92019f890 con 0x7fe920102b50 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe925d60640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe92019edd0 0x7fe92019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe92555f640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 0x7fe92019a500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c920 tx=0x7fe91000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910007c70 con 0x7fe920102b50 2024-09-19T05:03:52.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92019fb80 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe910007e10 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910018820 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.873+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201a73b0 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.874+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe920100090 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.875+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe9100189c0 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.876+0000 7fe9167fc640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe90c0a3240 0x7fe90c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.876+0000 7fe924d5e640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe90c0a3240 0x7fe90c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.876+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fe9100cf8b0 con 0x7fe920102b50 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.876+0000 7fe924d5e640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe90c0a3240 0x7fe90c0a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9200feef0 tx=0x7fe91c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:52.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:52.879+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe910014030 con 0x7fe920102b50 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.100+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe92019b860 con 0x7fe920102b50 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.100+0000 7fe9167fc640 1 -- 172.21.15.64:0/3686517997 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8872 (secure 0 0 0) 0x7fe91009cae0 con 0x7fe920102b50 2024-09-19T05:03:53.102 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:53.102 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"btime":"2024-09-19T05:03:52:284375+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T05:03:53.102 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.102+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe90c0a3240 msgr2=0x7fe90c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.102+0000 7fe9277c3640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe90c0a3240 0x7fe90c0a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9200feef0 tx=0x7fe91c00b410 comp rx=0 tx=0).stop 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 msgr2=0x7fe92019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 0x7fe92019a500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c920 tx=0x7fe91000cde0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 shutdown_connections 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe90c0a3240 0x7fe90c0a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe92019edd0 0x7fe92019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe92010e810 0x7fe92019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 --2- 172.21.15.64:0/3686517997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe920102b50 0x7fe92019a500 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 >> 172.21.15.64:0/3686517997 conn(0x7fe920077210 msgr2=0x7fe92010fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 shutdown_connections 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:53.103+0000 7fe9277c3640 1 -- 172.21.15.64:0/3686517997 wait complete. 2024-09-19T05:03:53.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.64:6838/4284831800,v1:172.21.15.64:6839/4284831800] up:boot 2024-09-19T05:03:53.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: fsmap 10 up:standby 2024-09-19T05:03:53.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:03:53.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:53.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:53.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:53.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:53 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3686517997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:53.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.64:6838/4284831800,v1:172.21.15.64:6839/4284831800] up:boot 2024-09-19T05:03:53.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: fsmap 10 up:standby 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:53 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3686517997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.64:6838/4284831800,v1:172.21.15.64:6839/4284831800] up:boot 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: fsmap 10 up:standby 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:53 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3686517997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:54.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:54 smithi064 ceph-mon[30960]: Deploying daemon mds.a on smithi064 2024-09-19T05:03:54.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:54 smithi064 ceph-mon[30960]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:54.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:54 smithi103 ceph-mon[34446]: Deploying daemon mds.a on smithi064 2024-09-19T05:03:54.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:54 smithi103 ceph-mon[34446]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:54 smithi174 ceph-mon[34488]: Deploying daemon mds.a on smithi064 2024-09-19T05:03:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:54 smithi174 ceph-mon[34488]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:55.579 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 podman[70367]: 2024-09-19 05:03:55.578068661 +0000 UTC m=+0.154457218 container create 4e6b160f101019f8dd2ba41854e0f29b7eba3a6ba7bae0c6820a68639af14b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T05:03:55.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:56.006 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:56.128 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 podman[70367]: 2024-09-19 05:03:55.83047438 +0000 UTC m=+0.406862936 container init 4e6b160f101019f8dd2ba41854e0f29b7eba3a6ba7bae0c6820a68639af14b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-19T05:03:56.129 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 podman[70367]: 2024-09-19 05:03:55.834300567 +0000 UTC m=+0.410689117 container start 4e6b160f101019f8dd2ba41854e0f29b7eba3a6ba7bae0c6820a68639af14b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:03:56.129 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 ceph-mds[70404]: -- 172.21.15.64:0/1160715136 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e589c5860 con 0x560e589a3000 2024-09-19T05:03:56.129 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a[70382]: starting mds.a at 2024-09-19T05:03:56.129 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 bash[70367]: 4e6b160f101019f8dd2ba41854e0f29b7eba3a6ba7bae0c6820a68639af14b42 2024-09-19T05:03:56.129 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:03:55 smithi064 systemd[1]: Started Ceph mds.a for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:56.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:56.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:56.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:56.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:56 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:56.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.435+0000 7f6709482640 1 -- 172.21.15.64:0/1000259335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704101de0 msgr2=0x7f67041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.435+0000 7f6709482640 1 --2- 172.21.15.64:0/1000259335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704101de0 0x7f67041021c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009900 tx=0x7f66f402a410 comp rx=0 tx=0).stop 2024-09-19T05:03:56.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.435+0000 7f6709482640 1 -- 172.21.15.64:0/1000259335 shutdown_connections 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 --2- 172.21.15.64:0/1000259335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704106f10 0x7f670410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 --2- 172.21.15.64:0/1000259335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704102790 0x7f67041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 --2- 172.21.15.64:0/1000259335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704101de0 0x7f67041021c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 -- 172.21.15.64:0/1000259335 >> 172.21.15.64:0/1000259335 conn(0x7f67040fd8c0 msgr2=0x7f67040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 -- 172.21.15.64:0/1000259335 shutdown_connections 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 -- 172.21.15.64:0/1000259335 wait complete. 2024-09-19T05:03:56.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 Processor -- start 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.436+0000 7f6709482640 1 -- start start 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704101de0 0x7f67040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704102790 0x7f6704074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 0x7f6704074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6704111e20 con 0x7f6704106f10 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6704111ca0 con 0x7f6704101de0 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6704111fa0 con 0x7f6704102790 2024-09-19T05:03:56.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 0x7f6704074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 0x7f6704074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35734/0 (socket says 172.21.15.64:35734) 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 -- 172.21.15.64:0/3795000559 learned_addr learned my addr 172.21.15.64:0/3795000559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6702ffd640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704101de0 0x7f67040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67027fc640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704102790 0x7f6704074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 -- 172.21.15.64:0/3795000559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704102790 msgr2=0x7f6704074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704102790 0x7f6704074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 -- 172.21.15.64:0/3795000559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704101de0 msgr2=0x7f67040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704101de0 0x7f67040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 -- 172.21.15.64:0/3795000559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67041a6820 con 0x7f6704106f10 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6702ffd640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704101de0 0x7f67040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:56.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f67037fe640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 0x7f6704074f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00eaa0 tx=0x7f66fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00cdb0 con 0x7f6704106f10 2024-09-19T05:03:56.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66fc005020 con 0x7f6704106f10 2024-09-19T05:03:56.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67041a6b10 con 0x7f6704106f10 2024-09-19T05:03:56.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.437+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0116d0 con 0x7f6704106f10 2024-09-19T05:03:56.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.438+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a7050 con 0x7f6704106f10 2024-09-19T05:03:56.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.439+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67041021c0 con 0x7f6704106f10 2024-09-19T05:03:56.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.440+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f66fc0051d0 con 0x7f6704106f10 2024-09-19T05:03:56.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.440+0000 7f66ebfff640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f66ec0a3240 0x7f66ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.441+0000 7f6702ffd640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f66ec0a3240 0x7f66ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.441+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f66fc0cf680 con 0x7f6704106f10 2024-09-19T05:03:56.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.441+0000 7f6702ffd640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f66ec0a3240 0x7f66ec0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66f4031440 tx=0x7f66f4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.445+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66fc016030 con 0x7f6704106f10 2024-09-19T05:03:56.568 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:56.568 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.568 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.568 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.568 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:56.568 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:56.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:56 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.657+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f670410a0f0 con 0x7f6704106f10 2024-09-19T05:03:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.658+0000 7f66ebfff640 1 -- 172.21.15.64:0/3795000559 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9698 (secure 0 0 0) 0x7f66fc09c840 con 0x7f6704106f10 2024-09-19T05:03:56.659 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:03:56.659 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"btime":"2024-09-19T05:03:56:295840+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T05:03:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-09-19T05:03:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f66ec0a3240 msgr2=0x7f66ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f66ec0a3240 0x7f66ec0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66f4031440 tx=0x7f66f4005a20 comp rx=0 tx=0).stop 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 msgr2=0x7f6704074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 0x7f6704074f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00eaa0 tx=0x7f66fc00ef60 comp rx=0 tx=0).stop 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 shutdown_connections 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f66ec0a3240 0x7f66ec0a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6704106f10 0x7f6704074f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704102790 0x7f6704074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 --2- 172.21.15.64:0/3795000559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6704101de0 0x7f67040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 >> 172.21.15.64:0/3795000559 conn(0x7f67040fd8c0 msgr2=0x7f67040fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 shutdown_connections 2024-09-19T05:03:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:03:56.660+0000 7f6709482640 1 -- 172.21.15.64:0/3795000559 wait complete. 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:03:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:56 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:57.615 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:57 smithi103 ceph-mon[34446]: Deploying daemon mds.b on smithi103 2024-09-19T05:03:57.615 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:57 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.64:6840/3250181965,v1:172.21.15.64:6841/3250181965] up:boot 2024-09-19T05:03:57.615 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:57 smithi103 ceph-mon[34446]: fsmap 11 up:standby 2024-09-19T05:03:57.615 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:57 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:03:57.616 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:57 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3795000559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:57 smithi064 ceph-mon[30960]: Deploying daemon mds.b on smithi103 2024-09-19T05:03:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:57 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.64:6840/3250181965,v1:172.21.15.64:6841/3250181965] up:boot 2024-09-19T05:03:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:57 smithi064 ceph-mon[30960]: fsmap 11 up:standby 2024-09-19T05:03:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:57 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:03:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:57 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3795000559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:57.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:57 smithi174 ceph-mon[34488]: Deploying daemon mds.b on smithi103 2024-09-19T05:03:57.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:57 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.64:6840/3250181965,v1:172.21.15.64:6841/3250181965] up:boot 2024-09-19T05:03:57.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:57 smithi174 ceph-mon[34488]: fsmap 11 up:standby 2024-09-19T05:03:57.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:57 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:03:57.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:57 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3795000559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:58.311 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 podman[64562]: 2024-09-19 05:03:58.118980265 +0000 UTC m=+0.015785265 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:03:58.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:58 smithi064 ceph-mon[30960]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:58.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:58 smithi103 ceph-mon[34446]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:58.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 podman[64562]: 2024-09-19 05:03:58.436345089 +0000 UTC m=+0.333150088 container init bad5c262f7406920a5cb4986fd8f73f0057a867f7b8eead3443655b443e46679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:03:58.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 podman[64562]: 2024-09-19 05:03:58.439672166 +0000 UTC m=+0.336477149 container start bad5c262f7406920a5cb4986fd8f73f0057a867f7b8eead3443655b443e46679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:03:58.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 ceph-mds[64581]: -- 172.21.15.103:0/3365045885 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56417877f860 con 0x56417875d400 2024-09-19T05:03:58.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: starting mds.b at 2024-09-19T05:03:58.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 bash[64562]: bad5c262f7406920a5cb4986fd8f73f0057a867f7b8eead3443655b443e46679 2024-09-19T05:03:58.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:03:58 smithi103 systemd[1]: Started Ceph mds.b for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:03:58.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:58 smithi174 ceph-mon[34488]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:59.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:59.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:59.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:59.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] up:boot 2024-09-19T05:03:59.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: fsmap 12 up:standby 2024-09-19T05:03:59.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:03:59 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:03:59.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:03:59.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:59.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] up:boot 2024-09-19T05:03:59.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: fsmap 12 up:standby 2024-09-19T05:03:59.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:03:59 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:04:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] up:boot 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: fsmap 12 up:standby 2024-09-19T05:04:00.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:03:59 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:04:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 -- 172.21.15.64:0/1302940098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24101de0 msgr2=0x7fbb241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 --2- 172.21.15.64:0/1302940098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24101de0 0x7fbb241021c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009860 tx=0x7fbb1402a340 comp rx=0 tx=0).stop 2024-09-19T05:04:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 -- 172.21.15.64:0/1302940098 shutdown_connections 2024-09-19T05:04:00.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 --2- 172.21.15.64:0/1302940098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24106f10 0x7fbb2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 --2- 172.21.15.64:0/1302940098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb24102790 0x7fbb241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 --2- 172.21.15.64:0/1302940098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24101de0 0x7fbb241021c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 -- 172.21.15.64:0/1302940098 >> 172.21.15.64:0/1302940098 conn(0x7fbb240fd8c0 msgr2=0x7fbb240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 -- 172.21.15.64:0/1302940098 shutdown_connections 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.396+0000 7fbb2a079640 1 -- 172.21.15.64:0/1302940098 wait complete. 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 Processor -- start 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 -- start start 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 0x7fbb241983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24102790 0x7fbb241988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb24106f10 0x7fbb2419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24111e20 con 0x7fbb24101de0 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24111ca0 con 0x7fbb24102790 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb2a079640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24111fa0 con 0x7fbb24106f10 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 0x7fbb241983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 0x7fbb241983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35750/0 (socket says 172.21.15.64:35750) 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb237fe640 1 -- 172.21.15.64:0/83553276 learned_addr learned my addr 172.21.15.64:0/83553276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb22ffd640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24102790 0x7fbb241988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.397+0000 7fbb23fff640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb24106f10 0x7fbb2419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb237fe640 1 -- 172.21.15.64:0/83553276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb24106f10 msgr2=0x7fbb2419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb237fe640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb24106f10 0x7fbb2419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb237fe640 1 -- 172.21.15.64:0/83553276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24102790 msgr2=0x7fbb241988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb237fe640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24102790 0x7fbb241988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb237fe640 1 -- 172.21.15.64:0/83553276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2419d350 con 0x7fbb24101de0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb237fe640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 0x7fbb241983a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbb140303f0 tx=0x7fbb14030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14037050 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb14043070 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14003bf0 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2419d5e0 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.398+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241a4f20 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.399+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb241032c0 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.400+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbb14030450 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.401+0000 7fbb20ff9640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbb100a3240 0x7fbb100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.401+0000 7fbb22ffd640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbb100a3240 0x7fbb100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.402+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbb140ef6e0 con 0x7fbb24101de0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.402+0000 7fbb22ffd640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbb100a3240 0x7fbb100a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbb240ffe50 tx=0x7fbb0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.406+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb1404c020 con 0x7fbb24101de0 2024-09-19T05:04:00.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.651+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb2410a0f0 con 0x7fbb24101de0 2024-09-19T05:04:00.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.652+0000 7fbb20ff9640 1 -- 172.21.15.64:0/83553276 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7fbb140bc8a0 con 0x7fbb24101de0 2024-09-19T05:04:00.654 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:00.654 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"btime":"2024-09-19T05:03:59:310092+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T05:04:00.654 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-09-19T05:04:00.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.654+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbb100a3240 msgr2=0x7fbb100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.654+0000 7fbb2a079640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbb100a3240 0x7fbb100a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbb240ffe50 tx=0x7fbb0c005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:00.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.654+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 msgr2=0x7fbb241983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.654+0000 7fbb2a079640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 0x7fbb241983a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbb140303f0 tx=0x7fbb14030e00 comp rx=0 tx=0).stop 2024-09-19T05:04:00.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 shutdown_connections 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbb100a3240 0x7fbb100a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb24106f10 0x7fbb2419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24102790 0x7fbb241988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 --2- 172.21.15.64:0/83553276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24101de0 0x7fbb241983a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 >> 172.21.15.64:0/83553276 conn(0x7fbb240fd8c0 msgr2=0x7fbb240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 shutdown_connections 2024-09-19T05:04:00.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:00.655+0000 7fbb2a079640 1 -- 172.21.15.64:0/83553276 wait complete. 2024-09-19T05:04:01.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/83553276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:01 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:01.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T05:04:01.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/83553276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:01 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:01.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/83553276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:01 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:02.244 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:02.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:02 smithi103 ceph-mon[34446]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:02.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:02 smithi064 ceph-mon[30960]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:02 smithi174 ceph-mon[34488]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:02.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 -- 172.21.15.64:0/2486056760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 msgr2=0x7f24f0110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 --2- 172.21.15.64:0/2486056760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 0x7f24f0110910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009ca0 tx=0x7f24e802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 -- 172.21.15.64:0/2486056760 shutdown_connections 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 --2- 172.21.15.64:0/2486056760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 0x7f24f0110910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 --2- 172.21.15.64:0/2486056760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f0101440 0x7f24f010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 --2- 172.21.15.64:0/2486056760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0100b20 0x7f24f0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 -- 172.21.15.64:0/2486056760 >> 172.21.15.64:0/2486056760 conn(0x7f24f00fc7d0 msgr2=0x7f24f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 -- 172.21.15.64:0/2486056760 shutdown_connections 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.725+0000 7f24f5525640 1 -- 172.21.15.64:0/2486056760 wait complete. 2024-09-19T05:04:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 Processor -- start 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 -- start start 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f0100b20 0x7f24f019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 0x7f24f019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 0x7f24f019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0114180 con 0x7f24f0101440 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0114000 con 0x7f24f010e500 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.726+0000 7f24f5525640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0114300 con 0x7f24f0100b20 2024-09-19T05:04:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 0x7f24f019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 0x7f24f019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35780/0 (socket says 172.21.15.64:35780) 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 -- 172.21.15.64:0/2889820503 learned_addr learned my addr 172.21.15.64:0/2889820503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ef7fe640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 0x7f24f019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24eeffd640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f0100b20 0x7f24f019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 -- 172.21.15.64:0/2889820503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f0100b20 msgr2=0x7f24f019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f0100b20 0x7f24f019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 -- 172.21.15.64:0/2889820503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 msgr2=0x7f24f019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 0x7f24f019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 -- 172.21.15.64:0/2889820503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f019f700 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24ee7fc640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 0x7f24f019ac90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24e400b450 tx=0x7f24e400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4015020 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e4011d40 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f019f9f0 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e400f340 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.727+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a72d0 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.728+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0100f00 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.729+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f24e4018020 con 0x7f24f0101440 2024-09-19T05:04:02.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.730+0000 7f24d7fff640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f24d80a3240 0x7f24d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.730+0000 7f24eeffd640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f24d80a3240 0x7f24d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.730+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f24e40ceeb0 con 0x7f24f0101440 2024-09-19T05:04:02.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.731+0000 7f24eeffd640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f24d80a3240 0x7f24d80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24e00095f0 tx=0x7f24e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:02.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.735+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f24e401f020 con 0x7f24f0101440 2024-09-19T05:04:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.925+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f24f0109600 con 0x7f24f0101440 2024-09-19T05:04:02.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.925+0000 7f24d7fff640 1 -- 172.21.15.64:0/2889820503 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f24e400d040 con 0x7f24f0101440 2024-09-19T05:04:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:02.927 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-09-19T05:04:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:04:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-09-19T05:04:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-09-19T05:04:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:04:02.927 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f24d80a3240 msgr2=0x7f24d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f24d80a3240 0x7f24d80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24e00095f0 tx=0x7f24e0005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 msgr2=0x7f24f019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 0x7f24f019ac90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24e400b450 tx=0x7f24e400b910 comp rx=0 tx=0).stop 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 shutdown_connections 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f24d80a3240 0x7f24d80a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f010e500 0x7f24f019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24f0101440 0x7f24f019ac90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 --2- 172.21.15.64:0/2889820503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f0100b20 0x7f24f019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.927+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 >> 172.21.15.64:0/2889820503 conn(0x7f24f00fc7d0 msgr2=0x7f24f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:02.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.928+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 shutdown_connections 2024-09-19T05:04:02.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:02.928+0000 7f24f5525640 1 -- 172.21.15.64:0/2889820503 wait complete. 2024-09-19T05:04:03.567 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T05:04:03.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T05:04:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2889820503' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:03 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2889820503' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:03 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.823 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2889820503' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:03.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:03.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:03 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:04:04.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 -- 172.21.15.64:0/3384386077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 msgr2=0x7f3a2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/3384386077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2410ea90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009ca0 tx=0x7f3a1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 -- 172.21.15.64:0/3384386077 shutdown_connections 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/3384386077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2410ea90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/3384386077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a24102790 0x7f3a241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/3384386077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24101de0 0x7f3a241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 -- 172.21.15.64:0/3384386077 >> 172.21.15.64:0/3384386077 conn(0x7f3a240fd8c0 msgr2=0x7f3a240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 -- 172.21.15.64:0/3384386077 shutdown_connections 2024-09-19T05:04:04.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.262+0000 7f3a28d6b640 1 -- 172.21.15.64:0/3384386077 wait complete. 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 Processor -- start 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 -- start start 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24101de0 0x7f3a241983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a24102790 0x7f3a241988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24111e20 con 0x7f3a24106f10 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24111ca0 con 0x7f3a24102790 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a28d6b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24111fa0 con 0x7f3a24101de0 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a22d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a22d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36272/0 (socket says 172.21.15.64:36272) 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a22d9d640 1 -- 172.21.15.64:0/1825636758 learned_addr learned my addr 172.21.15.64:0/1825636758 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.263+0000 7f3a21d9b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a24102790 0x7f3a241988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a2259c640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24101de0 0x7f3a241983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a22d9d640 1 -- 172.21.15.64:0/1825636758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24101de0 msgr2=0x7f3a241983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a22d9d640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24101de0 0x7f3a241983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a22d9d640 1 -- 172.21.15.64:0/1825636758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a24102790 msgr2=0x7f3a241988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a22d9d640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a24102790 0x7f3a241988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a22d9d640 1 -- 172.21.15.64:0/1825636758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2419d350 con 0x7f3a24106f10 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a22d9d640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2419cc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009dc0 tx=0x7f3a1c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003cf0 con 0x7f3a24106f10 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a1c0044f0 con 0x7f3a24106f10 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c030bf0 con 0x7f3a24106f10 2024-09-19T05:04:04.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2419d5e0 con 0x7f3a24106f10 2024-09-19T05:04:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.264+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a241a4f20 con 0x7f3a24106f10 2024-09-19T05:04:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.265+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a241021c0 con 0x7f3a24106f10 2024-09-19T05:04:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.266+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3a1c004040 con 0x7f3a24106f10 2024-09-19T05:04:04.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.267+0000 7f3a137fe640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3a0c0a3240 0x7f3a0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.267+0000 7f3a2259c640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3a0c0a3240 0x7f3a0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.267+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f3a1c0f0ab0 con 0x7f3a24106f10 2024-09-19T05:04:04.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.268+0000 7f3a2259c640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3a0c0a3240 0x7f3a0c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a14005fd0 tx=0x7f3a14005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:04.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.271+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a1c0bdc70 con 0x7f3a24106f10 2024-09-19T05:04:04.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.446+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3a2410a150 con 0x7f3a24106f10 2024-09-19T05:04:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.931+0000 7f3a137fe640 1 -- 172.21.15.64:0/1825636758 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f3a1c0dd070 con 0x7f3a24106f10 2024-09-19T05:04:04.933 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-09-19T05:04:04.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3a0c0a3240 msgr2=0x7f3a0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3a0c0a3240 0x7f3a0c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a14005fd0 tx=0x7f3a14005850 comp rx=0 tx=0).stop 2024-09-19T05:04:04.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 msgr2=0x7f3a2419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2419cc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009dc0 tx=0x7f3a1c032ff0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 shutdown_connections 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3a0c0a3240 0x7f3a0c0a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a24106f10 0x7f3a2419cc70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a24102790 0x7f3a241988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 --2- 172.21.15.64:0/1825636758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24101de0 0x7f3a241983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 >> 172.21.15.64:0/1825636758 conn(0x7f3a240fd8c0 msgr2=0x7f3a240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:04.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.934+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 shutdown_connections 2024-09-19T05:04:04.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:04.935+0000 7f3a28d6b640 1 -- 172.21.15.64:0/1825636758 wait complete. 2024-09-19T05:04:05.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:04 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1825636758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:04:05.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:04 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1825636758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:04:05.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:04 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1825636758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:04:05.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T05:04:05.862 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:06.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:05 smithi064 ceph-mon[30960]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:06.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:05 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1825636758' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:04:06.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:05 smithi064 ceph-mon[30960]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T05:04:06.217 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:05 smithi064 ceph-mon[30960]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T05:04:06.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:05 smithi103 ceph-mon[34446]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:06.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:05 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1825636758' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:04:06.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:05 smithi103 ceph-mon[34446]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T05:04:06.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:05 smithi103 ceph-mon[34446]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T05:04:06.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:05 smithi174 ceph-mon[34488]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:06.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:05 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1825636758' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:04:06.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:05 smithi174 ceph-mon[34488]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T05:04:06.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:05 smithi174 ceph-mon[34488]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T05:04:06.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 -- 172.21.15.64:0/1533022033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 msgr2=0x7f54f0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 --2- 172.21.15.64:0/1533022033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0107b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009930 tx=0x7f54e402a420 comp rx=0 tx=0).stop 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 -- 172.21.15.64:0/1533022033 shutdown_connections 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 --2- 172.21.15.64:0/1533022033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54f01082a0 0x7f54f010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 --2- 172.21.15.64:0/1533022033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0107b90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 --2- 172.21.15.64:0/1533022033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f0102d30 0x7f54f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 -- 172.21.15.64:0/1533022033 >> 172.21.15.64:0/1533022033 conn(0x7f54f00fc7d0 msgr2=0x7f54f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 -- 172.21.15.64:0/1533022033 shutdown_connections 2024-09-19T05:04:06.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 -- 172.21.15.64:0/1533022033 wait complete. 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.305+0000 7f54f78bf640 1 Processor -- start 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 -- start start 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54f0102d30 0x7f54f0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01082a0 0x7f54f019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0111f70 con 0x7f54f01036e0 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0111df0 con 0x7f54f0102d30 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f78bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01120f0 con 0x7f54f01082a0 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36290/0 (socket says 172.21.15.64:36290) 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 -- 172.21.15.64:0/856895917 learned_addr learned my addr 172.21.15.64:0/856895917 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f565b640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54f0102d30 0x7f54f0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f5e5c640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01082a0 0x7f54f019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 -- 172.21.15.64:0/856895917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01082a0 msgr2=0x7f54f019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01082a0 0x7f54f019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 -- 172.21.15.64:0/856895917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54f0102d30 msgr2=0x7f54f0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54f0102d30 0x7f54f0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f4e5a640 1 -- 172.21.15.64:0/856895917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f019d540 con 0x7f54f01036e0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.306+0000 7f54f5e5c640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01082a0 0x7f54f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.307+0000 7f54f4e5a640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0198ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009230 tx=0x7f54e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.307+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e403f030 con 0x7f54f01036e0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.307+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f019d7d0 con 0x7f54f01036e0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.307+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40035f0 con 0x7f54f01036e0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.307+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4035070 con 0x7f54f01036e0 2024-09-19T05:04:06.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.307+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a5110 con 0x7f54f01036e0 2024-09-19T05:04:06.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.308+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f0103110 con 0x7f54f01036e0 2024-09-19T05:04:06.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.309+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f54e4003aa0 con 0x7f54f01036e0 2024-09-19T05:04:06.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.310+0000 7f54ee7fc640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f54d80a3240 0x7f54d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.310+0000 7f54f565b640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f54d80a3240 0x7f54d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.310+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f54e40effa0 con 0x7f54f01036e0 2024-09-19T05:04:06.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.310+0000 7f54f565b640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f54d80a3240 0x7f54d80a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009990 tx=0x7f54e8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:06.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.314+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f54e40bcff0 con 0x7f54f01036e0 2024-09-19T05:04:06.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.491+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f54f010a2b0 con 0x7f54f01036e0 2024-09-19T05:04:06.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.952+0000 7f54ee7fc640 1 -- 172.21.15.64:0/856895917 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f54e40bc8f0 con 0x7f54f01036e0 2024-09-19T05:04:06.954 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f54d80a3240 msgr2=0x7f54d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f54d80a3240 0x7f54d80a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009990 tx=0x7f54e8009110 comp rx=0 tx=0).stop 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 msgr2=0x7f54f0198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0198ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009230 tx=0x7f54e4004270 comp rx=0 tx=0).stop 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 shutdown_connections 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f54d80a3240 0x7f54d80a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.954+0000 7f54f78bf640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01082a0 0x7f54f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.955+0000 7f54f78bf640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f01036e0 0x7f54f0198ad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.955+0000 7f54f78bf640 1 --2- 172.21.15.64:0/856895917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54f0102d30 0x7f54f0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.955+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 >> 172.21.15.64:0/856895917 conn(0x7f54f00fc7d0 msgr2=0x7f54f0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.955+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 shutdown_connections 2024-09-19T05:04:06.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:06.955+0000 7f54f78bf640 1 -- 172.21.15.64:0/856895917 wait complete. 2024-09-19T05:04:07.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:06 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/856895917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:04:07.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:06 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/856895917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:04:07.304 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:06 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/856895917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:04:07.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T05:04:07.887 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:08.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:07 smithi103 ceph-mon[34446]: pgmap v268: 33 pgs: 18 creating+peering, 14 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:08.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:07 smithi103 ceph-mon[34446]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:04:08.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:07 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/856895917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:04:08.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:07 smithi103 ceph-mon[34446]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T05:04:08.224 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:07 smithi064 ceph-mon[30960]: pgmap v268: 33 pgs: 18 creating+peering, 14 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:08.225 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:07 smithi064 ceph-mon[30960]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:04:08.225 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:07 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/856895917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:04:08.225 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:07 smithi064 ceph-mon[30960]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T05:04:08.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:07 smithi174 ceph-mon[34488]: pgmap v268: 33 pgs: 18 creating+peering, 14 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:08.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:07 smithi174 ceph-mon[34488]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:04:08.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:07 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/856895917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:04:08.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:07 smithi174 ceph-mon[34488]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T05:04:08.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 -- 172.21.15.64:0/3574435286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c102d30 msgr2=0x7f8b0c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3574435286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c102d30 0x7f8b0c103110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009930 tx=0x7f8b0402a440 comp rx=0 tx=0).stop 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 -- 172.21.15.64:0/3574435286 shutdown_connections 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3574435286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3574435286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1036e0 0x7f8b0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3574435286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c102d30 0x7f8b0c103110 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 -- 172.21.15.64:0/3574435286 >> 172.21.15.64:0/3574435286 conn(0x7f8b0c0fc7d0 msgr2=0x7f8b0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 -- 172.21.15.64:0/3574435286 shutdown_connections 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.308+0000 7f8b10e71640 1 -- 172.21.15.64:0/3574435286 wait complete. 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 Processor -- start 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 -- start start 2024-09-19T05:04:08.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c102d30 0x7f8b0c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1036e0 0x7f8b0c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c111f90 con 0x7f8b0c1082a0 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c111e10 con 0x7f8b0c1036e0 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b10e71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c112110 con 0x7f8b0c102d30 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36310/0 (socket says 172.21.15.64:36310) 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 -- 172.21.15.64:0/3503978633 learned_addr learned my addr 172.21.15.64:0/3503978633 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0a59c640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c102d30 0x7f8b0c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b09d9b640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1036e0 0x7f8b0c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 -- 172.21.15.64:0/3503978633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c102d30 msgr2=0x7f8b0c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c102d30 0x7f8b0c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 -- 172.21.15.64:0/3503978633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1036e0 msgr2=0x7f8b0c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1036e0 0x7f8b0c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b0ad9d640 1 -- 172.21.15.64:0/3503978633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c19d500 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.309+0000 7f8b09d9b640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1036e0 0x7f8b0c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.310+0000 7f8b0ad9d640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c19ce20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00eaa0 tx=0x7f8afc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.310+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00cdb0 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.310+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8afc005020 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.310+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c19d7f0 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.310+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0116d0 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.310+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a50d0 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.311+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0c06a850 con 0x7f8b0c1082a0 2024-09-19T05:04:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.312+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8afc0051d0 con 0x7f8b0c1082a0 2024-09-19T05:04:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.313+0000 7f8b037fe640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8af80a32c0 0x7f8af80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.314+0000 7f8b0a59c640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8af80a32c0 0x7f8af80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.314+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7879+0+0 (secure 0 0 0) 0x7f8afc0cfcd0 con 0x7f8b0c1082a0 2024-09-19T05:04:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.314+0000 7f8b0a59c640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8af80a32c0 0x7f8af80a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009900 tx=0x7f8b04000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.317+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8afc016030 con 0x7f8b0c1082a0 2024-09-19T05:04:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.533+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8b0c103110 con 0x7f8b0c1082a0 2024-09-19T05:04:08.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.965+0000 7f8b037fe640 1 -- 172.21.15.64:0/3503978633 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8b0c103110 con 0x7f8b0c1082a0 2024-09-19T05:04:08.967 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8af80a32c0 msgr2=0x7f8af80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8af80a32c0 0x7f8af80a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009900 tx=0x7f8b04000bc0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 msgr2=0x7f8b0c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c19ce20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00eaa0 tx=0x7f8afc00ef60 comp rx=0 tx=0).stop 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 shutdown_connections 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f8af80a32c0 0x7f8af80a5780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b0c1082a0 0x7f8b0c19ce20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1036e0 0x7f8b0c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 --2- 172.21.15.64:0/3503978633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c102d30 0x7f8b0c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.967+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 >> 172.21.15.64:0/3503978633 conn(0x7f8b0c0fc7d0 msgr2=0x7f8b0c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.968+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 shutdown_connections 2024-09-19T05:04:08.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:08.968+0000 7f8b10e71640 1 -- 172.21.15.64:0/3503978633 wait complete. 2024-09-19T05:04:09.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:08 smithi174 ceph-mon[34488]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:04:09.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:08 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3503978633' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:04:09.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:08 smithi174 ceph-mon[34488]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T05:04:09.285 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:08 smithi064 ceph-mon[30960]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:04:09.286 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:08 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3503978633' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:04:09.286 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:08 smithi064 ceph-mon[30960]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T05:04:09.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:08 smithi103 ceph-mon[34446]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:04:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:08 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3503978633' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:04:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:08 smithi103 ceph-mon[34446]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T05:04:09.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:09.966 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:10.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:09 smithi103 ceph-mon[34446]: pgmap v271: 97 pgs: 18 creating+peering, 78 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:09 smithi103 ceph-mon[34446]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:04:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:09 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3503978633' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:04:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:09 smithi103 ceph-mon[34446]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T05:04:10.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:09 smithi103 ceph-mon[34446]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T05:04:10.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:09 smithi064 ceph-mon[30960]: pgmap v271: 97 pgs: 18 creating+peering, 78 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:10.226 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:09 smithi064 ceph-mon[30960]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:04:10.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:09 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3503978633' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:04:10.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:09 smithi064 ceph-mon[30960]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T05:04:10.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:09 smithi064 ceph-mon[30960]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T05:04:10.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:09 smithi174 ceph-mon[34488]: pgmap v271: 97 pgs: 18 creating+peering, 78 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:10.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:09 smithi174 ceph-mon[34488]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:04:10.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:09 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3503978633' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:04:10.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:09 smithi174 ceph-mon[34488]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T05:04:10.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:09 smithi174 ceph-mon[34488]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T05:04:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 -- 172.21.15.64:0/3837547026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 msgr2=0x7f5e80106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 --2- 172.21.15.64:0/3837547026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e80106e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009930 tx=0x7f5e7c02a420 comp rx=0 tx=0).stop 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 -- 172.21.15.64:0/3837547026 shutdown_connections 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 --2- 172.21.15.64:0/3837547026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e80101420 0x7f5e8010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 --2- 172.21.15.64:0/3837547026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e80100a80 0x7f5e80100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 --2- 172.21.15.64:0/3837547026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e80106e60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 -- 172.21.15.64:0/3837547026 >> 172.21.15.64:0/3837547026 conn(0x7f5e800fc7a0 msgr2=0x7f5e800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 -- 172.21.15.64:0/3837547026 shutdown_connections 2024-09-19T05:04:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 -- 172.21.15.64:0/3837547026 wait complete. 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 Processor -- start 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.396+0000 7f5e87665640 1 -- start start 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e87665640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e80100a80 0x7f5e80102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e87665640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e80101420 0x7f5e80102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e87665640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e801030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e87665640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80110d80 con 0x7f5e80106a80 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e87665640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80110c00 con 0x7f5e80100a80 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e87665640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80110f00 con 0x7f5e80101420 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e801030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e801030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36326/0 (socket says 172.21.15.64:36326) 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 -- 172.21.15.64:0/1567844096 learned_addr learned my addr 172.21.15.64:0/1567844096 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85401640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e80100a80 0x7f5e80102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e84c00640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e80101420 0x7f5e80102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 -- 172.21.15.64:0/1567844096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e80101420 msgr2=0x7f5e80102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e80101420 0x7f5e80102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 -- 172.21.15.64:0/1567844096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e80100a80 msgr2=0x7f5e80102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e80100a80 0x7f5e80102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 -- 172.21.15.64:0/1567844096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e801a6890 con 0x7f5e80106a80 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e85c02640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e801030d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009fd0 tx=0x7f5e7800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78017070 con 0x7f5e80106a80 2024-09-19T05:04:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.397+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e78003e20 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.398+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e801a6b80 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.398+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78010460 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.398+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801a7070 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.399+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e80100ee0 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.400+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5e78003970 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.400+0000 7f5e767fc640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.401+0000 7f5e85401640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.401+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f5e780cf4a0 con 0x7f5e80106a80 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.401+0000 7f5e85401640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c031440 tx=0x7f5e7c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.403+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5e7809bcc0 con 0x7f5e80106a80 2024-09-19T05:04:10.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.582+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e801a72a0 con 0x7f5e80106a80 2024-09-19T05:04:10.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.583+0000 7f5e767fc640 1 -- 172.21.15.64:0/1567844096 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22293 (secure 0 0 0) 0x7f5e7809ffc0 con 0x7f5e80106a80 2024-09-19T05:04:10.585 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:10.585 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:08.534485+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:10.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5e6c0a3240 msgr2=0x7f5e6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c031440 tx=0x7f5e7c005a20 comp rx=0 tx=0).stop 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 msgr2=0x7f5e801030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e801030d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009fd0 tx=0x7f5e7800b260 comp rx=0 tx=0).stop 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 shutdown_connections 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e80106a80 0x7f5e801030d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e80101420 0x7f5e80102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 --2- 172.21.15.64:0/1567844096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e80100a80 0x7f5e80102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 >> 172.21.15.64:0/1567844096 conn(0x7f5e800fc7a0 msgr2=0x7f5e80104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 shutdown_connections 2024-09-19T05:04:10.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:10.586+0000 7f5e87665640 1 -- 172.21.15.64:0/1567844096 wait complete. 2024-09-19T05:04:11.252 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] up:active 2024-09-19T05:04:11.253 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:10 smithi064 ceph-mon[30960]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:11.253 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:10 smithi064 ceph-mon[30960]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:11.253 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:10 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1567844096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:11.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] up:active 2024-09-19T05:04:11.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:10 smithi174 ceph-mon[34488]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:11.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:10 smithi174 ceph-mon[34488]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:11.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:10 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1567844096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:11.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:11.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] up:active 2024-09-19T05:04:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:10 smithi103 ceph-mon[34446]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:10 smithi103 ceph-mon[34446]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:10 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1567844096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:11.542 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:12.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.020+0000 7f0ccc493640 1 -- 172.21.15.64:0/177081244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4102790 msgr2=0x7f0cc41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.020+0000 7f0ccc493640 1 --2- 172.21.15.64:0/177081244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4102790 0x7f0cc41067e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009900 tx=0x7f0cb402a410 comp rx=0 tx=0).stop 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 -- 172.21.15.64:0/177081244 shutdown_connections 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 --2- 172.21.15.64:0/177081244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cc4106f10 0x7f0cc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 --2- 172.21.15.64:0/177081244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4102790 0x7f0cc41067e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 --2- 172.21.15.64:0/177081244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cc4101de0 0x7f0cc41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 -- 172.21.15.64:0/177081244 >> 172.21.15.64:0/177081244 conn(0x7f0cc40fd8c0 msgr2=0x7f0cc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 -- 172.21.15.64:0/177081244 shutdown_connections 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 -- 172.21.15.64:0/177081244 wait complete. 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 Processor -- start 2024-09-19T05:04:12.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 -- start start 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 0x7f0cc40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cc4102790 0x7f0cc4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cc4106f10 0x7f0cc4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.021+0000 7f0ccc493640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc4111e40 con 0x7f0cc4101de0 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0ccc493640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc4111cc0 con 0x7f0cc4102790 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0ccc493640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc4111fc0 con 0x7f0cc4106f10 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 0x7f0cc40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 0x7f0cc40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36346/0 (socket says 172.21.15.64:36346) 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 -- 172.21.15.64:0/3970359660 learned_addr learned my addr 172.21.15.64:0/3970359660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cc9a2e640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cc4102790 0x7f0cc4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 -- 172.21.15.64:0/3970359660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cc4106f10 msgr2=0x7f0cc4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cc4106f10 0x7f0cc4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 -- 172.21.15.64:0/3970359660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cc4102790 msgr2=0x7f0cc4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cc4102790 0x7f0cc4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 -- 172.21.15.64:0/3970359660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc41a68e0 con 0x7f0cc4101de0 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cca22f640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 0x7f0cc40744f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000b4a0 tx=0x7f0cc000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0014070 con 0x7f0cc4101de0 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc41a6bd0 con 0x7f0cc4101de0 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cc0010ac0 con 0x7f0cc4101de0 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0017c30 con 0x7f0cc4101de0 2024-09-19T05:04:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.022+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc41a7090 con 0x7f0cc4101de0 2024-09-19T05:04:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.024+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc41021c0 con 0x7f0cc4101de0 2024-09-19T05:04:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.025+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0cc0017070 con 0x7f0cc4101de0 2024-09-19T05:04:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.025+0000 7f0cbb7fe640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0cb00a3240 0x7f0cb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.025+0000 7f0cc9a2e640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0cb00a3240 0x7f0cb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.025+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f0cc00ceb70 con 0x7f0cc4101de0 2024-09-19T05:04:12.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.026+0000 7f0cc9a2e640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0cb00a3240 0x7f0cb00a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4031440 tx=0x7f0cb4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.030+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0cc009b9e0 con 0x7f0cc4101de0 2024-09-19T05:04:12.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.205+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cc4075a20 con 0x7f0cc4101de0 2024-09-19T05:04:12.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:11 smithi064 ceph-mon[30960]: pgmap v273: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-09-19T05:04:12.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:11 smithi064 ceph-mon[30960]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:04:12.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:11 smithi064 ceph-mon[30960]: Cluster is now healthy 2024-09-19T05:04:12.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.207+0000 7f0cbb7fe640 1 -- 172.21.15.64:0/3970359660 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22293 (secure 0 0 0) 0x7f0cc009b2e0 con 0x7f0cc4101de0 2024-09-19T05:04:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:12.209 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:08.534485+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0cb00a3240 msgr2=0x7f0cb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0cb00a3240 0x7f0cb00a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4031440 tx=0x7f0cb4005b50 comp rx=0 tx=0).stop 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 msgr2=0x7f0cc40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 0x7f0cc40744f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000b4a0 tx=0x7f0cc000b960 comp rx=0 tx=0).stop 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 shutdown_connections 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0cb00a3240 0x7f0cb00a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.209+0000 7f0ccc493640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cc4106f10 0x7f0cc4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.210+0000 7f0ccc493640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cc4102790 0x7f0cc4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.210+0000 7f0ccc493640 1 --2- 172.21.15.64:0/3970359660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cc4101de0 0x7f0cc40744f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.210+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 >> 172.21.15.64:0/3970359660 conn(0x7f0cc40fd8c0 msgr2=0x7f0cc40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.210+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 shutdown_connections 2024-09-19T05:04:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:12.210+0000 7f0ccc493640 1 -- 172.21.15.64:0/3970359660 wait complete. 2024-09-19T05:04:12.237 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:11 smithi103 ceph-mon[34446]: pgmap v273: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-09-19T05:04:12.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:11 smithi103 ceph-mon[34446]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:04:12.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:11 smithi103 ceph-mon[34446]: Cluster is now healthy 2024-09-19T05:04:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:11 smithi174 ceph-mon[34488]: pgmap v273: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-09-19T05:04:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:11 smithi174 ceph-mon[34488]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:04:12.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:11 smithi174 ceph-mon[34488]: Cluster is now healthy 2024-09-19T05:04:12.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T05:04:13.080 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:13.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:12 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3970359660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:13.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:12 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3970359660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:13.307 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:12 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3970359660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.505+0000 7fc6c69b2640 1 -- 172.21.15.64:0/2739230666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0103660 msgr2=0x7fc6c010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.505+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/2739230666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0103660 0x7fc6c010bb50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009860 tx=0x7fc6a802a3d0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- 172.21.15.64:0/2739230666 shutdown_connections 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/2739230666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/2739230666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0103660 0x7fc6c010bb50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/2739230666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0102d40 0x7fc6c0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- 172.21.15.64:0/2739230666 >> 172.21.15.64:0/2739230666 conn(0x7fc6c00fc7a0 msgr2=0x7fc6c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- 172.21.15.64:0/2739230666 shutdown_connections 2024-09-19T05:04:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- 172.21.15.64:0/2739230666 wait complete. 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 Processor -- start 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- start start 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0102d40 0x7fc6c0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0103660 0x7fc6c01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111e20 con 0x7fc6c010c170 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111ca0 con 0x7fc6c0102d40 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.506+0000 7fc6c69b2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111fa0 con 0x7fc6c0103660 2024-09-19T05:04:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36352/0 (socket says 172.21.15.64:36352) 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 -- 172.21.15.64:0/796654037 learned_addr learned my addr 172.21.15.64:0/796654037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6bffff640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0102d40 0x7fc6c0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6b7fff640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0103660 0x7fc6c01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 -- 172.21.15.64:0/796654037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0103660 msgr2=0x7fc6c01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0103660 0x7fc6c01988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 -- 172.21.15.64:0/796654037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0102d40 msgr2=0x7fc6c0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0102d40 0x7fc6c0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 -- 172.21.15.64:0/796654037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c019d310 con 0x7fc6c010c170 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c4f4f640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c019cc30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009fd0 tx=0x7fc6b800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8004b20 con 0x7fc6c010c170 2024-09-19T05:04:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c019d600 con 0x7fc6c010c170 2024-09-19T05:04:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b8003e20 con 0x7fc6c010c170 2024-09-19T05:04:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8004820 con 0x7fc6c010c170 2024-09-19T05:04:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.507+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01a4ee0 con 0x7fc6c010c170 2024-09-19T05:04:13.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.508+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c006b070 con 0x7fc6c010c170 2024-09-19T05:04:13.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.510+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc6b801d020 con 0x7fc6c010c170 2024-09-19T05:04:13.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.511+0000 7fc6bdffb640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc6ac0a3240 0x7fc6ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:13.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.511+0000 7fc6bffff640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc6ac0a3240 0x7fc6ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:13.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.511+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fc6b80cf7c0 con 0x7fc6c010c170 2024-09-19T05:04:13.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.511+0000 7fc6bffff640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc6ac0a3240 0x7fc6ac0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000bbe0 tx=0x7fc6b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:13.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.515+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6b809c630 con 0x7fc6c010c170 2024-09-19T05:04:13.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.726+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc6c0107240 con 0x7fc6c010c170 2024-09-19T05:04:13.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.982+0000 7fc6bdffb640 1 -- 172.21.15.64:0/796654037 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc6b809bf30 con 0x7fc6c010c170 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc6ac0a3240 msgr2=0x7fc6ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc6ac0a3240 0x7fc6ac0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000bbe0 tx=0x7fc6b000b390 comp rx=0 tx=0).stop 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 msgr2=0x7fc6c019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c019cc30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009fd0 tx=0x7fc6b800c800 comp rx=0 tx=0).stop 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 shutdown_connections 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc6ac0a3240 0x7fc6ac0a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6c010c170 0x7fc6c019cc30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0103660 0x7fc6c01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 --2- 172.21.15.64:0/796654037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6c0102d40 0x7fc6c0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.984+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 >> 172.21.15.64:0/796654037 conn(0x7fc6c00fc7a0 msgr2=0x7fc6c00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.985+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 shutdown_connections 2024-09-19T05:04:13.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:13.985+0000 7fc6c69b2640 1 -- 172.21.15.64:0/796654037 wait complete. 2024-09-19T05:04:14.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:13 smithi174 ceph-mon[34488]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T05:04:14.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:13 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/796654037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:04:14.283 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:13 smithi064 ceph-mon[30960]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T05:04:14.283 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:13 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/796654037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:04:14.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:13 smithi103 ceph-mon[34446]: pgmap v274: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T05:04:14.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:13 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/796654037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:04:14.706 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-19T05:04:14.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-19T05:04:14.718 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:04:14 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: 2024-09-19T05:04:14.322+0000 7ff5cad6b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T05:04:14.966 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:15.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:14 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/796654037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:04:15.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:14 smithi064 ceph-mon[30960]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:15.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:14 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/796654037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:04:15.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:14 smithi174 ceph-mon[34488]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:14 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/796654037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:04:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:14 smithi103 ceph-mon[34446]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 -- 172.21.15.64:0/2892572940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40101420 msgr2=0x7fdc4010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 --2- 172.21.15.64:0/2892572940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40101420 0x7fdc4010d950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009cd0 tx=0x7fdc3802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 -- 172.21.15.64:0/2892572940 shutdown_connections 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 --2- 172.21.15.64:0/2892572940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40101420 0x7fdc4010d950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 --2- 172.21.15.64:0/2892572940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc40100a80 0x7fdc40100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 --2- 172.21.15.64:0/2892572940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40106a80 0x7fdc40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 -- 172.21.15.64:0/2892572940 >> 172.21.15.64:0/2892572940 conn(0x7fdc400fc7a0 msgr2=0x7fdc400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 -- 172.21.15.64:0/2892572940 shutdown_connections 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.462+0000 7fdc47445640 1 -- 172.21.15.64:0/2892572940 wait complete. 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 Processor -- start 2024-09-19T05:04:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 -- start start 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 0x7fdc40197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40101420 0x7fdc40197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc40106a80 0x7fdc4019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40110d50 con 0x7fdc40100a80 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40110bd0 con 0x7fdc40106a80 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc47445640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40110ed0 con 0x7fdc40101420 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc451e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 0x7fdc40197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc451e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 0x7fdc40197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36578/0 (socket says 172.21.15.64:36578) 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc451e1640 1 -- 172.21.15.64:0/2221611673 learned_addr learned my addr 172.21.15.64:0/2221611673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.463+0000 7fdc459e2640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc40106a80 0x7fdc4019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc449e0640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40101420 0x7fdc40197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc451e1640 1 -- 172.21.15.64:0/2221611673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40101420 msgr2=0x7fdc40197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc451e1640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40101420 0x7fdc40197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc451e1640 1 -- 172.21.15.64:0/2221611673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc40106a80 msgr2=0x7fdc4019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc451e1640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc40106a80 0x7fdc4019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc451e1640 1 -- 172.21.15.64:0/2221611673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4019c1e0 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc449e0640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40101420 0x7fdc40197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc451e1640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 0x7fdc40197230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00b4a0 tx=0x7fdc3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c014070 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc3c010ac0 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4019c4d0 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c017c30 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.464+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401a3db0 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.465+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc40100ee0 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.467+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdc3c017070 con 0x7fdc40100a80 2024-09-19T05:04:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.468+0000 7fdc367fc640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdc2c0a3240 0x7fdc2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.468+0000 7fdc449e0640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdc2c0a3240 0x7fdc2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.468+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7fdc3c0cef90 con 0x7fdc40100a80 2024-09-19T05:04:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.468+0000 7fdc449e0640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdc2c0a3240 0x7fdc2c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc400fed50 tx=0x7fdc30005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.471+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc3c09bde0 con 0x7fdc40100a80 2024-09-19T05:04:15.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.688+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fdc4002d7c0 con 0x7fdc40100a80 2024-09-19T05:04:15.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.997+0000 7fdc367fc640 1 -- 172.21.15.64:0/2221611673 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fdc3c09b6e0 con 0x7fdc40100a80 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.999+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdc2c0a3240 msgr2=0x7fdc2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.999+0000 7fdc47445640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdc2c0a3240 0x7fdc2c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc400fed50 tx=0x7fdc30005d90 comp rx=0 tx=0).stop 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.999+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 msgr2=0x7fdc40197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:15.999+0000 7fdc47445640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 0x7fdc40197230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00b4a0 tx=0x7fdc3c00b960 comp rx=0 tx=0).stop 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 shutdown_connections 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdc2c0a3240 0x7fdc2c0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc40106a80 0x7fdc4019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc40101420 0x7fdc40197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 --2- 172.21.15.64:0/2221611673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc40100a80 0x7fdc40197230 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 >> 172.21.15.64:0/2221611673 conn(0x7fdc400fc7a0 msgr2=0x7fdc400fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:16.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 shutdown_connections 2024-09-19T05:04:16.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:16.000+0000 7fdc47445640 1 -- 172.21.15.64:0/2221611673 wait complete. 2024-09-19T05:04:16.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:15 smithi174 ceph-mon[34488]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:04:16.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:15 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2221611673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T05:04:16.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:15 smithi064 ceph-mon[30960]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:04:16.296 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:15 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2221611673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T05:04:16.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:15 smithi103 ceph-mon[34446]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:04:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:15 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2221611673' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T05:04:16.645 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T05:04:16.911 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:17.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:16 smithi174 ceph-mon[34488]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:04:17.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:16 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2221611673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:16 smithi174 ceph-mon[34488]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:16 smithi174 ceph-mon[34488]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:16 smithi174 ceph-mon[34488]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:16 smithi174 ceph-mon[34488]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:16 smithi064 ceph-mon[30960]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:16 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2221611673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:16 smithi064 ceph-mon[30960]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:04:17.278 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:16 smithi064 ceph-mon[30960]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:17.279 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:16 smithi064 ceph-mon[30960]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T05:04:17.279 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:16 smithi064 ceph-mon[30960]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 -- 172.21.15.64:0/96992335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102570 msgr2=0x7f3b4c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/96992335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102570 0x7f3b4c1029b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02a480 comp rx=0 tx=0).stop 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 -- 172.21.15.64:0/96992335 shutdown_connections 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/96992335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/96992335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102570 0x7f3b4c1029b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/96992335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 0x7f3b4c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 -- 172.21.15.64:0/96992335 >> 172.21.15.64:0/96992335 conn(0x7f3b4c0fc7d0 msgr2=0x7f3b4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 -- 172.21.15.64:0/96992335 shutdown_connections 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.364+0000 7f3b50aa8640 1 -- 172.21.15.64:0/96992335 wait complete. 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 Processor -- start 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 -- start start 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 0x7f3b4c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c102570 0x7f3b4c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c110e50 con 0x7f3b4c102ef0 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c110cd0 con 0x7f3b4c100e20 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b50aa8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c110fd0 con 0x7f3b4c102570 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36598/0 (socket says 172.21.15.64:36598) 2024-09-19T05:04:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 -- 172.21.15.64:0/4256463723 learned_addr learned my addr 172.21.15.64:0/4256463723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b49d9b640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c102570 0x7f3b4c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4a59c640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 0x7f3b4c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 -- 172.21.15.64:0/4256463723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c102570 msgr2=0x7f3b4c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c102570 0x7f3b4c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 -- 172.21.15.64:0/4256463723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 msgr2=0x7f3b4c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 0x7f3b4c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4ad9d640 1 -- 172.21.15.64:0/4256463723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1a6920 con 0x7f3b4c102ef0 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.365+0000 7f3b4a59c640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 0x7f3b4c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.366+0000 7f3b4ad9d640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c0fd700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b970 tx=0x7f3b4400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.366+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44019070 con 0x7f3b4c102ef0 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.366+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b44014440 con 0x7f3b4c102ef0 2024-09-19T05:04:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.366+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1a6c10 con 0x7f3b4c102ef0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.366+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440075d0 con 0x7f3b4c102ef0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.366+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a7170 con 0x7f3b4c102ef0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.367+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c101200 con 0x7f3b4c102ef0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.368+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3b4401c070 con 0x7f3b4c102ef0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.369+0000 7f3b437fe640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3b380a32c0 0x7f3b380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.369+0000 7f3b4a59c640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3b380a32c0 0x7f3b380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.369+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7979+0+0 (secure 0 0 0) 0x7f3b440cf180 con 0x7f3b4c102ef0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.369+0000 7f3b4a59c640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3b380a32c0 0x7f3b380a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009b40 tx=0x7f3b34005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.372+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3b4409b9a0 con 0x7f3b4c102ef0 2024-09-19T05:04:17.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:16 smithi103 ceph-mon[34446]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:04:17.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:16 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2221611673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T05:04:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:16 smithi103 ceph-mon[34446]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:04:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:16 smithi103 ceph-mon[34446]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:04:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:16 smithi103 ceph-mon[34446]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T05:04:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:16 smithi103 ceph-mon[34446]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T05:04:17.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:17.586+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3b4c10a0a0 con 0x7f3b4c102ef0 2024-09-19T05:04:18.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.011+0000 7f3b437fe640 1 -- 172.21.15.64:0/4256463723 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f3b4409fca0 con 0x7f3b4c102ef0 2024-09-19T05:04:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3b380a32c0 msgr2=0x7f3b380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3b380a32c0 0x7f3b380a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009b40 tx=0x7f3b34005e50 comp rx=0 tx=0).stop 2024-09-19T05:04:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 msgr2=0x7f3b4c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c0fd700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b970 tx=0x7f3b4400be30 comp rx=0 tx=0).stop 2024-09-19T05:04:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 shutdown_connections 2024-09-19T05:04:18.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f3b380a32c0 0x7f3b380a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b4c102ef0 0x7f3b4c0fd700 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c102570 0x7f3b4c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 --2- 172.21.15.64:0/4256463723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c100e20 0x7f3b4c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 >> 172.21.15.64:0/4256463723 conn(0x7f3b4c0fc7d0 msgr2=0x7f3b4c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 shutdown_connections 2024-09-19T05:04:18.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:18.013+0000 7f3b50aa8640 1 -- 172.21.15.64:0/4256463723 wait complete. 2024-09-19T05:04:18.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T05:04:18.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] up:active 2024-09-19T05:04:18.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:04:18.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:04:18.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: Cluster is now healthy 2024-09-19T05:04:18.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T05:04:18.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T05:04:18.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T05:04:18.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:18 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4256463723' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] up:active 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: Cluster is now healthy 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T05:04:18.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T05:04:18.333 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T05:04:18.333 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:18 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4256463723' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:04:18.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T05:04:18.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] up:active 2024-09-19T05:04:18.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:04:18.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:04:18.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: Cluster is now healthy 2024-09-19T05:04:18.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T05:04:18.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-19T05:04:18.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T05:04:18.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:18 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4256463723' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:04:18.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T05:04:18.913 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4256463723' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] up:active 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T05:04:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-19T05:04:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-19T05:04:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-09-19T05:04:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T05:04:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T05:04:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:19 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T05:04:19.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4256463723' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:04:19.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] up:active 2024-09-19T05:04:19.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T05:04:19.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:04:19.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-09-19T05:04:19.294 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T05:04:19.295 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:19 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T05:04:19.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4256463723' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:04:19.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] up:active 2024-09-19T05:04:19.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-19T05:04:19.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T05:04:19.324 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T05:04:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 -- 172.21.15.64:0/2768588371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 msgr2=0x7f26d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 --2- 172.21.15.64:0/2768588371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d81067e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009900 tx=0x7f26c402a410 comp rx=0 tx=0).stop 2024-09-19T05:04:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 -- 172.21.15.64:0/2768588371 shutdown_connections 2024-09-19T05:04:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 --2- 172.21.15.64:0/2768588371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8106f10 0x7f26d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 --2- 172.21.15.64:0/2768588371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d81067e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 --2- 172.21.15.64:0/2768588371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101de0 0x7f26d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 -- 172.21.15.64:0/2768588371 >> 172.21.15.64:0/2768588371 conn(0x7f26d80fd8c0 msgr2=0x7f26d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 -- 172.21.15.64:0/2768588371 shutdown_connections 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.377+0000 7f26dec68640 1 -- 172.21.15.64:0/2768588371 wait complete. 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 Processor -- start 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 -- start start 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101de0 0x7f26d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8106f10 0x7f26d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8111e20 con 0x7f26d8102790 2024-09-19T05:04:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8111ca0 con 0x7f26d8106f10 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dec68640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8111fa0 con 0x7f26d8101de0 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36620/0 (socket says 172.21.15.64:36620) 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 -- 172.21.15.64:0/1157115668 learned_addr learned my addr 172.21.15.64:0/1157115668 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dca04640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101de0 0x7f26d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26dd205640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8106f10 0x7f26d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 -- 172.21.15.64:0/1157115668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101de0 msgr2=0x7f26d80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101de0 0x7f26d80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 -- 172.21.15.64:0/1157115668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8106f10 msgr2=0x7f26d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.378+0000 7f26d7fff640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8106f10 0x7f26d8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26d7fff640 1 -- 172.21.15.64:0/1157115668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d81a6860 con 0x7f26d8102790 2024-09-19T05:04:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26d7fff640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d8074a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26c4005940 tx=0x7f26c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c403f030 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26c4003aa0 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4035070 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d81a6af0 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.379+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81a6fd0 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.380+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d81021c0 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.381+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f26c4042030 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.382+0000 7f26d5ffb640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f26c80a3240 0x7f26c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.382+0000 7f26dca04640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f26c80a3240 0x7f26c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.382+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8269+0+0 (secure 0 0 0) 0x7f26c40f01d0 con 0x7f26d8102790 2024-09-19T05:04:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.382+0000 7f26dca04640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f26c80a3240 0x7f26c80a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.386+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f26c40bced0 con 0x7f26d8102790 2024-09-19T05:04:19.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:19.599+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f26d810a150 con 0x7f26d8102790 2024-09-19T05:04:19.718 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: 2024-09-19T05:04:19.322+0000 7ff5cad6b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T05:04:19.719 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:04:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: 2024-09-19T05:04:19.322+0000 7ff5cad6b640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T05:04:20.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.030+0000 7f26d5ffb640 1 -- 172.21.15.64:0/1157115668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f26c40bc7d0 con 0x7f26d8102790 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f26c80a3240 msgr2=0x7f26c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f26c80a3240 0x7f26c80a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005e20 comp rx=0 tx=0).stop 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 msgr2=0x7f26d8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d8074a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f26c4005940 tx=0x7f26c4004270 comp rx=0 tx=0).stop 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 shutdown_connections 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f26c80a3240 0x7f26c80a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d8106f10 0x7f26d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d8102790 0x7f26d8074a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 --2- 172.21.15.64:0/1157115668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101de0 0x7f26d80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.032+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 >> 172.21.15.64:0/1157115668 conn(0x7f26d80fd8c0 msgr2=0x7f26d80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:20.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.033+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 shutdown_connections 2024-09-19T05:04:20.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:20.033+0000 7f26dec68640 1 -- 172.21.15.64:0/1157115668 wait complete. 2024-09-19T05:04:20.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-19T05:04:20.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-09-19T05:04:20.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T05:04:20.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T05:04:20.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:04:20.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:20 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1157115668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T05:04:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T05:04:20.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:04:20.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:20 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1157115668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:04:20.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-19T05:04:20.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T05:04:20.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T05:04:20.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T05:04:20.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:04:20.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:20 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1157115668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:04:20.704 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T05:04:20.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T05:04:20.964 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:21.352 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1157115668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:04:21.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-mon[30960]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:04:21.353 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-mon[30960]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 -- 172.21.15.64:0/3472166224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c102ef0 msgr2=0x7fee0c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 --2- 172.21.15.64:0/3472166224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c102ef0 0x7fee0c10da40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee04009ca0 tx=0x7fee0402a950 comp rx=0 tx=0).stop 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 -- 172.21.15.64:0/3472166224 shutdown_connections 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 --2- 172.21.15.64:0/3472166224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c102ef0 0x7fee0c10da40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 --2- 172.21.15.64:0/3472166224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c102570 0x7fee0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 --2- 172.21.15.64:0/3472166224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c100e20 0x7fee0c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 -- 172.21.15.64:0/3472166224 >> 172.21.15.64:0/3472166224 conn(0x7fee0c0fc7d0 msgr2=0x7fee0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 -- 172.21.15.64:0/3472166224 shutdown_connections 2024-09-19T05:04:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.435+0000 7fee11898640 1 -- 172.21.15.64:0/3472166224 wait complete. 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 Processor -- start 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 -- start start 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 0x7fee0c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c102570 0x7fee0c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c102ef0 0x7fee0c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c110e50 con 0x7fee0c100e20 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c110cd0 con 0x7fee0c102570 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee11898640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c110fd0 con 0x7fee0c102ef0 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 0x7fee0c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 0x7fee0c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36630/0 (socket says 172.21.15.64:36630) 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 -- 172.21.15.64:0/2385842860 learned_addr learned my addr 172.21.15.64:0/2385842860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0a7fc640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c102570 0x7fee0c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:21.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0b7fe640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c102ef0 0x7fee0c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 -- 172.21.15.64:0/2385842860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c102ef0 msgr2=0x7fee0c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c102ef0 0x7fee0c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 -- 172.21.15.64:0/2385842860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c102570 msgr2=0x7fee0c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c102570 0x7fee0c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.436+0000 7fee0affd640 1 -- 172.21.15.64:0/2385842860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c19d390 con 0x7fee0c100e20 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee0a7fc640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c102570 0x7fee0c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee0affd640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 0x7fee0c1983e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d950 tx=0x7fedfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc017070 con 0x7fee0c100e20 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedfc009c30 con 0x7fee0c100e20 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c19d680 con 0x7fee0c100e20 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00f620 con 0x7fee0c100e20 2024-09-19T05:04:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.437+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1a4f60 con 0x7fee0c100e20 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.438+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c101200 con 0x7fee0c100e20 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.440+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fedfc0109a0 con 0x7fee0c100e20 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.441+0000 7fee10896640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf40a3240 0x7fedf40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.441+0000 7fee0a7fc640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf40a3240 0x7fedf40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.441+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fedfc0cf5f0 con 0x7fee0c100e20 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.441+0000 7fee0a7fc640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf40a3240 0x7fedf40a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0ff060 tx=0x7fee00008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.444+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fedfc009de0 con 0x7fee0c100e20 2024-09-19T05:04:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:21 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1157115668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:04:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:21 smithi103 ceph-mon[34446]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:04:21.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:21 smithi103 ceph-mon[34446]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:04:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:21 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1157115668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:04:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:21 smithi174 ceph-mon[34488]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:04:21.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:21 smithi174 ceph-mon[34488]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:04:21.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.643+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fee0c062630 con 0x7fedf40a3240 2024-09-19T05:04:21.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.668+0000 7fee10896640 1 -- 172.21.15.64:0/2385842860 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee0c062630 con 0x7fedf40a3240 2024-09-19T05:04:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf40a3240 msgr2=0x7fedf40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf40a3240 0x7fedf40a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0ff060 tx=0x7fee00008040 comp rx=0 tx=0).stop 2024-09-19T05:04:21.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 msgr2=0x7fee0c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 0x7fee0c1983e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d950 tx=0x7fedfc00de10 comp rx=0 tx=0).stop 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 shutdown_connections 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fedf40a3240 0x7fedf40a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0c102ef0 0x7fee0c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee0c102570 0x7fee0c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 --2- 172.21.15.64:0/2385842860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee0c100e20 0x7fee0c1983e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 >> 172.21.15.64:0/2385842860 conn(0x7fee0c0fc7d0 msgr2=0x7fee0c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 shutdown_connections 2024-09-19T05:04:21.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:21.670+0000 7fee11898640 1 -- 172.21.15.64:0/2385842860 wait complete. 2024-09-19T05:04:21.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:04:21.648+0000 7f851442e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:21.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:04:21.648+0000 7f851442e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:21.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:04:21.648+0000 7f851442e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:21.901 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:04:21.648+0000 7f851442e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:21.901 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:04:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:04:21.648+0000 7f851442e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:22.258 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:22 smithi064 ceph-mon[30960]: pgmap v280: 97 pgs: 6 peering, 91 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T05:04:22.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T05:04:22.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:22 smithi103 ceph-mon[34446]: pgmap v280: 97 pgs: 6 peering, 91 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T05:04:22.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:22 smithi174 ceph-mon[34488]: pgmap v280: 97 pgs: 6 peering, 91 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-09-19T05:04:22.543 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/3850375696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 msgr2=0x7f6b28106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/3850375696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b28106840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009900 tx=0x7f6b1c02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/3850375696 shutdown_connections 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/3850375696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28106f70 0x7f6b2810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/3850375696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b28106840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/3850375696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b28100500 0x7f6b281008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/3850375696 >> 172.21.15.64:0/3850375696 conn(0x7f6b280fc5c0 msgr2=0x7f6b280fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/3850375696 shutdown_connections 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.035+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/3850375696 wait complete. 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 Processor -- start 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 -- start start 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100500 0x7f6b28198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b281988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b28106f70 0x7f6b2819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111e40 con 0x7f6b28100eb0 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111cc0 con 0x7f6b28100500 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b2e7c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111fc0 con 0x7f6b28106f70 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b281988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b281988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36670/0 (socket says 172.21.15.64:36670) 2024-09-19T05:04:23.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.036+0000 7f6b277fe640 1 -- 172.21.15.64:0/2750371316 learned_addr learned my addr 172.21.15.64:0/2750371316 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b27fff640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100500 0x7f6b28198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b2cd66640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b28106f70 0x7f6b2819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b277fe640 1 -- 172.21.15.64:0/2750371316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b28106f70 msgr2=0x7f6b2819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b277fe640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b28106f70 0x7f6b2819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b277fe640 1 -- 172.21.15.64:0/2750371316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100500 msgr2=0x7f6b28198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b277fe640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100500 0x7f6b28198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b277fe640 1 -- 172.21.15.64:0/2750371316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2819d330 con 0x7f6b28100eb0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b277fe640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b281988c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c02a930 tx=0x7f6b1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c03f030 con 0x7f6b28100eb0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b1c004490 con 0x7f6b28100eb0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2819d5c0 con 0x7f6b28100eb0 2024-09-19T05:04:23.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c035070 con 0x7f6b28100eb0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.037+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a4f00 con 0x7f6b28100eb0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.038+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2806a4a0 con 0x7f6b28100eb0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.040+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6b1c042030 con 0x7f6b28100eb0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.040+0000 7f6b257fa640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6b100a32c0 0x7f6b100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.040+0000 7f6b27fff640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6b100a32c0 0x7f6b100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.040+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f6b1c0f0000 con 0x7f6b28100eb0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.041+0000 7f6b27fff640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6b100a32c0 0x7f6b100a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b28069fc0 tx=0x7f6b1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:23.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.044+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b1c0bc750 con 0x7f6b28100eb0 2024-09-19T05:04:23.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.061+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6b1c0b8dc0 con 0x7f6b28100eb0 2024-09-19T05:04:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.254+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6b2810a110 con 0x7f6b100a32c0 2024-09-19T05:04:23.256 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:23 smithi064 ceph-mon[30960]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:23.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.279+0000 7f6b257fa640 1 -- 172.21.15.64:0/2750371316 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b2810a110 con 0x7f6b100a32c0 2024-09-19T05:04:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6b100a32c0 msgr2=0x7f6b100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6b100a32c0 0x7f6b100a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b28069fc0 tx=0x7f6b1800b390 comp rx=0 tx=0).stop 2024-09-19T05:04:23.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 msgr2=0x7f6b281988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b281988c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c02a930 tx=0x7f6b1c004270 comp rx=0 tx=0).stop 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 shutdown_connections 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6b100a32c0 0x7f6b100a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b28106f70 0x7f6b2819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b28100eb0 0x7f6b281988c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 --2- 172.21.15.64:0/2750371316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b28100500 0x7f6b28198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 >> 172.21.15.64:0/2750371316 conn(0x7f6b280fc5c0 msgr2=0x7f6b281053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 shutdown_connections 2024-09-19T05:04:23.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:23.281+0000 7f6b2e7c9640 1 -- 172.21.15.64:0/2750371316 wait complete. 2024-09-19T05:04:23.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:23 smithi103 ceph-mon[34446]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:23.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:23 smithi174 ceph-mon[34488]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:23.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T05:04:24.213 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:24.334 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:24 smithi064 ceph-mon[30960]: pgmap v281: 97 pgs: 6 peering, 91 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-19T05:04:24.335 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:24 smithi064 ceph-mon[30960]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:04:24.335 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:24 smithi064 ceph-mon[30960]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:24.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:24 smithi103 ceph-mon[34446]: pgmap v281: 97 pgs: 6 peering, 91 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-19T05:04:24.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:24 smithi103 ceph-mon[34446]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:04:24.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:24 smithi103 ceph-mon[34446]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:24.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:24 smithi174 ceph-mon[34488]: pgmap v281: 97 pgs: 6 peering, 91 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-19T05:04:24.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:24 smithi174 ceph-mon[34488]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:04:24.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:24 smithi174 ceph-mon[34488]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3349671047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4102790 msgr2=0x7f4ad41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3349671047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4102790 0x7f4ad41067e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009900 tx=0x7f4ac802a410 comp rx=0 tx=0).stop 2024-09-19T05:04:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3349671047 shutdown_connections 2024-09-19T05:04:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3349671047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4106f10 0x7f4ad410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3349671047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4102790 0x7f4ad41067e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3349671047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 0x7f4ad41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.719+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3349671047 >> 172.21.15.64:0/3349671047 conn(0x7f4ad40fd8c0 msgr2=0x7f4ad40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.720+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3349671047 shutdown_connections 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.720+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3349671047 wait complete. 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.720+0000 7f4ad8a28640 1 Processor -- start 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.720+0000 7f4ad8a28640 1 -- start start 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.720+0000 7f4ad8a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 0x7f4ad41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.720+0000 7f4ad8a28640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4102790 0x7f4ad4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad8a28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 0x7f4ad419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad8a28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4111e40 con 0x7f4ad4106f10 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad8a28640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4111cc0 con 0x7f4ad4101de0 2024-09-19T05:04:24.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad8a28640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4111fc0 con 0x7f4ad4102790 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 0x7f4ad419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 0x7f4ad419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54536/0 (socket says 172.21.15.64:54536) 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 -- 172.21.15.64:0/3166537634 learned_addr learned my addr 172.21.15.64:0/3166537634 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad259c640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 0x7f4ad41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad1d9b640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4102790 0x7f4ad4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 -- 172.21.15.64:0/3166537634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4102790 msgr2=0x7f4ad4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4102790 0x7f4ad4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 -- 172.21.15.64:0/3166537634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 msgr2=0x7f4ad41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 0x7f4ad41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 -- 172.21.15.64:0/3166537634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad419d370 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad1d9b640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4102790 0x7f4ad4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad259c640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 0x7f4ad41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad2d9d640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 0x7f4ad419cc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00b970 tx=0x7f4acc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc019070 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad419d660 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4acc014440 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0075d0 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.721+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad41a4f40 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.722+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad41021c0 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.723+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4acc01c070 con 0x7f4ad4106f10 2024-09-19T05:04:24.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.724+0000 7f4ac37fe640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4abc0a32d0 0x7f4abc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.725+0000 7f4ad259c640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4abc0a32d0 0x7f4abc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.725+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f4acc0ceff0 con 0x7f4ad4106f10 2024-09-19T05:04:24.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.725+0000 7f4ad259c640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4abc0a32d0 0x7f4abc0a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40058c0 tx=0x7f4ac4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.729+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4acc09bca0 con 0x7f4ad4106f10 2024-09-19T05:04:24.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.931+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4ad4062630 con 0x7f4abc0a32d0 2024-09-19T05:04:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.935+0000 7f4ac37fe640 1 -- 172.21.15.64:0/3166537634 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4ad4062630 con 0x7f4abc0a32d0 2024-09-19T05:04:24.937 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_0/32bf837e-c916-4a18-9e8b-e6b294dbeebe 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4abc0a32d0 msgr2=0x7f4abc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4abc0a32d0 0x7f4abc0a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40058c0 tx=0x7f4ac4005850 comp rx=0 tx=0).stop 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 msgr2=0x7f4ad419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 0x7f4ad419cc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00b970 tx=0x7f4acc00be30 comp rx=0 tx=0).stop 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 shutdown_connections 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4abc0a32d0 0x7f4abc0a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ad4106f10 0x7f4ad419cc90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad4102790 0x7f4ad4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 --2- 172.21.15.64:0/3166537634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad4101de0 0x7f4ad41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 >> 172.21.15.64:0/3166537634 conn(0x7f4ad40fd8c0 msgr2=0x7f4ad40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 shutdown_connections 2024-09-19T05:04:24.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:24.937+0000 7f4ad8a28640 1 -- 172.21.15.64:0/3166537634 wait complete. 2024-09-19T05:04:25.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T05:04:25.860 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:26.241 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:26 smithi064 ceph-mon[30960]: pgmap v282: 97 pgs: 2 peering, 95 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.6 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T05:04:26.241 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:26 smithi064 ceph-mon[30960]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 -- 172.21.15.64:0/1507662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 msgr2=0x7f208c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 --2- 172.21.15.64:0/1507662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10da40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2084009ca0 tx=0x7f208402a950 comp rx=0 tx=0).stop 2024-09-19T05:04:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 -- 172.21.15.64:0/1507662865 shutdown_connections 2024-09-19T05:04:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 --2- 172.21.15.64:0/1507662865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10da40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 --2- 172.21.15.64:0/1507662865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 0x7f208c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 --2- 172.21.15.64:0/1507662865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f208c100e20 0x7f208c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 -- 172.21.15.64:0/1507662865 >> 172.21.15.64:0/1507662865 conn(0x7f208c0fc7d0 msgr2=0x7f208c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 -- 172.21.15.64:0/1507662865 shutdown_connections 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.325+0000 7f2094351640 1 -- 172.21.15.64:0/1507662865 wait complete. 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 Processor -- start 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 -- start start 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f208c100e20 0x7f208c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 0x7f208c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f208c111f50 con 0x7f208c102ef0 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f208c111dd0 con 0x7f208c100e20 2024-09-19T05:04:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f2094351640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1120d0 con 0x7f208c102570 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f20928ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f20928ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54544/0 (socket says 172.21.15.64:54544) 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f20928ee640 1 -- 172.21.15.64:0/3394258184 learned_addr learned my addr 172.21.15.64:0/3394258184 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f20918ec640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 0x7f208c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.326+0000 7f20920ed640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f208c100e20 0x7f208c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20928ee640 1 -- 172.21.15.64:0/3394258184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 msgr2=0x7f208c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20928ee640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 0x7f208c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20928ee640 1 -- 172.21.15.64:0/3394258184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f208c100e20 msgr2=0x7f208c10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20928ee640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f208c100e20 0x7f208c10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20928ee640 1 -- 172.21.15.64:0/3394258184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c1a67f0 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20918ec640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 0x7f208c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20928ee640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10ef50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2084004af0 tx=0x7f208402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084034040 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c1a6a80 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20840308f0 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084004670 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.327+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c1a6f60 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.328+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208c101200 con 0x7f208c102ef0 2024-09-19T05:04:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.329+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2084037020 con 0x7f208c102ef0 2024-09-19T05:04:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.330+0000 7f20837fe640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f20780a32d0 0x7f20780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.330+0000 7f20920ed640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f20780a32d0 0x7f20780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.330+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f20840f0980 con 0x7f208c102ef0 2024-09-19T05:04:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.331+0000 7f20920ed640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f20780a32d0 0x7f20780a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2088009b60 tx=0x7f2088005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.334+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20840bd630 con 0x7f208c102ef0 2024-09-19T05:04:26.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:26 smithi103 ceph-mon[34446]: pgmap v282: 97 pgs: 2 peering, 95 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.6 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T05:04:26.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:26 smithi103 ceph-mon[34446]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:26.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:26 smithi174 ceph-mon[34488]: pgmap v282: 97 pgs: 2 peering, 95 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.6 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T05:04:26.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:26 smithi174 ceph-mon[34488]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:26.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.534+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f208c106f40 con 0x7f20780a32d0 2024-09-19T05:04:26.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.558+0000 7f20837fe640 1 -- 172.21.15.64:0/3394258184 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f208c106f40 con 0x7f20780a32d0 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.559+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f20780a32d0 msgr2=0x7f20780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.559+0000 7f2094351640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f20780a32d0 0x7f20780a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2088009b60 tx=0x7f2088005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.559+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 msgr2=0x7f208c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10ef50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2084004af0 tx=0x7f208402f5d0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 shutdown_connections 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f20780a32d0 0x7f20780a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208c102ef0 0x7f208c10ef50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c102570 0x7f208c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 --2- 172.21.15.64:0/3394258184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f208c100e20 0x7f208c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 >> 172.21.15.64:0/3394258184 conn(0x7f208c0fc7d0 msgr2=0x7f208c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 shutdown_connections 2024-09-19T05:04:26.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:26.560+0000 7f2094351640 1 -- 172.21.15.64:0/3394258184 wait complete. 2024-09-19T05:04:27.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T05:04:27.516 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/3901944695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 msgr2=0x7f2d9810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/3901944695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9810eac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009ca0 tx=0x7f2d8802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/3901944695 shutdown_connections 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/3901944695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9810eac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/3901944695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 0x7f2d98106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/3901944695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d98101e10 0x7f2d981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/3901944695 >> 172.21.15.64:0/3901944695 conn(0x7f2d980fd8d0 msgr2=0x7f2d980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/3901944695 shutdown_connections 2024-09-19T05:04:27.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/3901944695 wait complete. 2024-09-19T05:04:27.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 Processor -- start 2024-09-19T05:04:27.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.983+0000 7f2d9ecc3640 1 -- start start 2024-09-19T05:04:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ecc3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d98101e10 0x7f2d98198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ecc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 0x7f2d981988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ecc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ecc3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98111e70 con 0x7f2d98106f40 2024-09-19T05:04:27.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ecc3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98111cf0 con 0x7f2d98101e10 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ecc3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98111ff0 con 0x7f2d981027c0 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54560/0 (socket says 172.21.15.64:54560) 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 -- 172.21.15.64:0/4210890222 learned_addr learned my addr 172.21.15.64:0/4210890222 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9ca5f640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d98101e10 0x7f2d98198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d97fff640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 0x7f2d981988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 -- 172.21.15.64:0/4210890222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 msgr2=0x7f2d981988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 0x7f2d981988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 -- 172.21.15.64:0/4210890222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d98101e10 msgr2=0x7f2d98198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d98101e10 0x7f2d98198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 -- 172.21.15.64:0/4210890222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9819d320 con 0x7f2d98106f40 2024-09-19T05:04:27.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d97fff640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 0x7f2d981988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.984+0000 7f2d9d260640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9819cc40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009dc0 tx=0x7f2d88003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.985+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004040 con 0x7f2d98106f40 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.985+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d88004690 con 0x7f2d98106f40 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.985+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9819d5b0 con 0x7f2d98106f40 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.985+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88030d90 con 0x7f2d98106f40 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.985+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a4ef0 con 0x7f2d98106f40 2024-09-19T05:04:27.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.986+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d981021f0 con 0x7f2d98106f40 2024-09-19T05:04:27.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.987+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2d880305c0 con 0x7f2d98106f40 2024-09-19T05:04:27.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.988+0000 7f2d95ffb640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2d840a32d0 0x7f2d840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.988+0000 7f2d9ca5f640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2d840a32d0 0x7f2d840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.988+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f2d880f1420 con 0x7f2d98106f40 2024-09-19T05:04:27.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.988+0000 7f2d9ca5f640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2d840a32d0 0x7f2d840a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d900058c0 tx=0x7f2d9000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:27.991+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d880be0d0 con 0x7f2d98106f40 2024-09-19T05:04:28.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:28 smithi064 ceph-mon[30960]: pgmap v283: 97 pgs: 97 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.9 KiB/s wr, 19 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-19T05:04:28.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:28 smithi064 ceph-mon[30960]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.199+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2d98062630 con 0x7f2d840a32d0 2024-09-19T05:04:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.204+0000 7f2d95ffb640 1 -- 172.21.15.64:0/4210890222 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2d98062630 con 0x7f2d840a32d0 2024-09-19T05:04:28.209 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_1/58c2dab3-4279-4d9a-9e54-0cbdefa5b9d2 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2d840a32d0 msgr2=0x7f2d840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2d840a32d0 0x7f2d840a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d900058c0 tx=0x7f2d9000a4c0 comp rx=0 tx=0).stop 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 msgr2=0x7f2d9819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9819cc40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009dc0 tx=0x7f2d88003ba0 comp rx=0 tx=0).stop 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 shutdown_connections 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2d840a32d0 0x7f2d840a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d98106f40 0x7f2d9819cc40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027c0 0x7f2d981988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 --2- 172.21.15.64:0/4210890222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d98101e10 0x7f2d98198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 >> 172.21.15.64:0/4210890222 conn(0x7f2d980fd8d0 msgr2=0x7f2d980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 shutdown_connections 2024-09-19T05:04:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:28.207+0000 7f2d9ecc3640 1 -- 172.21.15.64:0/4210890222 wait complete. 2024-09-19T05:04:28.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:28 smithi103 ceph-mon[34446]: pgmap v283: 97 pgs: 97 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.9 KiB/s wr, 19 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-19T05:04:28.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:28 smithi103 ceph-mon[34446]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:28 smithi174 ceph-mon[34488]: pgmap v283: 97 pgs: 97 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.9 KiB/s wr, 19 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-09-19T05:04:28.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:28 smithi174 ceph-mon[34488]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:28.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:04:29.142 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:29.357 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:29 smithi064 ceph-mon[30960]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:29.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:29 smithi103 ceph-mon[34446]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:29 smithi174 ceph-mon[34488]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.578+0000 7fbdca52d640 1 -- 172.21.15.64:0/3454221657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4106f10 msgr2=0x7fbdc410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.578+0000 7fbdca52d640 1 --2- 172.21.15.64:0/3454221657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4106f10 0x7fbdc410ea90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009ca0 tx=0x7fbdbc02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 -- 172.21.15.64:0/3454221657 shutdown_connections 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 --2- 172.21.15.64:0/3454221657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4106f10 0x7fbdc410ea90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 --2- 172.21.15.64:0/3454221657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc4102790 0x7fbdc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 --2- 172.21.15.64:0/3454221657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdc4101de0 0x7fbdc41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 -- 172.21.15.64:0/3454221657 >> 172.21.15.64:0/3454221657 conn(0x7fbdc40fd8c0 msgr2=0x7fbdc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 -- 172.21.15.64:0/3454221657 shutdown_connections 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 -- 172.21.15.64:0/3454221657 wait complete. 2024-09-19T05:04:29.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 Processor -- start 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 -- start start 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.579+0000 7fbdca52d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 0x7fbdc4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdc4102790 0x7fbdc4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc4106f10 0x7fbdc419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4111e40 con 0x7fbdc4101de0 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4111cc0 con 0x7fbdc4102790 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4111fc0 con 0x7fbdc4106f10 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 0x7fbdc4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 0x7fbdc4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54570/0 (socket says 172.21.15.64:54570) 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 -- 172.21.15.64:0/55198869 learned_addr learned my addr 172.21.15.64:0/55198869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdbbfff640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdc4102790 0x7fbdc4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 -- 172.21.15.64:0/55198869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc4106f10 msgr2=0x7fbdc419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc4106f10 0x7fbdc419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 -- 172.21.15.64:0/55198869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdc4102790 msgr2=0x7fbdc4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdc4102790 0x7fbdc4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 -- 172.21.15.64:0/55198869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc419d3b0 con 0x7fbdc4101de0 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc3fff640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 0x7fbdc4198400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d950 tx=0x7fbdb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4017070 con 0x7fbdc4101de0 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdb4009c30 con 0x7fbdc4101de0 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb400f620 con 0x7fbdc4101de0 2024-09-19T05:04:29.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc419d6a0 con 0x7fbdc4101de0 2024-09-19T05:04:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.580+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41a4f80 con 0x7fbdc4101de0 2024-09-19T05:04:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.582+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc41021c0 con 0x7fbdc4101de0 2024-09-19T05:04:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.583+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbdb40109a0 con 0x7fbdc4101de0 2024-09-19T05:04:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.583+0000 7fbdc1ffb640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbdb00a32d0 0x7fbdb00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.583+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fbdb40cf7a0 con 0x7fbdc4101de0 2024-09-19T05:04:29.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.583+0000 7fbdbbfff640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbdb00a32d0 0x7fbdb00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.584+0000 7fbdbbfff640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbdb00a32d0 0x7fbdb00a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40ffe30 tx=0x7fbdac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.588+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbdb4009de0 con 0x7fbdc4101de0 2024-09-19T05:04:29.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.800+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdc410a1b0 con 0x7fbdc4101de0 2024-09-19T05:04:29.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.801+0000 7fbdc1ffb640 1 -- 172.21.15.64:0/55198869 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7fbdb409c340 con 0x7fbdc4101de0 2024-09-19T05:04:29.802 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:29.803 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-19T05:04:20:028478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:04:19.601259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24448,24466,24479]},"id":1}]} 2024-09-19T05:04:29.803 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:04:29.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbdb00a32d0 msgr2=0x7fbdb00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbdb00a32d0 0x7fbdb00a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40ffe30 tx=0x7fbdac008040 comp rx=0 tx=0).stop 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 msgr2=0x7fbdc4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 0x7fbdc4198400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d950 tx=0x7fbdb400de10 comp rx=0 tx=0).stop 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 shutdown_connections 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fbdb00a32d0 0x7fbdb00a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc4106f10 0x7fbdc419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdc4102790 0x7fbdc4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 --2- 172.21.15.64:0/55198869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdc4101de0 0x7fbdc4198400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 >> 172.21.15.64:0/55198869 conn(0x7fbdc40fd8c0 msgr2=0x7fbdc40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 shutdown_connections 2024-09-19T05:04:29.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:29.803+0000 7fbdca52d640 1 -- 172.21.15.64:0/55198869 wait complete. 2024-09-19T05:04:30.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:30 smithi064 ceph-mon[30960]: pgmap v284: 97 pgs: 97 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 13 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-09-19T05:04:30.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:30 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/55198869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:30.429 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:30 smithi174 ceph-mon[34488]: pgmap v284: 97 pgs: 97 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 13 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-09-19T05:04:30.429 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:30 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/55198869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:30.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:30 smithi103 ceph-mon[34446]: pgmap v284: 97 pgs: 97 active+clean; 607 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 13 op/s; 3 B/s, 2 keys/s, 0 objects/s recovering 2024-09-19T05:04:30.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:30 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/55198869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:30.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:30.923 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 -- 172.21.15.64:0/3873787116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102570 msgr2=0x7f4fa41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 --2- 172.21.15.64:0/3873787116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102570 0x7f4fa41029b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009900 tx=0x7f4f9802a480 comp rx=0 tx=0).stop 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 -- 172.21.15.64:0/3873787116 shutdown_connections 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 --2- 172.21.15.64:0/3873787116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4102ef0 0x7f4fa410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 --2- 172.21.15.64:0/3873787116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102570 0x7f4fa41029b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 --2- 172.21.15.64:0/3873787116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa4100e20 0x7f4fa4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 -- 172.21.15.64:0/3873787116 >> 172.21.15.64:0/3873787116 conn(0x7f4fa40fc7d0 msgr2=0x7f4fa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 -- 172.21.15.64:0/3873787116 shutdown_connections 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 -- 172.21.15.64:0/3873787116 wait complete. 2024-09-19T05:04:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 Processor -- start 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.395+0000 7f4fabd34640 1 -- start start 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa4100e20 0x7f4fa4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4102570 0x7f4fa41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 0x7f4fa419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4111f80 con 0x7f4fa4102ef0 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4111e00 con 0x7f4fa4100e20 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4112100 con 0x7f4fa4102570 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 0x7f4fa419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 0x7f4fa419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54594/0 (socket says 172.21.15.64:54594) 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 -- 172.21.15.64:0/2111202683 learned_addr learned my addr 172.21.15.64:0/2111202683 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fa92cf640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4102570 0x7f4fa41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fa9ad0640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa4100e20 0x7f4fa4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 -- 172.21.15.64:0/2111202683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4102570 msgr2=0x7f4fa41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4102570 0x7f4fa41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 -- 172.21.15.64:0/2111202683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa4100e20 msgr2=0x7f4fa4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa4100e20 0x7f4fa4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 -- 172.21.15.64:0/2111202683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa419d420 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4faa2d1640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 0x7f4fa419cd40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400eb10 tx=0x7f4f9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004b20 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f94004cc0 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa419d710 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.396+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94010690 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.397+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa41a4ff0 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.397+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa4101200 con 0x7f4fa4102ef0 2024-09-19T05:04:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.399+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4f94005020 con 0x7f4fa4102ef0 2024-09-19T05:04:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.400+0000 7f4fa2ffd640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4f8c0a3350 0x7f4f8c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.400+0000 7f4fa9ad0640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4f8c0a3350 0x7f4f8c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.400+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f4f940d0140 con 0x7f4fa4102ef0 2024-09-19T05:04:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.400+0000 7f4fa9ad0640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4f8c0a3350 0x7f4f8c0a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009b60 tx=0x7f4f9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.404+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f9409cce0 con 0x7f4fa4102ef0 2024-09-19T05:04:31.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.581+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fa41090c0 con 0x7f4fa4102ef0 2024-09-19T05:04:31.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.583+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2111202683 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22821 (secure 0 0 0) 0x7f4f9409c5e0 con 0x7f4fa4102ef0 2024-09-19T05:04:31.584 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:31.585 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:20.028795+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4f8c0a3350 msgr2=0x7f4f8c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4f8c0a3350 0x7f4f8c0a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009b60 tx=0x7f4f9c005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 msgr2=0x7f4fa419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 0x7f4fa419cd40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400eb10 tx=0x7f4f9400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 shutdown_connections 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f4f8c0a3350 0x7f4f8c0a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa4102ef0 0x7f4fa419cd40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4102570 0x7f4fa41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 --2- 172.21.15.64:0/2111202683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa4100e20 0x7f4fa4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 >> 172.21.15.64:0/2111202683 conn(0x7f4fa40fc7d0 msgr2=0x7f4fa40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 shutdown_connections 2024-09-19T05:04:31.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:31.585+0000 7f4fabd34640 1 -- 172.21.15.64:0/2111202683 wait complete. 2024-09-19T05:04:32.343 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:32 smithi064 ceph-mon[30960]: pgmap v285: 97 pgs: 97 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.4 KiB/s wr, 13 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:32.343 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:32 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2111202683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:32.372 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:04:32.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:04:32.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:32 smithi103 ceph-mon[34446]: pgmap v285: 97 pgs: 97 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.4 KiB/s wr, 13 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:32.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:32 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2111202683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:32.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:32 smithi174 ceph-mon[34488]: pgmap v285: 97 pgs: 97 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.4 KiB/s wr, 13 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:32.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:32 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2111202683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:32.629 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:33.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.051+0000 7fc9a0769640 1 -- 172.21.15.64:0/1927104901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981082a0 msgr2=0x7fc99810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 --2- 172.21.15.64:0/1927104901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981082a0 0x7fc99810ec40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc990009cd0 tx=0x7fc99002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 -- 172.21.15.64:0/1927104901 shutdown_connections 2024-09-19T05:04:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 --2- 172.21.15.64:0/1927104901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981082a0 0x7fc99810ec40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 --2- 172.21.15.64:0/1927104901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9981036e0 0x7fc998107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 --2- 172.21.15.64:0/1927104901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc998102d30 0x7fc998103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 -- 172.21.15.64:0/1927104901 >> 172.21.15.64:0/1927104901 conn(0x7fc9980fc7d0 msgr2=0x7fc9980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 -- 172.21.15.64:0/1927104901 shutdown_connections 2024-09-19T05:04:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 -- 172.21.15.64:0/1927104901 wait complete. 2024-09-19T05:04:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.052+0000 7fc9a0769640 1 Processor -- start 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 -- start start 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc998102d30 0x7fc9981984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 0x7fc9981989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9981082a0 0x7fc99819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc998111f50 con 0x7fc9981036e0 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc998111dd0 con 0x7fc998102d30 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc9a0769640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981120d0 con 0x7fc9981082a0 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 0x7fc9981989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 0x7fc9981989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54612/0 (socket says 172.21.15.64:54612) 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 -- 172.21.15.64:0/241639082 learned_addr learned my addr 172.21.15.64:0/241639082 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99e505640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc998102d30 0x7fc9981984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99ed06640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9981082a0 0x7fc99819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 -- 172.21.15.64:0/241639082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9981082a0 msgr2=0x7fc99819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9981082a0 0x7fc99819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 -- 172.21.15.64:0/241639082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc998102d30 msgr2=0x7fc9981984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc998102d30 0x7fc9981984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.053+0000 7fc99dd04640 1 -- 172.21.15.64:0/241639082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99819d450 con 0x7fc9981036e0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.054+0000 7fc99dd04640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 0x7fc9981989e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e9c0 tx=0x7fc98800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.054+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98800ccc0 con 0x7fc9981036e0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.054+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9880054d0 con 0x7fc9981036e0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.054+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99819d740 con 0x7fc9981036e0 2024-09-19T05:04:33.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.054+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988010430 con 0x7fc9981036e0 2024-09-19T05:04:33.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.054+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981a5020 con 0x7fc9981036e0 2024-09-19T05:04:33.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.055+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc998103110 con 0x7fc9981036e0 2024-09-19T05:04:33.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.056+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc988005000 con 0x7fc9981036e0 2024-09-19T05:04:33.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.057+0000 7fc98f7fe640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9840a32d0 0x7fc9840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.057+0000 7fc99e505640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9840a32d0 0x7fc9840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.057+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fc9880d0360 con 0x7fc9981036e0 2024-09-19T05:04:33.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.057+0000 7fc99e505640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9840a32d0 0x7fc9840a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc994009bd0 tx=0x7fc994005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:33.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.060+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc9880052f0 con 0x7fc9981036e0 2024-09-19T05:04:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.233+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc99810a2e0 con 0x7fc9840a32d0 2024-09-19T05:04:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.240+0000 7fc98f7fe640 1 -- 172.21.15.64:0/241639082 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258717 (secure 0 0 0) 0x7fc99810a2e0 con 0x7fc9840a32d0 2024-09-19T05:04:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9840a32d0 msgr2=0x7fc9840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9840a32d0 0x7fc9840a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc994009bd0 tx=0x7fc994005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 msgr2=0x7fc9981989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 0x7fc9981989e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e9c0 tx=0x7fc98800ee80 comp rx=0 tx=0).stop 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 shutdown_connections 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc9840a32d0 0x7fc9840a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9981082a0 0x7fc99819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9981036e0 0x7fc9981989e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 --2- 172.21.15.64:0/241639082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc998102d30 0x7fc9981984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 >> 172.21.15.64:0/241639082 conn(0x7fc9980fc7d0 msgr2=0x7fc998105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 shutdown_connections 2024-09-19T05:04:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:33.244+0000 7fc9a0769640 1 -- 172.21.15.64:0/241639082 wait complete. 2024-09-19T05:04:34.021 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-19T05:04:32.382591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644208,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":228,"num_read_kb":214,"num_write":335,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365680,"kb_used_data":12256,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576144,"statfs":{"total":1151940427776,"available":1151565971456,"internally_reserved":0,"allocated":12550144,"data_stored":6990294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48618,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":-6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":71,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002982"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045762+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:04:20.045762+0000","last_peered":"2024-09-19T05:04:20.045762+0000","last_clean":"2024-09-19T05:04:20.045762+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:04:20.045762+0000","last_undegraded":"2024-09-19T05:04:20.045762+0000","last_fullsized":"2024-09-19T05:04:20.045762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042748+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:04:20.042748+0000","last_peered":"2024-09-19T05:04:20.042748+0000","last_clean":"2024-09-19T05:04:20.042748+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:04:20.042748+0000","last_undegraded":"2024-09-19T05:04:20.042748+0000","last_fullsized":"2024-09-19T05:04:20.042748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042862+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:04:20.042862+0000","last_peered":"2024-09-19T05:04:20.042862+0000","last_clean":"2024-09-19T05:04:20.042862+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:04:20.042862+0000","last_undegraded":"2024-09-19T05:04:20.042862+0000","last_fullsized":"2024-09-19T05:04:20.042862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218922+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:04:20.218922+0000","last_peered":"2024-09-19T05:04:20.218922+0000","last_clean":"2024-09-19T05:04:20.218922+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:04:20.218922+0000","last_undegraded":"2024-09-19T05:04:20.218922+0000","last_fullsized":"2024-09-19T05:04:20.218922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044933+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:04:20.044933+0000","last_peered":"2024-09-19T05:04:20.044933+0000","last_clean":"2024-09-19T05:04:20.044933+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:04:20.044933+0000","last_undegraded":"2024-09-19T05:04:20.044933+0000","last_fullsized":"2024-09-19T05:04:20.044933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046644+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:04:20.046644+0000","last_peered":"2024-09-19T05:04:20.046644+0000","last_clean":"2024-09-19T05:04:20.046644+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:04:20.046644+0000","last_undegraded":"2024-09-19T05:04:20.046644+0000","last_fullsized":"2024-09-19T05:04:20.046644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041281+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:04:20.041281+0000","last_peered":"2024-09-19T05:04:20.041281+0000","last_clean":"2024-09-19T05:04:20.041281+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:04:20.041281+0000","last_undegraded":"2024-09-19T05:04:20.041281+0000","last_fullsized":"2024-09-19T05:04:20.041281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045982+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:04:20.045982+0000","last_peered":"2024-09-19T05:04:20.045982+0000","last_clean":"2024-09-19T05:04:20.045982+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:04:20.045982+0000","last_undegraded":"2024-09-19T05:04:20.045982+0000","last_fullsized":"2024-09-19T05:04:20.045982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039683+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:04:20.039683+0000","last_peered":"2024-09-19T05:04:20.039683+0000","last_clean":"2024-09-19T05:04:20.039683+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:04:20.039683+0000","last_undegraded":"2024-09-19T05:04:20.039683+0000","last_fullsized":"2024-09-19T05:04:20.039683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043775+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:04:20.043775+0000","last_peered":"2024-09-19T05:04:20.043775+0000","last_clean":"2024-09-19T05:04:20.043775+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:04:20.043775+0000","last_undegraded":"2024-09-19T05:04:20.043775+0000","last_fullsized":"2024-09-19T05:04:20.043775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041842+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:04:20.041842+0000","last_peered":"2024-09-19T05:04:20.041842+0000","last_clean":"2024-09-19T05:04:20.041842+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:04:20.041842+0000","last_undegraded":"2024-09-19T05:04:20.041842+0000","last_fullsized":"2024-09-19T05:04:20.041842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219054+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:04:20.219054+0000","last_peered":"2024-09-19T05:04:20.219054+0000","last_clean":"2024-09-19T05:04:20.219054+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:04:20.219054+0000","last_undegraded":"2024-09-19T05:04:20.219054+0000","last_fullsized":"2024-09-19T05:04:20.219054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038234+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:04:20.038234+0000","last_peered":"2024-09-19T05:04:20.038234+0000","last_clean":"2024-09-19T05:04:20.038234+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:04:20.038234+0000","last_undegraded":"2024-09-19T05:04:20.038234+0000","last_fullsized":"2024-09-19T05:04:20.038234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042859+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:04:20.042859+0000","last_peered":"2024-09-19T05:04:20.042859+0000","last_clean":"2024-09-19T05:04:20.042859+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:04:20.042859+0000","last_undegraded":"2024-09-19T05:04:20.042859+0000","last_fullsized":"2024-09-19T05:04:20.042859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042616+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:04:20.042616+0000","last_peered":"2024-09-19T05:04:20.042616+0000","last_clean":"2024-09-19T05:04:20.042616+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:04:20.042616+0000","last_undegraded":"2024-09-19T05:04:20.042616+0000","last_fullsized":"2024-09-19T05:04:20.042616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039034+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:04:20.039034+0000","last_peered":"2024-09-19T05:04:20.039034+0000","last_clean":"2024-09-19T05:04:20.039034+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:04:20.039034+0000","last_undegraded":"2024-09-19T05:04:20.039034+0000","last_fullsized":"2024-09-19T05:04:20.039034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045863+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:04:20.045863+0000","last_peered":"2024-09-19T05:04:20.045863+0000","last_clean":"2024-09-19T05:04:20.045863+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:04:20.045863+0000","last_undegraded":"2024-09-19T05:04:20.045863+0000","last_fullsized":"2024-09-19T05:04:20.045863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040764+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:04:20.040764+0000","last_peered":"2024-09-19T05:04:20.040764+0000","last_clean":"2024-09-19T05:04:20.040764+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:04:20.040764+0000","last_undegraded":"2024-09-19T05:04:20.040764+0000","last_fullsized":"2024-09-19T05:04:20.040764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045627+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:04:20.045627+0000","last_peered":"2024-09-19T05:04:20.045627+0000","last_clean":"2024-09-19T05:04:20.045627+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:04:20.045627+0000","last_undegraded":"2024-09-19T05:04:20.045627+0000","last_fullsized":"2024-09-19T05:04:20.045627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045190+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:04:20.045190+0000","last_peered":"2024-09-19T05:04:20.045190+0000","last_clean":"2024-09-19T05:04:20.045190+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:04:20.045190+0000","last_undegraded":"2024-09-19T05:04:20.045190+0000","last_fullsized":"2024-09-19T05:04:20.045190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038280+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:04:20.038280+0000","last_peered":"2024-09-19T05:04:20.038280+0000","last_clean":"2024-09-19T05:04:20.038280+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:04:20.038280+0000","last_undegraded":"2024-09-19T05:04:20.038280+0000","last_fullsized":"2024-09-19T05:04:20.038280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039404+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:04:20.039404+0000","last_peered":"2024-09-19T05:04:20.039404+0000","last_clean":"2024-09-19T05:04:20.039404+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:04:20.039404+0000","last_undegraded":"2024-09-19T05:04:20.039404+0000","last_fullsized":"2024-09-19T05:04:20.039404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046472+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:04:20.046472+0000","last_peered":"2024-09-19T05:04:20.046472+0000","last_clean":"2024-09-19T05:04:20.046472+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:04:20.046472+0000","last_undegraded":"2024-09-19T05:04:20.046472+0000","last_fullsized":"2024-09-19T05:04:20.046472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040224+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:04:20.040224+0000","last_peered":"2024-09-19T05:04:20.040224+0000","last_clean":"2024-09-19T05:04:20.040224+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:04:20.040224+0000","last_undegraded":"2024-09-19T05:04:20.040224+0000","last_fullsized":"2024-09-19T05:04:20.040224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220980+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:04:20.220980+0000","last_peered":"2024-09-19T05:04:20.220980+0000","last_clean":"2024-09-19T05:04:20.220980+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:04:20.220980+0000","last_undegraded":"2024-09-19T05:04:20.220980+0000","last_fullsized":"2024-09-19T05:04:20.220980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044307+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:04:20.044307+0000","last_peered":"2024-09-19T05:04:20.044307+0000","last_clean":"2024-09-19T05:04:20.044307+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:04:20.044307+0000","last_undegraded":"2024-09-19T05:04:20.044307+0000","last_fullsized":"2024-09-19T05:04:20.044307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046219+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:04:20.046219+0000","last_peered":"2024-09-19T05:04:20.046219+0000","last_clean":"2024-09-19T05:04:20.046219+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:04:20.046219+0000","last_undegraded":"2024-09-19T05:04:20.046219+0000","last_fullsized":"2024-09-19T05:04:20.046219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038227+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:04:20.038227+0000","last_peered":"2024-09-19T05:04:20.038227+0000","last_clean":"2024-09-19T05:04:20.038227+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:04:20.038227+0000","last_undegraded":"2024-09-19T05:04:20.038227+0000","last_fullsized":"2024-09-19T05:04:20.038227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042890+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:04:20.042890+0000","last_peered":"2024-09-19T05:04:20.042890+0000","last_clean":"2024-09-19T05:04:20.042890+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:04:20.042890+0000","last_undegraded":"2024-09-19T05:04:20.042890+0000","last_fullsized":"2024-09-19T05:04:20.042890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044794+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:04:20.044794+0000","last_peered":"2024-09-19T05:04:20.044794+0000","last_clean":"2024-09-19T05:04:20.044794+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:04:20.044794+0000","last_undegraded":"2024-09-19T05:04:20.044794+0000","last_fullsized":"2024-09-19T05:04:20.044794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044665+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:04:20.044665+0000","last_peered":"2024-09-19T05:04:20.044665+0000","last_clean":"2024-09-19T05:04:20.044665+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:04:20.044665+0000","last_undegraded":"2024-09-19T05:04:20.044665+0000","last_fullsized":"2024-09-19T05:04:20.044665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045176+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:04:20.045176+0000","last_peered":"2024-09-19T05:04:20.045176+0000","last_clean":"2024-09-19T05:04:20.045176+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:04:20.045176+0000","last_undegraded":"2024-09-19T05:04:20.045176+0000","last_fullsized":"2024-09-19T05:04:20.045176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045192+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:04:20.045192+0000","last_peered":"2024-09-19T05:04:20.045192+0000","last_clean":"2024-09-19T05:04:20.045192+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:04:20.045192+0000","last_undegraded":"2024-09-19T05:04:20.045192+0000","last_fullsized":"2024-09-19T05:04:20.045192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":42,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:27.037406+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:04:27.037406+0000","last_peered":"2024-09-19T05:04:27.037406+0000","last_clean":"2024-09-19T05:04:27.037406+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:04:27.037406+0000","last_undegraded":"2024-09-19T05:04:27.037406+0000","last_fullsized":"2024-09-19T05:04:27.037406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220294+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:04:20.220294+0000","last_peered":"2024-09-19T05:04:20.220294+0000","last_clean":"2024-09-19T05:04:20.220294+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:04:20.220294+0000","last_undegraded":"2024-09-19T05:04:20.220294+0000","last_fullsized":"2024-09-19T05:04:20.220294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043292+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:04:20.043292+0000","last_peered":"2024-09-19T05:04:20.043292+0000","last_clean":"2024-09-19T05:04:20.043292+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:04:20.043292+0000","last_undegraded":"2024-09-19T05:04:20.043292+0000","last_fullsized":"2024-09-19T05:04:20.043292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037995+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:04:20.037995+0000","last_peered":"2024-09-19T05:04:20.037995+0000","last_clean":"2024-09-19T05:04:20.037995+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:04:20.037995+0000","last_undegraded":"2024-09-19T05:04:20.037995+0000","last_fullsized":"2024-09-19T05:04:20.037995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046054+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:04:20.046054+0000","last_peered":"2024-09-19T05:04:20.046054+0000","last_clean":"2024-09-19T05:04:20.046054+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:04:20.046054+0000","last_undegraded":"2024-09-19T05:04:20.046054+0000","last_fullsized":"2024-09-19T05:04:20.046054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040479+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:04:20.040479+0000","last_peered":"2024-09-19T05:04:20.040479+0000","last_clean":"2024-09-19T05:04:20.040479+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:04:20.040479+0000","last_undegraded":"2024-09-19T05:04:20.040479+0000","last_fullsized":"2024-09-19T05:04:20.040479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041190+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:04:20.041190+0000","last_peered":"2024-09-19T05:04:20.041190+0000","last_clean":"2024-09-19T05:04:20.041190+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:04:20.041190+0000","last_undegraded":"2024-09-19T05:04:20.041190+0000","last_fullsized":"2024-09-19T05:04:20.041190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:27.036855+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:04:27.036855+0000","last_peered":"2024-09-19T05:04:27.036855+0000","last_clean":"2024-09-19T05:04:27.036855+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:04:27.036855+0000","last_undegraded":"2024-09-19T05:04:27.036855+0000","last_fullsized":"2024-09-19T05:04:27.036855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039611+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:04:20.039611+0000","last_peered":"2024-09-19T05:04:20.039611+0000","last_clean":"2024-09-19T05:04:20.039611+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:04:20.039611+0000","last_undegraded":"2024-09-19T05:04:20.039611+0000","last_fullsized":"2024-09-19T05:04:20.039611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.223302+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:04:20.223302+0000","last_peered":"2024-09-19T05:04:20.223302+0000","last_clean":"2024-09-19T05:04:20.223302+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:04:20.223302+0000","last_undegraded":"2024-09-19T05:04:20.223302+0000","last_fullsized":"2024-09-19T05:04:20.223302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044799+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:04:20.044799+0000","last_peered":"2024-09-19T05:04:20.044799+0000","last_clean":"2024-09-19T05:04:20.044799+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:04:20.044799+0000","last_undegraded":"2024-09-19T05:04:20.044799+0000","last_fullsized":"2024-09-19T05:04:20.044799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040554+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:04:20.040554+0000","last_peered":"2024-09-19T05:04:20.040554+0000","last_clean":"2024-09-19T05:04:20.040554+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:04:20.040554+0000","last_undegraded":"2024-09-19T05:04:20.040554+0000","last_fullsized":"2024-09-19T05:04:20.040554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039165+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:04:20.039165+0000","last_peered":"2024-09-19T05:04:20.039165+0000","last_clean":"2024-09-19T05:04:20.039165+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:04:20.039165+0000","last_undegraded":"2024-09-19T05:04:20.039165+0000","last_fullsized":"2024-09-19T05:04:20.039165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'25","reported_seq":74,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:28.471000+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:04:28.471000+0000","last_peered":"2024-09-19T05:04:28.471000+0000","last_clean":"2024-09-19T05:04:28.471000+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:04:28.471000+0000","last_undegraded":"2024-09-19T05:04:28.471000+0000","last_fullsized":"2024-09-19T05:04:28.471000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49603,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":53,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044465+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:04:20.044465+0000","last_peered":"2024-09-19T05:04:20.044465+0000","last_clean":"2024-09-19T05:04:20.044465+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:04:20.044465+0000","last_undegraded":"2024-09-19T05:04:20.044465+0000","last_fullsized":"2024-09-19T05:04:20.044465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.058790+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:04:20.058790+0000","last_peered":"2024-09-19T05:04:20.058790+0000","last_clean":"2024-09-19T05:04:20.058790+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:04:20.058790+0000","last_undegraded":"2024-09-19T05:04:20.058790+0000","last_fullsized":"2024-09-19T05:04:20.058790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038290+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:04:20.038290+0000","last_peered":"2024-09-19T05:04:20.038290+0000","last_clean":"2024-09-19T05:04:20.038290+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:04:20.038290+0000","last_undegraded":"2024-09-19T05:04:20.038290+0000","last_fullsized":"2024-09-19T05:04:20.038290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040388+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:04:20.040388+0000","last_peered":"2024-09-19T05:04:20.040388+0000","last_clean":"2024-09-19T05:04:20.040388+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:04:20.040388+0000","last_undegraded":"2024-09-19T05:04:20.040388+0000","last_fullsized":"2024-09-19T05:04:20.040388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041839+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:04:20.041839+0000","last_peered":"2024-09-19T05:04:20.041839+0000","last_clean":"2024-09-19T05:04:20.041839+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:04:20.041839+0000","last_undegraded":"2024-09-19T05:04:20.041839+0000","last_fullsized":"2024-09-19T05:04:20.041839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'3","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:28.038521+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:04:28.038521+0000","last_peered":"2024-09-19T05:04:28.038521+0000","last_clean":"2024-09-19T05:04:28.038521+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:04:28.038521+0000","last_undegraded":"2024-09-19T05:04:28.038521+0000","last_fullsized":"2024-09-19T05:04:28.038521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041986+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:04:20.041986+0000","last_peered":"2024-09-19T05:04:20.041986+0000","last_clean":"2024-09-19T05:04:20.041986+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:04:20.041986+0000","last_undegraded":"2024-09-19T05:04:20.041986+0000","last_fullsized":"2024-09-19T05:04:20.041986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044042+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:04:20.044042+0000","last_peered":"2024-09-19T05:04:20.044042+0000","last_clean":"2024-09-19T05:04:20.044042+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:04:20.044042+0000","last_undegraded":"2024-09-19T05:04:20.044042+0000","last_fullsized":"2024-09-19T05:04:20.044042+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220988+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:04:20.220988+0000","last_peered":"2024-09-19T05:04:20.220988+0000","last_clean":"2024-09-19T05:04:20.220988+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:04:20.220988+0000","last_undegraded":"2024-09-19T05:04:20.220988+0000","last_fullsized":"2024-09-19T05:04:20.220988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044689+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:04:20.044689+0000","last_peered":"2024-09-19T05:04:20.044689+0000","last_clean":"2024-09-19T05:04:20.044689+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:04:20.044689+0000","last_undegraded":"2024-09-19T05:04:20.044689+0000","last_fullsized":"2024-09-19T05:04:20.044689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045469+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:04:20.045469+0000","last_peered":"2024-09-19T05:04:20.045469+0000","last_clean":"2024-09-19T05:04:20.045469+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:04:20.045469+0000","last_undegraded":"2024-09-19T05:04:20.045469+0000","last_fullsized":"2024-09-19T05:04:20.045469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038412+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:04:20.038412+0000","last_peered":"2024-09-19T05:04:20.038412+0000","last_clean":"2024-09-19T05:04:20.038412+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:04:20.038412+0000","last_undegraded":"2024-09-19T05:04:20.038412+0000","last_fullsized":"2024-09-19T05:04:20.038412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041819+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:04:20.041819+0000","last_peered":"2024-09-19T05:04:20.041819+0000","last_clean":"2024-09-19T05:04:20.041819+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:04:20.041819+0000","last_undegraded":"2024-09-19T05:04:20.041819+0000","last_fullsized":"2024-09-19T05:04:20.041819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040558+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:04:20.040558+0000","last_peered":"2024-09-19T05:04:20.040558+0000","last_clean":"2024-09-19T05:04:20.040558+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:04:20.040558+0000","last_undegraded":"2024-09-19T05:04:20.040558+0000","last_fullsized":"2024-09-19T05:04:20.040558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041588+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:04:20.041588+0000","last_peered":"2024-09-19T05:04:20.041588+0000","last_clean":"2024-09-19T05:04:20.041588+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:04:20.041588+0000","last_undegraded":"2024-09-19T05:04:20.041588+0000","last_fullsized":"2024-09-19T05:04:20.041588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044395+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:04:20.044395+0000","last_peered":"2024-09-19T05:04:20.044395+0000","last_clean":"2024-09-19T05:04:20.044395+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:04:20.044395+0000","last_undegraded":"2024-09-19T05:04:20.044395+0000","last_fullsized":"2024-09-19T05:04:20.044395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041954+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:04:20.041954+0000","last_peered":"2024-09-19T05:04:20.041954+0000","last_clean":"2024-09-19T05:04:20.041954+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:04:20.041954+0000","last_undegraded":"2024-09-19T05:04:20.041954+0000","last_fullsized":"2024-09-19T05:04:20.041954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":343,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042103+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:04:20.042103+0000","last_peered":"2024-09-19T05:04:20.042103+0000","last_clean":"2024-09-19T05:04:20.042103+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:04:20.042103+0000","last_undegraded":"2024-09-19T05:04:20.042103+0000","last_fullsized":"2024-09-19T05:04:20.042103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218980+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:04:20.218980+0000","last_peered":"2024-09-19T05:04:20.218980+0000","last_clean":"2024-09-19T05:04:20.218980+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:04:20.218980+0000","last_undegraded":"2024-09-19T05:04:20.218980+0000","last_fullsized":"2024-09-19T05:04:20.218980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046490+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:04:20.046490+0000","last_peered":"2024-09-19T05:04:20.046490+0000","last_clean":"2024-09-19T05:04:20.046490+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:04:20.046490+0000","last_undegraded":"2024-09-19T05:04:20.046490+0000","last_fullsized":"2024-09-19T05:04:20.046490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.048907+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:04:20.048907+0000","last_peered":"2024-09-19T05:04:20.048907+0000","last_clean":"2024-09-19T05:04:20.048907+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:04:20.048907+0000","last_undegraded":"2024-09-19T05:04:20.048907+0000","last_fullsized":"2024-09-19T05:04:20.048907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037984+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:04:20.037984+0000","last_peered":"2024-09-19T05:04:20.037984+0000","last_clean":"2024-09-19T05:04:20.037984+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:04:20.037984+0000","last_undegraded":"2024-09-19T05:04:20.037984+0000","last_fullsized":"2024-09-19T05:04:20.037984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:29.039331+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:04:29.039331+0000","last_peered":"2024-09-19T05:04:29.039331+0000","last_clean":"2024-09-19T05:04:29.039331+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:04:29.039331+0000","last_undegraded":"2024-09-19T05:04:29.039331+0000","last_fullsized":"2024-09-19T05:04:29.039331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040362+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:04:20.040362+0000","last_peered":"2024-09-19T05:04:20.040362+0000","last_clean":"2024-09-19T05:04:20.040362+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:04:20.040362+0000","last_undegraded":"2024-09-19T05:04:20.040362+0000","last_fullsized":"2024-09-19T05:04:20.040362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041750+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:04:20.041750+0000","last_peered":"2024-09-19T05:04:20.041750+0000","last_clean":"2024-09-19T05:04:20.041750+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:04:20.041750+0000","last_undegraded":"2024-09-19T05:04:20.041750+0000","last_fullsized":"2024-09-19T05:04:20.041750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043344+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:04:20.043344+0000","last_peered":"2024-09-19T05:04:20.043344+0000","last_clean":"2024-09-19T05:04:20.043344+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:04:20.043344+0000","last_undegraded":"2024-09-19T05:04:20.043344+0000","last_fullsized":"2024-09-19T05:04:20.043344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040735+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:04:20.040735+0000","last_peered":"2024-09-19T05:04:20.040735+0000","last_clean":"2024-09-19T05:04:20.040735+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:04:20.040735+0000","last_undegraded":"2024-09-19T05:04:20.040735+0000","last_fullsized":"2024-09-19T05:04:20.040735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043291+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:04:20.043291+0000","last_peered":"2024-09-19T05:04:20.043291+0000","last_clean":"2024-09-19T05:04:20.043291+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:04:20.043291+0000","last_undegraded":"2024-09-19T05:04:20.043291+0000","last_fullsized":"2024-09-19T05:04:20.043291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044730+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:04:20.044730+0000","last_peered":"2024-09-19T05:04:20.044730+0000","last_clean":"2024-09-19T05:04:20.044730+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:04:20.044730+0000","last_undegraded":"2024-09-19T05:04:20.044730+0000","last_fullsized":"2024-09-19T05:04:20.044730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038415+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:04:20.038415+0000","last_peered":"2024-09-19T05:04:20.038415+0000","last_clean":"2024-09-19T05:04:20.038415+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:04:20.038415+0000","last_undegraded":"2024-09-19T05:04:20.038415+0000","last_fullsized":"2024-09-19T05:04:20.038415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:26.976440+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:04:26.976440+0000","last_peered":"2024-09-19T05:04:26.976440+0000","last_clean":"2024-09-19T05:04:26.976440+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:04:26.976440+0000","last_undegraded":"2024-09-19T05:04:26.976440+0000","last_fullsized":"2024-09-19T05:04:26.976440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046734+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:04:20.046734+0000","last_peered":"2024-09-19T05:04:20.046734+0000","last_clean":"2024-09-19T05:04:20.046734+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:04:20.046734+0000","last_undegraded":"2024-09-19T05:04:20.046734+0000","last_fullsized":"2024-09-19T05:04:20.046734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219379+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:04:20.219379+0000","last_peered":"2024-09-19T05:04:20.219379+0000","last_clean":"2024-09-19T05:04:20.219379+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:04:20.219379+0000","last_undegraded":"2024-09-19T05:04:20.219379+0000","last_fullsized":"2024-09-19T05:04:20.219379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.047866+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:04:20.047866+0000","last_peered":"2024-09-19T05:04:20.047866+0000","last_clean":"2024-09-19T05:04:20.047866+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:04:20.047866+0000","last_undegraded":"2024-09-19T05:04:20.047866+0000","last_fullsized":"2024-09-19T05:04:20.047866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039180+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:04:20.039180+0000","last_peered":"2024-09-19T05:04:20.039180+0000","last_clean":"2024-09-19T05:04:20.039180+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:04:20.039180+0000","last_undegraded":"2024-09-19T05:04:20.039180+0000","last_fullsized":"2024-09-19T05:04:20.039180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043811+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:04:20.043811+0000","last_peered":"2024-09-19T05:04:20.043811+0000","last_clean":"2024-09-19T05:04:20.043811+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:04:20.043811+0000","last_undegraded":"2024-09-19T05:04:20.043811+0000","last_fullsized":"2024-09-19T05:04:20.043811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046197+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:04:20.046197+0000","last_peered":"2024-09-19T05:04:20.046197+0000","last_clean":"2024-09-19T05:04:20.046197+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:04:20.046197+0000","last_undegraded":"2024-09-19T05:04:20.046197+0000","last_fullsized":"2024-09-19T05:04:20.046197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045294+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:04:20.045294+0000","last_peered":"2024-09-19T05:04:20.045294+0000","last_clean":"2024-09-19T05:04:20.045294+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:04:20.045294+0000","last_undegraded":"2024-09-19T05:04:20.045294+0000","last_fullsized":"2024-09-19T05:04:20.045294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044905+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:04:20.044905+0000","last_peered":"2024-09-19T05:04:20.044905+0000","last_clean":"2024-09-19T05:04:20.044905+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:04:20.044905+0000","last_undegraded":"2024-09-19T05:04:20.044905+0000","last_fullsized":"2024-09-19T05:04:20.044905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046267+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:04:20.046267+0000","last_peered":"2024-09-19T05:04:20.046267+0000","last_clean":"2024-09-19T05:04:20.046267+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:04:20.046267+0000","last_undegraded":"2024-09-19T05:04:20.046267+0000","last_fullsized":"2024-09-19T05:04:20.046267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044834+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:04:20.044834+0000","last_peered":"2024-09-19T05:04:20.044834+0000","last_clean":"2024-09-19T05:04:20.044834+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:04:20.044834+0000","last_undegraded":"2024-09-19T05:04:20.044834+0000","last_fullsized":"2024-09-19T05:04:20.044834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041613+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:04:20.041613+0000","last_peered":"2024-09-19T05:04:20.041613+0000","last_clean":"2024-09-19T05:04:20.041613+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:04:20.041613+0000","last_undegraded":"2024-09-19T05:04:20.041613+0000","last_fullsized":"2024-09-19T05:04:20.041613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040445+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:04:20.040445+0000","last_peered":"2024-09-19T05:04:20.040445+0000","last_clean":"2024-09-19T05:04:20.040445+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:04:20.040445+0000","last_undegraded":"2024-09-19T05:04:20.040445+0000","last_fullsized":"2024-09-19T05:04:20.040445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044233+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:04:20.044233+0000","last_peered":"2024-09-19T05:04:20.044233+0000","last_clean":"2024-09-19T05:04:20.044233+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:04:20.044233+0000","last_undegraded":"2024-09-19T05:04:20.044233+0000","last_fullsized":"2024-09-19T05:04:20.044233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039172+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:04:20.039172+0000","last_peered":"2024-09-19T05:04:20.039172+0000","last_clean":"2024-09-19T05:04:20.039172+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:04:20.039172+0000","last_undegraded":"2024-09-19T05:04:20.039172+0000","last_fullsized":"2024-09-19T05:04:20.039172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.052380+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:04:20.052380+0000","last_peered":"2024-09-19T05:04:20.052380+0000","last_clean":"2024-09-19T05:04:20.052380+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:04:20.052380+0000","last_undegraded":"2024-09-19T05:04:20.052380+0000","last_fullsized":"2024-09-19T05:04:20.052380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043121+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:04:20.043121+0000","last_peered":"2024-09-19T05:04:20.043121+0000","last_clean":"2024-09-19T05:04:20.043121+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:04:20.043121+0000","last_undegraded":"2024-09-19T05:04:20.043121+0000","last_fullsized":"2024-09-19T05:04:20.043121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046887+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:04:20.046887+0000","last_peered":"2024-09-19T05:04:20.046887+0000","last_clean":"2024-09-19T05:04:20.046887+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:04:20.046887+0000","last_undegraded":"2024-09-19T05:04:20.046887+0000","last_fullsized":"2024-09-19T05:04:20.046887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046641+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:04:20.046641+0000","last_peered":"2024-09-19T05:04:20.046641+0000","last_clean":"2024-09-19T05:04:20.046641+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:04:20.046641+0000","last_undegraded":"2024-09-19T05:04:20.046641+0000","last_fullsized":"2024-09-19T05:04:20.046641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041192+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:04:20.041192+0000","last_peered":"2024-09-19T05:04:20.041192+0000","last_clean":"2024-09-19T05:04:20.041192+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:04:20.041192+0000","last_undegraded":"2024-09-19T05:04:20.041192+0000","last_fullsized":"2024-09-19T05:04:20.041192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53606,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":92,"num_read_kb":95,"num_write":152,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":160818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":422800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1055312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070490,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":423002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":422922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528356,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1014854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888110,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32640,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712512,"statfs":{"total":95995035648,"available":95961612288,"internally_reserved":0,"allocated":1490944,"data_stored":1014538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":422828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247863,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":422702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444093,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":473496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":471787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":423662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":51312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:04:34.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:04:34.284 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:34.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:34 smithi064 ceph-mon[30960]: pgmap v286: 97 pgs: 97 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.0 KiB/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:34.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:34 smithi064 ceph-mon[30960]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:34.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:34 smithi103 ceph-mon[34446]: pgmap v286: 97 pgs: 97 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.0 KiB/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:34.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:34 smithi103 ceph-mon[34446]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:34.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:34 smithi174 ceph-mon[34488]: pgmap v286: 97 pgs: 97 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.0 KiB/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:34.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:34 smithi174 ceph-mon[34488]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 -- 172.21.15.64:0/3913045910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840101450 msgr2=0x7ff840105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 --2- 172.21.15.64:0/3913045910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840101450 0x7ff840105920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff830009900 tx=0x7ff83002a470 comp rx=0 tx=0).stop 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 -- 172.21.15.64:0/3913045910 shutdown_connections 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 --2- 172.21.15.64:0/3913045910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840105f40 0x7ff8401063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 --2- 172.21.15.64:0/3913045910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840101450 0x7ff840105920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 --2- 172.21.15.64:0/3913045910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840100b30 0x7ff840100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 -- 172.21.15.64:0/3913045910 >> 172.21.15.64:0/3913045910 conn(0x7ff8400fc7a0 msgr2=0x7ff8400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 -- 172.21.15.64:0/3913045910 shutdown_connections 2024-09-19T05:04:34.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.700+0000 7ff84774e640 1 -- 172.21.15.64:0/3913045910 wait complete. 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 Processor -- start 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 -- start start 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 0x7ff840073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840101450 0x7ff84006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840105f40 0x7ff84006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff840118470 con 0x7ff840100b30 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401182f0 con 0x7ff840105f40 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff84774e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401185f0 con 0x7ff840101450 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff8454ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 0x7ff840073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff8454ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 0x7ff840073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48232/0 (socket says 172.21.15.64:48232) 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff8454ea640 1 -- 172.21.15.64:0/3365290931 learned_addr learned my addr 172.21.15.64:0/3365290931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff844ce9640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840101450 0x7ff84006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:34.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.701+0000 7ff845ceb640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840105f40 0x7ff84006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8454ea640 1 -- 172.21.15.64:0/3365290931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840101450 msgr2=0x7ff84006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8454ea640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840101450 0x7ff84006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8454ea640 1 -- 172.21.15.64:0/3365290931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840105f40 msgr2=0x7ff84006c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8454ea640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840105f40 0x7ff84006c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8454ea640 1 -- 172.21.15.64:0/3365290931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84006ca40 con 0x7ff840100b30 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8454ea640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 0x7ff840073d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00ba50 tx=0x7ff83c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c019070 con 0x7ff840100b30 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff83c00cbe0 con 0x7ff840100b30 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0075d0 con 0x7ff840100b30 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401b3780 con 0x7ff840100b30 2024-09-19T05:04:34.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.702+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401b3c40 con 0x7ff840100b30 2024-09-19T05:04:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.703+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84006b070 con 0x7ff840100b30 2024-09-19T05:04:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.705+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff83c01c070 con 0x7ff840100b30 2024-09-19T05:04:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.705+0000 7ff8367fc640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff82c0a32d0 0x7ff82c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.705+0000 7ff844ce9640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff82c0a32d0 0x7ff82c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:34.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.705+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7ff83c0cefe0 con 0x7ff840100b30 2024-09-19T05:04:34.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.706+0000 7ff844ce9640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff82c0a32d0 0x7ff82c0a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff830031440 tx=0x7ff830005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:34.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.709+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff83c09b640 con 0x7ff840100b30 2024-09-19T05:04:34.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.881+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8400fe120 con 0x7ff82c0a32d0 2024-09-19T05:04:34.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.888+0000 7ff8367fc640 1 -- 172.21.15.64:0/3365290931 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258720 (secure 0 0 0) 0x7ff8400fe120 con 0x7ff82c0a32d0 2024-09-19T05:04:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:34.892 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff82c0a32d0 msgr2=0x7ff82c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff82c0a32d0 0x7ff82c0a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff830031440 tx=0x7ff830005e80 comp rx=0 tx=0).stop 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 msgr2=0x7ff840073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 0x7ff840073d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00ba50 tx=0x7ff83c00bf10 comp rx=0 tx=0).stop 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 shutdown_connections 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7ff82c0a32d0 0x7ff82c0a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840105f40 0x7ff84006c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840101450 0x7ff84006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 --2- 172.21.15.64:0/3365290931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff840100b30 0x7ff840073d10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 >> 172.21.15.64:0/3365290931 conn(0x7ff8400fc7a0 msgr2=0x7ff8400fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 shutdown_connections 2024-09-19T05:04:34.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:34.891+0000 7ff84774e640 1 -- 172.21.15.64:0/3365290931 wait complete. 2024-09-19T05:04:35.686 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-19T05:04:34.382960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648465,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":249,"num_read_kb":233,"num_write":336,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365692,"kb_used_data":12268,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576132,"statfs":{"total":1151940427776,"available":1151565959168,"internally_reserved":0,"allocated":12562432,"data_stored":7003089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52875,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":-6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":90,"num_write":30,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002763"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045762+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:04:20.045762+0000","last_peered":"2024-09-19T05:04:20.045762+0000","last_clean":"2024-09-19T05:04:20.045762+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:04:20.045762+0000","last_undegraded":"2024-09-19T05:04:20.045762+0000","last_fullsized":"2024-09-19T05:04:20.045762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042748+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:04:20.042748+0000","last_peered":"2024-09-19T05:04:20.042748+0000","last_clean":"2024-09-19T05:04:20.042748+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:04:20.042748+0000","last_undegraded":"2024-09-19T05:04:20.042748+0000","last_fullsized":"2024-09-19T05:04:20.042748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042862+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:04:20.042862+0000","last_peered":"2024-09-19T05:04:20.042862+0000","last_clean":"2024-09-19T05:04:20.042862+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:04:20.042862+0000","last_undegraded":"2024-09-19T05:04:20.042862+0000","last_fullsized":"2024-09-19T05:04:20.042862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218922+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:04:20.218922+0000","last_peered":"2024-09-19T05:04:20.218922+0000","last_clean":"2024-09-19T05:04:20.218922+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:04:20.218922+0000","last_undegraded":"2024-09-19T05:04:20.218922+0000","last_fullsized":"2024-09-19T05:04:20.218922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044933+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:04:20.044933+0000","last_peered":"2024-09-19T05:04:20.044933+0000","last_clean":"2024-09-19T05:04:20.044933+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:04:20.044933+0000","last_undegraded":"2024-09-19T05:04:20.044933+0000","last_fullsized":"2024-09-19T05:04:20.044933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046644+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:04:20.046644+0000","last_peered":"2024-09-19T05:04:20.046644+0000","last_clean":"2024-09-19T05:04:20.046644+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:04:20.046644+0000","last_undegraded":"2024-09-19T05:04:20.046644+0000","last_fullsized":"2024-09-19T05:04:20.046644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041281+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:04:20.041281+0000","last_peered":"2024-09-19T05:04:20.041281+0000","last_clean":"2024-09-19T05:04:20.041281+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:04:20.041281+0000","last_undegraded":"2024-09-19T05:04:20.041281+0000","last_fullsized":"2024-09-19T05:04:20.041281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045982+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:04:20.045982+0000","last_peered":"2024-09-19T05:04:20.045982+0000","last_clean":"2024-09-19T05:04:20.045982+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:04:20.045982+0000","last_undegraded":"2024-09-19T05:04:20.045982+0000","last_fullsized":"2024-09-19T05:04:20.045982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039683+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:04:20.039683+0000","last_peered":"2024-09-19T05:04:20.039683+0000","last_clean":"2024-09-19T05:04:20.039683+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:04:20.039683+0000","last_undegraded":"2024-09-19T05:04:20.039683+0000","last_fullsized":"2024-09-19T05:04:20.039683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043775+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:04:20.043775+0000","last_peered":"2024-09-19T05:04:20.043775+0000","last_clean":"2024-09-19T05:04:20.043775+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:04:20.043775+0000","last_undegraded":"2024-09-19T05:04:20.043775+0000","last_fullsized":"2024-09-19T05:04:20.043775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041842+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:04:20.041842+0000","last_peered":"2024-09-19T05:04:20.041842+0000","last_clean":"2024-09-19T05:04:20.041842+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:04:20.041842+0000","last_undegraded":"2024-09-19T05:04:20.041842+0000","last_fullsized":"2024-09-19T05:04:20.041842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219054+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:04:20.219054+0000","last_peered":"2024-09-19T05:04:20.219054+0000","last_clean":"2024-09-19T05:04:20.219054+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:04:20.219054+0000","last_undegraded":"2024-09-19T05:04:20.219054+0000","last_fullsized":"2024-09-19T05:04:20.219054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038234+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:04:20.038234+0000","last_peered":"2024-09-19T05:04:20.038234+0000","last_clean":"2024-09-19T05:04:20.038234+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:04:20.038234+0000","last_undegraded":"2024-09-19T05:04:20.038234+0000","last_fullsized":"2024-09-19T05:04:20.038234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042859+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:04:20.042859+0000","last_peered":"2024-09-19T05:04:20.042859+0000","last_clean":"2024-09-19T05:04:20.042859+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:04:20.042859+0000","last_undegraded":"2024-09-19T05:04:20.042859+0000","last_fullsized":"2024-09-19T05:04:20.042859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042616+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:04:20.042616+0000","last_peered":"2024-09-19T05:04:20.042616+0000","last_clean":"2024-09-19T05:04:20.042616+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:04:20.042616+0000","last_undegraded":"2024-09-19T05:04:20.042616+0000","last_fullsized":"2024-09-19T05:04:20.042616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039034+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:04:20.039034+0000","last_peered":"2024-09-19T05:04:20.039034+0000","last_clean":"2024-09-19T05:04:20.039034+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:04:20.039034+0000","last_undegraded":"2024-09-19T05:04:20.039034+0000","last_fullsized":"2024-09-19T05:04:20.039034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045863+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:04:20.045863+0000","last_peered":"2024-09-19T05:04:20.045863+0000","last_clean":"2024-09-19T05:04:20.045863+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:04:20.045863+0000","last_undegraded":"2024-09-19T05:04:20.045863+0000","last_fullsized":"2024-09-19T05:04:20.045863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040764+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:04:20.040764+0000","last_peered":"2024-09-19T05:04:20.040764+0000","last_clean":"2024-09-19T05:04:20.040764+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:04:20.040764+0000","last_undegraded":"2024-09-19T05:04:20.040764+0000","last_fullsized":"2024-09-19T05:04:20.040764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045627+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:04:20.045627+0000","last_peered":"2024-09-19T05:04:20.045627+0000","last_clean":"2024-09-19T05:04:20.045627+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:04:20.045627+0000","last_undegraded":"2024-09-19T05:04:20.045627+0000","last_fullsized":"2024-09-19T05:04:20.045627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045190+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:04:20.045190+0000","last_peered":"2024-09-19T05:04:20.045190+0000","last_clean":"2024-09-19T05:04:20.045190+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:04:20.045190+0000","last_undegraded":"2024-09-19T05:04:20.045190+0000","last_fullsized":"2024-09-19T05:04:20.045190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038280+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:04:20.038280+0000","last_peered":"2024-09-19T05:04:20.038280+0000","last_clean":"2024-09-19T05:04:20.038280+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:04:20.038280+0000","last_undegraded":"2024-09-19T05:04:20.038280+0000","last_fullsized":"2024-09-19T05:04:20.038280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039404+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:04:20.039404+0000","last_peered":"2024-09-19T05:04:20.039404+0000","last_clean":"2024-09-19T05:04:20.039404+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:04:20.039404+0000","last_undegraded":"2024-09-19T05:04:20.039404+0000","last_fullsized":"2024-09-19T05:04:20.039404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046472+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:04:20.046472+0000","last_peered":"2024-09-19T05:04:20.046472+0000","last_clean":"2024-09-19T05:04:20.046472+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:04:20.046472+0000","last_undegraded":"2024-09-19T05:04:20.046472+0000","last_fullsized":"2024-09-19T05:04:20.046472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040224+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:04:20.040224+0000","last_peered":"2024-09-19T05:04:20.040224+0000","last_clean":"2024-09-19T05:04:20.040224+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:04:20.040224+0000","last_undegraded":"2024-09-19T05:04:20.040224+0000","last_fullsized":"2024-09-19T05:04:20.040224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220980+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:04:20.220980+0000","last_peered":"2024-09-19T05:04:20.220980+0000","last_clean":"2024-09-19T05:04:20.220980+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:04:20.220980+0000","last_undegraded":"2024-09-19T05:04:20.220980+0000","last_fullsized":"2024-09-19T05:04:20.220980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044307+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:04:20.044307+0000","last_peered":"2024-09-19T05:04:20.044307+0000","last_clean":"2024-09-19T05:04:20.044307+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:04:20.044307+0000","last_undegraded":"2024-09-19T05:04:20.044307+0000","last_fullsized":"2024-09-19T05:04:20.044307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046219+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:04:20.046219+0000","last_peered":"2024-09-19T05:04:20.046219+0000","last_clean":"2024-09-19T05:04:20.046219+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:04:20.046219+0000","last_undegraded":"2024-09-19T05:04:20.046219+0000","last_fullsized":"2024-09-19T05:04:20.046219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038227+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:04:20.038227+0000","last_peered":"2024-09-19T05:04:20.038227+0000","last_clean":"2024-09-19T05:04:20.038227+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:04:20.038227+0000","last_undegraded":"2024-09-19T05:04:20.038227+0000","last_fullsized":"2024-09-19T05:04:20.038227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042890+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:04:20.042890+0000","last_peered":"2024-09-19T05:04:20.042890+0000","last_clean":"2024-09-19T05:04:20.042890+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:04:20.042890+0000","last_undegraded":"2024-09-19T05:04:20.042890+0000","last_fullsized":"2024-09-19T05:04:20.042890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044794+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:04:20.044794+0000","last_peered":"2024-09-19T05:04:20.044794+0000","last_clean":"2024-09-19T05:04:20.044794+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:04:20.044794+0000","last_undegraded":"2024-09-19T05:04:20.044794+0000","last_fullsized":"2024-09-19T05:04:20.044794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044665+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:04:20.044665+0000","last_peered":"2024-09-19T05:04:20.044665+0000","last_clean":"2024-09-19T05:04:20.044665+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:04:20.044665+0000","last_undegraded":"2024-09-19T05:04:20.044665+0000","last_fullsized":"2024-09-19T05:04:20.044665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045176+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:04:20.045176+0000","last_peered":"2024-09-19T05:04:20.045176+0000","last_clean":"2024-09-19T05:04:20.045176+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:04:20.045176+0000","last_undegraded":"2024-09-19T05:04:20.045176+0000","last_fullsized":"2024-09-19T05:04:20.045176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045192+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:04:20.045192+0000","last_peered":"2024-09-19T05:04:20.045192+0000","last_clean":"2024-09-19T05:04:20.045192+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:04:20.045192+0000","last_undegraded":"2024-09-19T05:04:20.045192+0000","last_fullsized":"2024-09-19T05:04:20.045192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:32.043550+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:04:32.043550+0000","last_peered":"2024-09-19T05:04:32.043550+0000","last_clean":"2024-09-19T05:04:32.043550+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:04:32.043550+0000","last_undegraded":"2024-09-19T05:04:32.043550+0000","last_fullsized":"2024-09-19T05:04:32.043550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220294+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:04:20.220294+0000","last_peered":"2024-09-19T05:04:20.220294+0000","last_clean":"2024-09-19T05:04:20.220294+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:04:20.220294+0000","last_undegraded":"2024-09-19T05:04:20.220294+0000","last_fullsized":"2024-09-19T05:04:20.220294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043292+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:04:20.043292+0000","last_peered":"2024-09-19T05:04:20.043292+0000","last_clean":"2024-09-19T05:04:20.043292+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:04:20.043292+0000","last_undegraded":"2024-09-19T05:04:20.043292+0000","last_fullsized":"2024-09-19T05:04:20.043292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037995+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:04:20.037995+0000","last_peered":"2024-09-19T05:04:20.037995+0000","last_clean":"2024-09-19T05:04:20.037995+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:04:20.037995+0000","last_undegraded":"2024-09-19T05:04:20.037995+0000","last_fullsized":"2024-09-19T05:04:20.037995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046054+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:04:20.046054+0000","last_peered":"2024-09-19T05:04:20.046054+0000","last_clean":"2024-09-19T05:04:20.046054+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:04:20.046054+0000","last_undegraded":"2024-09-19T05:04:20.046054+0000","last_fullsized":"2024-09-19T05:04:20.046054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040479+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:04:20.040479+0000","last_peered":"2024-09-19T05:04:20.040479+0000","last_clean":"2024-09-19T05:04:20.040479+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:04:20.040479+0000","last_undegraded":"2024-09-19T05:04:20.040479+0000","last_fullsized":"2024-09-19T05:04:20.040479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041190+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:04:20.041190+0000","last_peered":"2024-09-19T05:04:20.041190+0000","last_clean":"2024-09-19T05:04:20.041190+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:04:20.041190+0000","last_undegraded":"2024-09-19T05:04:20.041190+0000","last_fullsized":"2024-09-19T05:04:20.041190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:32.043103+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:04:32.043103+0000","last_peered":"2024-09-19T05:04:32.043103+0000","last_clean":"2024-09-19T05:04:32.043103+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:04:32.043103+0000","last_undegraded":"2024-09-19T05:04:32.043103+0000","last_fullsized":"2024-09-19T05:04:32.043103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039611+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:04:20.039611+0000","last_peered":"2024-09-19T05:04:20.039611+0000","last_clean":"2024-09-19T05:04:20.039611+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:04:20.039611+0000","last_undegraded":"2024-09-19T05:04:20.039611+0000","last_fullsized":"2024-09-19T05:04:20.039611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.223302+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:04:20.223302+0000","last_peered":"2024-09-19T05:04:20.223302+0000","last_clean":"2024-09-19T05:04:20.223302+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:04:20.223302+0000","last_undegraded":"2024-09-19T05:04:20.223302+0000","last_fullsized":"2024-09-19T05:04:20.223302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044799+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:04:20.044799+0000","last_peered":"2024-09-19T05:04:20.044799+0000","last_clean":"2024-09-19T05:04:20.044799+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:04:20.044799+0000","last_undegraded":"2024-09-19T05:04:20.044799+0000","last_fullsized":"2024-09-19T05:04:20.044799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040554+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:04:20.040554+0000","last_peered":"2024-09-19T05:04:20.040554+0000","last_clean":"2024-09-19T05:04:20.040554+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:04:20.040554+0000","last_undegraded":"2024-09-19T05:04:20.040554+0000","last_fullsized":"2024-09-19T05:04:20.040554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039165+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:04:20.039165+0000","last_peered":"2024-09-19T05:04:20.039165+0000","last_clean":"2024-09-19T05:04:20.039165+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:04:20.039165+0000","last_undegraded":"2024-09-19T05:04:20.039165+0000","last_fullsized":"2024-09-19T05:04:20.039165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'26","reported_seq":86,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:33.049308+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:04:33.049308+0000","last_peered":"2024-09-19T05:04:33.049308+0000","last_clean":"2024-09-19T05:04:33.049308+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:04:33.049308+0000","last_undegraded":"2024-09-19T05:04:33.049308+0000","last_fullsized":"2024-09-19T05:04:33.049308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53860,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":67,"num_write":27,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044465+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:04:20.044465+0000","last_peered":"2024-09-19T05:04:20.044465+0000","last_clean":"2024-09-19T05:04:20.044465+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:04:20.044465+0000","last_undegraded":"2024-09-19T05:04:20.044465+0000","last_fullsized":"2024-09-19T05:04:20.044465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.058790+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:04:20.058790+0000","last_peered":"2024-09-19T05:04:20.058790+0000","last_clean":"2024-09-19T05:04:20.058790+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:04:20.058790+0000","last_undegraded":"2024-09-19T05:04:20.058790+0000","last_fullsized":"2024-09-19T05:04:20.058790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038290+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:04:20.038290+0000","last_peered":"2024-09-19T05:04:20.038290+0000","last_clean":"2024-09-19T05:04:20.038290+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:04:20.038290+0000","last_undegraded":"2024-09-19T05:04:20.038290+0000","last_fullsized":"2024-09-19T05:04:20.038290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040388+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:04:20.040388+0000","last_peered":"2024-09-19T05:04:20.040388+0000","last_clean":"2024-09-19T05:04:20.040388+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:04:20.040388+0000","last_undegraded":"2024-09-19T05:04:20.040388+0000","last_fullsized":"2024-09-19T05:04:20.040388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041839+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:04:20.041839+0000","last_peered":"2024-09-19T05:04:20.041839+0000","last_clean":"2024-09-19T05:04:20.041839+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:04:20.041839+0000","last_undegraded":"2024-09-19T05:04:20.041839+0000","last_fullsized":"2024-09-19T05:04:20.041839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'3","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:28.038521+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:04:28.038521+0000","last_peered":"2024-09-19T05:04:28.038521+0000","last_clean":"2024-09-19T05:04:28.038521+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:04:28.038521+0000","last_undegraded":"2024-09-19T05:04:28.038521+0000","last_fullsized":"2024-09-19T05:04:28.038521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041986+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:04:20.041986+0000","last_peered":"2024-09-19T05:04:20.041986+0000","last_clean":"2024-09-19T05:04:20.041986+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:04:20.041986+0000","last_undegraded":"2024-09-19T05:04:20.041986+0000","last_fullsized":"2024-09-19T05:04:20.041986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044042+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:04:20.044042+0000","last_peered":"2024-09-19T05:04:20.044042+0000","last_clean":"2024-09-19T05:04:20.044042+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:04:20.044042+0000","last_undegraded":"2024-09-19T05:04:20.044042+0000","last_fullsized":"2024-09-19T05:04:20.044042+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220988+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:04:20.220988+0000","last_peered":"2024-09-19T05:04:20.220988+0000","last_clean":"2024-09-19T05:04:20.220988+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:04:20.220988+0000","last_undegraded":"2024-09-19T05:04:20.220988+0000","last_fullsized":"2024-09-19T05:04:20.220988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044689+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:04:20.044689+0000","last_peered":"2024-09-19T05:04:20.044689+0000","last_clean":"2024-09-19T05:04:20.044689+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:04:20.044689+0000","last_undegraded":"2024-09-19T05:04:20.044689+0000","last_fullsized":"2024-09-19T05:04:20.044689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045469+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:04:20.045469+0000","last_peered":"2024-09-19T05:04:20.045469+0000","last_clean":"2024-09-19T05:04:20.045469+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:04:20.045469+0000","last_undegraded":"2024-09-19T05:04:20.045469+0000","last_fullsized":"2024-09-19T05:04:20.045469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038412+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:04:20.038412+0000","last_peered":"2024-09-19T05:04:20.038412+0000","last_clean":"2024-09-19T05:04:20.038412+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:04:20.038412+0000","last_undegraded":"2024-09-19T05:04:20.038412+0000","last_fullsized":"2024-09-19T05:04:20.038412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041819+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:04:20.041819+0000","last_peered":"2024-09-19T05:04:20.041819+0000","last_clean":"2024-09-19T05:04:20.041819+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:04:20.041819+0000","last_undegraded":"2024-09-19T05:04:20.041819+0000","last_fullsized":"2024-09-19T05:04:20.041819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040558+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:04:20.040558+0000","last_peered":"2024-09-19T05:04:20.040558+0000","last_clean":"2024-09-19T05:04:20.040558+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:04:20.040558+0000","last_undegraded":"2024-09-19T05:04:20.040558+0000","last_fullsized":"2024-09-19T05:04:20.040558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041588+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:04:20.041588+0000","last_peered":"2024-09-19T05:04:20.041588+0000","last_clean":"2024-09-19T05:04:20.041588+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:04:20.041588+0000","last_undegraded":"2024-09-19T05:04:20.041588+0000","last_fullsized":"2024-09-19T05:04:20.041588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044395+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:04:20.044395+0000","last_peered":"2024-09-19T05:04:20.044395+0000","last_clean":"2024-09-19T05:04:20.044395+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:04:20.044395+0000","last_undegraded":"2024-09-19T05:04:20.044395+0000","last_fullsized":"2024-09-19T05:04:20.044395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041954+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:04:20.041954+0000","last_peered":"2024-09-19T05:04:20.041954+0000","last_clean":"2024-09-19T05:04:20.041954+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:04:20.041954+0000","last_undegraded":"2024-09-19T05:04:20.041954+0000","last_fullsized":"2024-09-19T05:04:20.041954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":343,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042103+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:04:20.042103+0000","last_peered":"2024-09-19T05:04:20.042103+0000","last_clean":"2024-09-19T05:04:20.042103+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:04:20.042103+0000","last_undegraded":"2024-09-19T05:04:20.042103+0000","last_fullsized":"2024-09-19T05:04:20.042103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218980+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:04:20.218980+0000","last_peered":"2024-09-19T05:04:20.218980+0000","last_clean":"2024-09-19T05:04:20.218980+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:04:20.218980+0000","last_undegraded":"2024-09-19T05:04:20.218980+0000","last_fullsized":"2024-09-19T05:04:20.218980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046490+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:04:20.046490+0000","last_peered":"2024-09-19T05:04:20.046490+0000","last_clean":"2024-09-19T05:04:20.046490+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:04:20.046490+0000","last_undegraded":"2024-09-19T05:04:20.046490+0000","last_fullsized":"2024-09-19T05:04:20.046490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.048907+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:04:20.048907+0000","last_peered":"2024-09-19T05:04:20.048907+0000","last_clean":"2024-09-19T05:04:20.048907+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:04:20.048907+0000","last_undegraded":"2024-09-19T05:04:20.048907+0000","last_fullsized":"2024-09-19T05:04:20.048907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037984+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:04:20.037984+0000","last_peered":"2024-09-19T05:04:20.037984+0000","last_clean":"2024-09-19T05:04:20.037984+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:04:20.037984+0000","last_undegraded":"2024-09-19T05:04:20.037984+0000","last_fullsized":"2024-09-19T05:04:20.037984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:29.039331+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:04:29.039331+0000","last_peered":"2024-09-19T05:04:29.039331+0000","last_clean":"2024-09-19T05:04:29.039331+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:04:29.039331+0000","last_undegraded":"2024-09-19T05:04:29.039331+0000","last_fullsized":"2024-09-19T05:04:29.039331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040362+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:04:20.040362+0000","last_peered":"2024-09-19T05:04:20.040362+0000","last_clean":"2024-09-19T05:04:20.040362+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:04:20.040362+0000","last_undegraded":"2024-09-19T05:04:20.040362+0000","last_fullsized":"2024-09-19T05:04:20.040362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041750+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:04:20.041750+0000","last_peered":"2024-09-19T05:04:20.041750+0000","last_clean":"2024-09-19T05:04:20.041750+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:04:20.041750+0000","last_undegraded":"2024-09-19T05:04:20.041750+0000","last_fullsized":"2024-09-19T05:04:20.041750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043344+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:04:20.043344+0000","last_peered":"2024-09-19T05:04:20.043344+0000","last_clean":"2024-09-19T05:04:20.043344+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:04:20.043344+0000","last_undegraded":"2024-09-19T05:04:20.043344+0000","last_fullsized":"2024-09-19T05:04:20.043344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040735+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:04:20.040735+0000","last_peered":"2024-09-19T05:04:20.040735+0000","last_clean":"2024-09-19T05:04:20.040735+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:04:20.040735+0000","last_undegraded":"2024-09-19T05:04:20.040735+0000","last_fullsized":"2024-09-19T05:04:20.040735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043291+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:04:20.043291+0000","last_peered":"2024-09-19T05:04:20.043291+0000","last_clean":"2024-09-19T05:04:20.043291+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:04:20.043291+0000","last_undegraded":"2024-09-19T05:04:20.043291+0000","last_fullsized":"2024-09-19T05:04:20.043291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044730+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:04:20.044730+0000","last_peered":"2024-09-19T05:04:20.044730+0000","last_clean":"2024-09-19T05:04:20.044730+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:04:20.044730+0000","last_undegraded":"2024-09-19T05:04:20.044730+0000","last_fullsized":"2024-09-19T05:04:20.044730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038415+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:04:20.038415+0000","last_peered":"2024-09-19T05:04:20.038415+0000","last_clean":"2024-09-19T05:04:20.038415+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:04:20.038415+0000","last_undegraded":"2024-09-19T05:04:20.038415+0000","last_fullsized":"2024-09-19T05:04:20.038415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:26.976440+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:04:26.976440+0000","last_peered":"2024-09-19T05:04:26.976440+0000","last_clean":"2024-09-19T05:04:26.976440+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:04:26.976440+0000","last_undegraded":"2024-09-19T05:04:26.976440+0000","last_fullsized":"2024-09-19T05:04:26.976440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046734+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:04:20.046734+0000","last_peered":"2024-09-19T05:04:20.046734+0000","last_clean":"2024-09-19T05:04:20.046734+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:04:20.046734+0000","last_undegraded":"2024-09-19T05:04:20.046734+0000","last_fullsized":"2024-09-19T05:04:20.046734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219379+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:04:20.219379+0000","last_peered":"2024-09-19T05:04:20.219379+0000","last_clean":"2024-09-19T05:04:20.219379+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:04:20.219379+0000","last_undegraded":"2024-09-19T05:04:20.219379+0000","last_fullsized":"2024-09-19T05:04:20.219379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.047866+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:04:20.047866+0000","last_peered":"2024-09-19T05:04:20.047866+0000","last_clean":"2024-09-19T05:04:20.047866+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:04:20.047866+0000","last_undegraded":"2024-09-19T05:04:20.047866+0000","last_fullsized":"2024-09-19T05:04:20.047866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039180+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:04:20.039180+0000","last_peered":"2024-09-19T05:04:20.039180+0000","last_clean":"2024-09-19T05:04:20.039180+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:04:20.039180+0000","last_undegraded":"2024-09-19T05:04:20.039180+0000","last_fullsized":"2024-09-19T05:04:20.039180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043811+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:04:20.043811+0000","last_peered":"2024-09-19T05:04:20.043811+0000","last_clean":"2024-09-19T05:04:20.043811+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:04:20.043811+0000","last_undegraded":"2024-09-19T05:04:20.043811+0000","last_fullsized":"2024-09-19T05:04:20.043811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046197+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:04:20.046197+0000","last_peered":"2024-09-19T05:04:20.046197+0000","last_clean":"2024-09-19T05:04:20.046197+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:04:20.046197+0000","last_undegraded":"2024-09-19T05:04:20.046197+0000","last_fullsized":"2024-09-19T05:04:20.046197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045294+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:04:20.045294+0000","last_peered":"2024-09-19T05:04:20.045294+0000","last_clean":"2024-09-19T05:04:20.045294+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:04:20.045294+0000","last_undegraded":"2024-09-19T05:04:20.045294+0000","last_fullsized":"2024-09-19T05:04:20.045294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044905+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:04:20.044905+0000","last_peered":"2024-09-19T05:04:20.044905+0000","last_clean":"2024-09-19T05:04:20.044905+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:04:20.044905+0000","last_undegraded":"2024-09-19T05:04:20.044905+0000","last_fullsized":"2024-09-19T05:04:20.044905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046267+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:04:20.046267+0000","last_peered":"2024-09-19T05:04:20.046267+0000","last_clean":"2024-09-19T05:04:20.046267+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:04:20.046267+0000","last_undegraded":"2024-09-19T05:04:20.046267+0000","last_fullsized":"2024-09-19T05:04:20.046267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044834+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:04:20.044834+0000","last_peered":"2024-09-19T05:04:20.044834+0000","last_clean":"2024-09-19T05:04:20.044834+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:04:20.044834+0000","last_undegraded":"2024-09-19T05:04:20.044834+0000","last_fullsized":"2024-09-19T05:04:20.044834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041613+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:04:20.041613+0000","last_peered":"2024-09-19T05:04:20.041613+0000","last_clean":"2024-09-19T05:04:20.041613+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:04:20.041613+0000","last_undegraded":"2024-09-19T05:04:20.041613+0000","last_fullsized":"2024-09-19T05:04:20.041613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040445+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:04:20.040445+0000","last_peered":"2024-09-19T05:04:20.040445+0000","last_clean":"2024-09-19T05:04:20.040445+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:04:20.040445+0000","last_undegraded":"2024-09-19T05:04:20.040445+0000","last_fullsized":"2024-09-19T05:04:20.040445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044233+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:04:20.044233+0000","last_peered":"2024-09-19T05:04:20.044233+0000","last_clean":"2024-09-19T05:04:20.044233+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:04:20.044233+0000","last_undegraded":"2024-09-19T05:04:20.044233+0000","last_fullsized":"2024-09-19T05:04:20.044233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039172+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:04:20.039172+0000","last_peered":"2024-09-19T05:04:20.039172+0000","last_clean":"2024-09-19T05:04:20.039172+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:04:20.039172+0000","last_undegraded":"2024-09-19T05:04:20.039172+0000","last_fullsized":"2024-09-19T05:04:20.039172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.052380+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:04:20.052380+0000","last_peered":"2024-09-19T05:04:20.052380+0000","last_clean":"2024-09-19T05:04:20.052380+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:04:20.052380+0000","last_undegraded":"2024-09-19T05:04:20.052380+0000","last_fullsized":"2024-09-19T05:04:20.052380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043121+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:04:20.043121+0000","last_peered":"2024-09-19T05:04:20.043121+0000","last_clean":"2024-09-19T05:04:20.043121+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:04:20.043121+0000","last_undegraded":"2024-09-19T05:04:20.043121+0000","last_fullsized":"2024-09-19T05:04:20.043121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046887+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:04:20.046887+0000","last_peered":"2024-09-19T05:04:20.046887+0000","last_clean":"2024-09-19T05:04:20.046887+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:04:20.046887+0000","last_undegraded":"2024-09-19T05:04:20.046887+0000","last_fullsized":"2024-09-19T05:04:20.046887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046641+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:04:20.046641+0000","last_peered":"2024-09-19T05:04:20.046641+0000","last_clean":"2024-09-19T05:04:20.046641+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:04:20.046641+0000","last_undegraded":"2024-09-19T05:04:20.046641+0000","last_fullsized":"2024-09-19T05:04:20.046641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041192+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:04:20.041192+0000","last_peered":"2024-09-19T05:04:20.041192+0000","last_clean":"2024-09-19T05:04:20.041192+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:04:20.041192+0000","last_undegraded":"2024-09-19T05:04:20.041192+0000","last_fullsized":"2024-09-19T05:04:20.041192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57863,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":113,"num_read_kb":114,"num_write":153,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":165075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678033,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":422800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1068107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070490,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":423002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":422922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528357,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1014854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888111,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32640,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712512,"statfs":{"total":95995035648,"available":95961612288,"internally_reserved":0,"allocated":1490944,"data_stored":1014538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":422828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247863,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":422702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444093,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":473496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":471787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":423662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":51312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:04:35.690 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:04:35.691 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T05:04:35.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T05:04:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:36.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:36 smithi064 ceph-mon[30960]: pgmap v287: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:36.388 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:36 smithi064 ceph-mon[30960]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 -- 172.21.15.64:0/1416734546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 msgr2=0x7f7b94100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/1416734546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94100ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009900 tx=0x7f7b8402a410 comp rx=0 tx=0).stop 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 -- 172.21.15.64:0/1416734546 shutdown_connections 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/1416734546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b94102690 0x7f7b94108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/1416734546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b94101cd0 0x7f7b94102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/1416734546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94100ed0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.393+0000 7f7b98d9e640 1 -- 172.21.15.64:0/1416734546 >> 172.21.15.64:0/1416734546 conn(0x7f7b940fc280 msgr2=0x7f7b940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:36.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.394+0000 7f7b98d9e640 1 -- 172.21.15.64:0/1416734546 shutdown_connections 2024-09-19T05:04:36.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.394+0000 7f7b98d9e640 1 -- 172.21.15.64:0/1416734546 wait complete. 2024-09-19T05:04:36.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.394+0000 7f7b98d9e640 1 Processor -- start 2024-09-19T05:04:36.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 -- start start 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b94101cd0 0x7f7b94198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b94102690 0x7f7b9419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9410d730 con 0x7f7b94100ad0 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9410d5b0 con 0x7f7b94102690 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b98d9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9410d8b0 con 0x7f7b94101cd0 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48244/0 (socket says 172.21.15.64:48244) 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 -- 172.21.15.64:0/3257066150 learned_addr learned my addr 172.21.15.64:0/3257066150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b91d9b640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b94101cd0 0x7f7b94198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b92d9d640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b94102690 0x7f7b9419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:36.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 -- 172.21.15.64:0/3257066150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b94101cd0 msgr2=0x7f7b94198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b94101cd0 0x7f7b94198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 -- 172.21.15.64:0/3257066150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b94102690 msgr2=0x7f7b9419f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b94102690 0x7f7b9419f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 -- 172.21.15.64:0/3257066150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9419fa30 con 0x7f7b94100ad0 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.395+0000 7f7b9259c640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94197e00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009230 tx=0x7f7b84004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.396+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8403f030 con 0x7f7b94100ad0 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.396+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b84003cb0 con 0x7f7b94100ad0 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.396+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9419fcc0 con 0x7f7b94100ad0 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.396+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84035070 con 0x7f7b94100ad0 2024-09-19T05:04:36.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.396+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b941a01a0 con 0x7f7b94100ad0 2024-09-19T05:04:36.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.397+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b94107f20 con 0x7f7b94100ad0 2024-09-19T05:04:36.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.398+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7b84042030 con 0x7f7b94100ad0 2024-09-19T05:04:36.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.399+0000 7f7b837fe640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b7c0a3300 0x7f7b7c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:36.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.399+0000 7f7b91d9b640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b7c0a3300 0x7f7b7c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:36.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.399+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f7b840f0120 con 0x7f7b94100ad0 2024-09-19T05:04:36.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.399+0000 7f7b91d9b640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b7c0a3300 0x7f7b7c0a57c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b94199320 tx=0x7f7b8800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:36.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.403+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b840bccc0 con 0x7f7b94100ad0 2024-09-19T05:04:36.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:36 smithi103 ceph-mon[34446]: pgmap v287: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:36.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:36 smithi103 ceph-mon[34446]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:36.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:36 smithi174 ceph-mon[34488]: pgmap v287: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:36.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:36 smithi174 ceph-mon[34488]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:36.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.644+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7b9402d7c0 con 0x7f7b94100ad0 2024-09-19T05:04:36.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.647+0000 7f7b837fe640 1 -- 172.21.15.64:0/3257066150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7b840bc5c0 con 0x7f7b94100ad0 2024-09-19T05:04:36.649 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-09-19T05:04:36.649 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQDksOtmfIWBJhAAcRNUGYBDVkkHW8zGSecnTg== 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b7c0a3300 msgr2=0x7f7b7c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b7c0a3300 0x7f7b7c0a57c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b94199320 tx=0x7f7b8800a250 comp rx=0 tx=0).stop 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 msgr2=0x7f7b94197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94197e00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009230 tx=0x7f7b84004340 comp rx=0 tx=0).stop 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 shutdown_connections 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7b7c0a3300 0x7f7b7c0a57c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b94102690 0x7f7b9419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b94101cd0 0x7f7b94198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 --2- 172.21.15.64:0/3257066150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b94100ad0 0x7f7b94197e00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.649+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 >> 172.21.15.64:0/3257066150 conn(0x7f7b940fc280 msgr2=0x7f7b941050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.650+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 shutdown_connections 2024-09-19T05:04:36.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:36.650+0000 7f7b98d9e640 1 -- 172.21.15.64:0/3257066150 wait complete. 2024-09-19T05:04:37.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:37 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3257066150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:04:37.216 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:37 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3257066150' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:04:37.244 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T05:04:37.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T05:04:37.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T05:04:37.282 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T05:04:37.282 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T05:04:37.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T05:04:37.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:37 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3257066150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:04:37.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:37 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3257066150' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:04:37.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:37 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3257066150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:04:37.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:37 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3257066150' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:04:37.578 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:38.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.018+0000 7f2a00a90640 1 -- 172.21.15.64:0/4204632501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc101280 msgr2=0x7f29fc10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.018+0000 7f2a00a90640 1 --2- 172.21.15.64:0/4204632501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc101280 0x7f29fc10df40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009900 tx=0x7f29f002a410 comp rx=0 tx=0).stop 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 -- 172.21.15.64:0/4204632501 shutdown_connections 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 --2- 172.21.15.64:0/4204632501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc10e480 0x7f29fc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 --2- 172.21.15.64:0/4204632501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc101280 0x7f29fc10df40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 --2- 172.21.15.64:0/4204632501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc1008d0 0x7f29fc100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 -- 172.21.15.64:0/4204632501 >> 172.21.15.64:0/4204632501 conn(0x7f29fc0fc7a0 msgr2=0x7f29fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 -- 172.21.15.64:0/4204632501 shutdown_connections 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 -- 172.21.15.64:0/4204632501 wait complete. 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 Processor -- start 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.019+0000 7f2a00a90640 1 -- start start 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc1008d0 0x7f29fc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc101280 0x7f29fc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 0x7f29fc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc114060 con 0x7f29fc10e480 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc113ee0 con 0x7f29fc101280 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1141e0 con 0x7f29fc1008d0 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 0x7f29fc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 0x7f29fc19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48264/0 (socket says 172.21.15.64:48264) 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 -- 172.21.15.64:0/3355549506 learned_addr learned my addr 172.21.15.64:0/3355549506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29f9d9b640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc101280 0x7f29fc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 -- 172.21.15.64:0/3355549506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc1008d0 msgr2=0x7f29fc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fa59c640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc1008d0 0x7f29fc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc1008d0 0x7f29fc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 -- 172.21.15.64:0/3355549506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc101280 msgr2=0x7f29fc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc101280 0x7f29fc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 -- 172.21.15.64:0/3355549506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc19f570 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29f9d9b640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc101280 0x7f29fc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29fad9d640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 0x7f29fc19ee90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29f400eb10 tx=0x7f29f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004b20 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29f4004cc0 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc19f860 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4010720 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.020+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1a7140 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.022+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc100cb0 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.024+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f29f4005020 con 0x7f29fc10e480 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.024+0000 7f29eb7fe640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f29e40a32d0 0x7f29e40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.024+0000 7f29fa59c640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f29e40a32d0 0x7f29e40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.024+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f29f40d0310 con 0x7f29fc10e480 2024-09-19T05:04:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.025+0000 7f29fa59c640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f29e40a32d0 0x7f29e40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0058c0 tx=0x7f29ec005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.027+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29f409ceb0 con 0x7f29fc10e480 2024-09-19T05:04:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.255+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f29fc109080 con 0x7f29fc10e480 2024-09-19T05:04:38.257 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:38 smithi064 ceph-mon[30960]: pgmap v288: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 KiB/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.261+0000 7f29eb7fe640 1 -- 172.21.15.64:0/3355549506 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264049 (secure 0 0 0) 0x7f29f409c7b0 con 0x7f29fc10e480 2024-09-19T05:04:38.265 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f29e40a32d0 msgr2=0x7f29e40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f29e40a32d0 0x7f29e40a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0058c0 tx=0x7f29ec005c60 comp rx=0 tx=0).stop 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 msgr2=0x7f29fc19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 0x7f29fc19ee90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29f400eb10 tx=0x7f29f400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 shutdown_connections 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f29e40a32d0 0x7f29e40a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29fc10e480 0x7f29fc19ee90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc101280 0x7f29fc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 --2- 172.21.15.64:0/3355549506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc1008d0 0x7f29fc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 >> 172.21.15.64:0/3355549506 conn(0x7f29fc0fc7a0 msgr2=0x7f29fc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 shutdown_connections 2024-09-19T05:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:38.264+0000 7f2a00a90640 1 -- 172.21.15.64:0/3355549506 wait complete. 2024-09-19T05:04:38.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:38 smithi103 ceph-mon[34446]: pgmap v288: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 KiB/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:38 smithi174 ceph-mon[34488]: pgmap v288: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.0 KiB/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T05:04:38.999 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":3907259856},{"type":"v1","addr":"172.21.15.64:6801","nonce":3907259856}]},"active_addr":"172.21.15.64:6801/3907259856","active_change":"2024-09-19T04:57:18.300065+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1076683694}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":868126904}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2343862077}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2000236557}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4266527585}]}]} 2024-09-19T05:04:39.005 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T05:04:39.005 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T05:04:39.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:39.266 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:39.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:39 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3355549506' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:04:39.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:39 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3355549506' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:04:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:39 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3355549506' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 -- 172.21.15.64:0/1716371643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 msgr2=0x7f959810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 --2- 172.21.15.64:0/1716371643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f959810e610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f959002a480 comp rx=0 tx=0).stop 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 -- 172.21.15.64:0/1716371643 shutdown_connections 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 --2- 172.21.15.64:0/1716371643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f959810e610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 --2- 172.21.15.64:0/1716371643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9598103660 0x7f959810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 --2- 172.21.15.64:0/1716371643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9598102d40 0x7f9598103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 -- 172.21.15.64:0/1716371643 >> 172.21.15.64:0/1716371643 conn(0x7f95980fc7a0 msgr2=0x7f95980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 -- 172.21.15.64:0/1716371643 shutdown_connections 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.725+0000 7f959d07c640 1 -- 172.21.15.64:0/1716371643 wait complete. 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 Processor -- start 2024-09-19T05:04:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 -- start start 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9598102d40 0x7f95981009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9598103660 0x7f9598100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f9598101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111e20 con 0x7f959810c170 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111ca0 con 0x7f9598102d40 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959d07c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111fa0 con 0x7f9598103660 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f9598101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f9598101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48276/0 (socket says 172.21.15.64:48276) 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959759e640 1 -- 172.21.15.64:0/3730091132 learned_addr learned my addr 172.21.15.64:0/3730091132 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f959659c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9598103660 0x7f9598100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.726+0000 7f9596d9d640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9598102d40 0x7f95981009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959759e640 1 -- 172.21.15.64:0/3730091132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9598103660 msgr2=0x7f9598100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959759e640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9598103660 0x7f9598100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959759e640 1 -- 172.21.15.64:0/3730091132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9598102d40 msgr2=0x7f95981009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959759e640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9598102d40 0x7f95981009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959759e640 1 -- 172.21.15.64:0/3730091132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95981a67c0 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959759e640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f9598101420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9590004770 tx=0x7f9590004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959003f030 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981a6a50 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95900318e0 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035070 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.727+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981a6f80 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.728+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9598103120 con 0x7f959810c170 2024-09-19T05:04:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.729+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9590042030 con 0x7f959810c170 2024-09-19T05:04:39.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.730+0000 7f958ffff640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f95840a32d0 0x7f95840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.730+0000 7f9596d9d640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f95840a32d0 0x7f95840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.730+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f95900f0320 con 0x7f959810c170 2024-09-19T05:04:39.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.731+0000 7f9596d9d640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f95840a32d0 0x7f95840a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9588009bd0 tx=0x7f9588005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:39.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.733+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f959003bca0 con 0x7f959810c170 2024-09-19T05:04:39.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.909+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f959802d7c0 con 0x7f959810c170 2024-09-19T05:04:39.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.910+0000 7f958ffff640 1 -- 172.21.15.64:0/3730091132 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22821 (secure 0 0 0) 0x7f95900bc870 con 0x7f959810c170 2024-09-19T05:04:39.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:39.912 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:20.028795+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f95840a32d0 msgr2=0x7f95840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f95840a32d0 0x7f95840a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9588009bd0 tx=0x7f9588005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 msgr2=0x7f9598101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f9598101420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9590004770 tx=0x7f9590004270 comp rx=0 tx=0).stop 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 shutdown_connections 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f95840a32d0 0x7f95840a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f959810c170 0x7f9598101420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.912+0000 7f959d07c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9598103660 0x7f9598100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.913+0000 7f959d07c640 1 --2- 172.21.15.64:0/3730091132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9598102d40 0x7f95981009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.913+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 >> 172.21.15.64:0/3730091132 conn(0x7f95980fc7a0 msgr2=0x7f95980fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.913+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 shutdown_connections 2024-09-19T05:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:39.913+0000 7f959d07c640 1 -- 172.21.15.64:0/3730091132 wait complete. 2024-09-19T05:04:40.282 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:40 smithi064 ceph-mon[30960]: pgmap v289: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T05:04:40.282 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:40 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3730091132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:40.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:40 smithi103 ceph-mon[34446]: pgmap v289: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T05:04:40.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:40 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3730091132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:40.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:40 smithi174 ceph-mon[34488]: pgmap v289: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-09-19T05:04:40.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:40 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3730091132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:40.706 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T05:04:40.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:40.965 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 -- 172.21.15.64:0/1110332181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c106f10 msgr2=0x7f7a5c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 --2- 172.21.15.64:0/1110332181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c106f10 0x7f7a5c10ea90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009ca0 tx=0x7f7a5402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 -- 172.21.15.64:0/1110332181 shutdown_connections 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 --2- 172.21.15.64:0/1110332181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c106f10 0x7f7a5c10ea90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 --2- 172.21.15.64:0/1110332181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c102790 0x7f7a5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 --2- 172.21.15.64:0/1110332181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c101de0 0x7f7a5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 -- 172.21.15.64:0/1110332181 >> 172.21.15.64:0/1110332181 conn(0x7f7a5c0fd8c0 msgr2=0x7f7a5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 -- 172.21.15.64:0/1110332181 shutdown_connections 2024-09-19T05:04:41.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.441+0000 7f7a63820640 1 -- 172.21.15.64:0/1110332181 wait complete. 2024-09-19T05:04:41.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 Processor -- start 2024-09-19T05:04:41.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 -- start start 2024-09-19T05:04:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c101de0 0x7f7a5c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 0x7f7a5c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c106f10 0x7f7a5c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c111e20 con 0x7f7a5c102790 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c111ca0 con 0x7f7a5c106f10 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a63820640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c111fa0 con 0x7f7a5c101de0 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 0x7f7a5c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 0x7f7a5c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48296/0 (socket says 172.21.15.64:48296) 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 -- 172.21.15.64:0/1743683135 learned_addr learned my addr 172.21.15.64:0/1743683135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a615bc640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c101de0 0x7f7a5c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a61dbd640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c106f10 0x7f7a5c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 -- 172.21.15.64:0/1743683135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c101de0 msgr2=0x7f7a5c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c101de0 0x7f7a5c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 -- 172.21.15.64:0/1743683135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c106f10 msgr2=0x7f7a5c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c106f10 0x7f7a5c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.442+0000 7f7a60dbb640 1 -- 172.21.15.64:0/1743683135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c19d2f0 con 0x7f7a5c102790 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.443+0000 7f7a60dbb640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 0x7f7a5c198880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d770 tx=0x7f7a5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.443+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50017070 con 0x7f7a5c102790 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.443+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a500054d0 con 0x7f7a5c102790 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.443+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c19d5e0 con 0x7f7a5c102790 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.443+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50004cf0 con 0x7f7a5c102790 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.443+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1a4ec0 con 0x7f7a5c102790 2024-09-19T05:04:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.444+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c1021c0 con 0x7f7a5c102790 2024-09-19T05:04:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.445+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7a50004e90 con 0x7f7a5c102790 2024-09-19T05:04:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.446+0000 7f7a4e7fc640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a480a3350 0x7f7a480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.446+0000 7f7a615bc640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a480a3350 0x7f7a480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.446+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f7a500cf430 con 0x7f7a5c102790 2024-09-19T05:04:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.446+0000 7f7a615bc640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a480a3350 0x7f7a480a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800bbe0 tx=0x7f7a5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.450+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7a5009bfd0 con 0x7f7a5c102790 2024-09-19T05:04:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.625+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a5c10a150 con 0x7f7a5c102790 2024-09-19T05:04:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.626+0000 7f7a4e7fc640 1 -- 172.21.15.64:0/1743683135 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22821 (secure 0 0 0) 0x7f7a5009b8d0 con 0x7f7a5c102790 2024-09-19T05:04:41.628 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:04:41.628 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:20.028795+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a480a3350 msgr2=0x7f7a480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a480a3350 0x7f7a480a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800bbe0 tx=0x7f7a5800b390 comp rx=0 tx=0).stop 2024-09-19T05:04:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 msgr2=0x7f7a5c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 0x7f7a5c198880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d770 tx=0x7f7a5000dc30 comp rx=0 tx=0).stop 2024-09-19T05:04:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 shutdown_connections 2024-09-19T05:04:41.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a480a3350 0x7f7a480a5810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a5c106f10 0x7f7a5c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a5c102790 0x7f7a5c198880 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 --2- 172.21.15.64:0/1743683135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c101de0 0x7f7a5c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 >> 172.21.15.64:0/1743683135 conn(0x7f7a5c0fd8c0 msgr2=0x7f7a5c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 shutdown_connections 2024-09-19T05:04:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:41.629+0000 7f7a63820640 1 -- 172.21.15.64:0/1743683135 wait complete. 2024-09-19T05:04:42.343 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:42 smithi064 ceph-mon[30960]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-09-19T05:04:42.343 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:42 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1743683135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:42.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T05:04:42.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T05:04:42.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T05:04:42.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T05:04:42.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T05:04:42.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T05:04:42.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T05:04:42.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T05:04:42.349 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T05:04:42.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T05:04:42.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T05:04:42.351 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T05:04:42.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:42 smithi103 ceph-mon[34446]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-09-19T05:04:42.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:42 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1743683135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:42.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:42 smithi174 ceph-mon[34488]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-09-19T05:04:42.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:42 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1743683135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:43.017 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.018 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.035 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.825 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:43.825 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 -- 172.21.15.64:0/3083033541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 msgr2=0x7fed74100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 --2- 172.21.15.64:0/3083033541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed74100ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed68009930 tx=0x7fed6802a420 comp rx=0 tx=0).stop 2024-09-19T05:04:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 -- 172.21.15.64:0/3083033541 shutdown_connections 2024-09-19T05:04:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 --2- 172.21.15.64:0/3083033541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed74101420 0x7fed7410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 --2- 172.21.15.64:0/3083033541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed74100ee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 --2- 172.21.15.64:0/3083033541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed74106a80 0x7fed74106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 -- 172.21.15.64:0/3083033541 >> 172.21.15.64:0/3083033541 conn(0x7fed740fc7a0 msgr2=0x7fed740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 -- 172.21.15.64:0/3083033541 shutdown_connections 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.022+0000 7fed7b00f640 1 -- 172.21.15.64:0/3083033541 wait complete. 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.023+0000 7fed7b00f640 1 Processor -- start 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.023+0000 7fed7b00f640 1 -- start start 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.023+0000 7fed7b00f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed741972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed7b00f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed74101420 0x7fed74197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed7b00f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed74106a80 0x7fed7419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed7b00f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed74110da0 con 0x7fed74100a80 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed7b00f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed74110c20 con 0x7fed74101420 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed7b00f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed74110f20 con 0x7fed74106a80 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed741972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed741972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48308/0 (socket says 172.21.15.64:48308) 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 -- 172.21.15.64:0/38945306 learned_addr learned my addr 172.21.15.64:0/38945306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed73fff640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed74101420 0x7fed74197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed795ac640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed74106a80 0x7fed7419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 -- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed74106a80 msgr2=0x7fed7419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed74106a80 0x7fed7419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 -- 172.21.15.64:0/38945306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed74101420 msgr2=0x7fed74197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed74101420 0x7fed74197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 -- 172.21.15.64:0/38945306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7419c270 con 0x7fed74100a80 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed78dab640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed741972c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fed6400d990 tx=0x7fed6400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6401b070 con 0x7fed74100a80 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed6400edf0 con 0x7fed74100a80 2024-09-19T05:04:44.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7419c560 con 0x7fed74100a80 2024-09-19T05:04:44.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.024+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64014800 con 0x7fed74100a80 2024-09-19T05:04:44.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.025+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7406a790 con 0x7fed74100a80 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.025+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed7406ada0 con 0x7fed74100a80 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.027+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fed6401e070 con 0x7fed74100a80 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.027+0000 7fed71ffb640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fed5c0a32d0 0x7fed5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed73fff640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fed5c0a32d0 0x7fed5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fed640d1990 con 0x7fed74100a80 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed71ffb640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] conn(0x7fed5c0acf80 0x7fed5c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 --> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed5c0afaf0 con 0x7fed5c0acf80 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fed640d1de0 con 0x7fed74100a80 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed73fff640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fed5c0a32d0 0x7fed5c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fed68006010 tx=0x7fed68000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed795ac640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] conn(0x7fed5c0acf80 0x7fed5c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.028+0000 7fed795ac640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] conn(0x7fed5c0acf80 0x7fed5c0af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.029+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== osd.2 v2:172.21.15.174:6800/4215230653 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed5c0afaf0 con 0x7fed5c0acf80 2024-09-19T05:04:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.049+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 --> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed7406b220 con 0x7fed5c0acf80 2024-09-19T05:04:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.049+0000 7fed71ffb640 1 -- 172.21.15.64:0/38945306 <== osd.2 v2:172.21.15.174:6800/4215230653 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed7406b220 con 0x7fed5c0acf80 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.049+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] conn(0x7fed5c0acf80 msgr2=0x7fed5c0af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.049+0000 7fed7b00f640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] conn(0x7fed5c0acf80 0x7fed5c0af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fed5c0a32d0 msgr2=0x7fed5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fed5c0a32d0 0x7fed5c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fed68006010 tx=0x7fed68000bc0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 msgr2=0x7fed741972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 --2- 172.21.15.64:0/38945306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed74100a80 0x7fed741972c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fed6400d990 tx=0x7fed6400de50 comp rx=0 tx=0).stop 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed795ac640 1 -- 172.21.15.64:0/38945306 reap_dead start 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 shutdown_connections 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 >> 172.21.15.64:0/38945306 conn(0x7fed740fc7a0 msgr2=0x7fed740fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 shutdown_connections 2024-09-19T05:04:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.050+0000 7fed7b00f640 1 -- 172.21.15.64:0/38945306 wait complete. 2024-09-19T05:04:44.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1121084468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54106a80 msgr2=0x7f7a54106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1121084468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54106a80 0x7f7a54106e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009930 tx=0x7f7a4402a420 comp rx=0 tx=0).stop 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1121084468 shutdown_connections 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1121084468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a54101420 0x7f7a5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1121084468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54100a80 0x7f7a54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1121084468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54106a80 0x7f7a54106e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1121084468 >> 172.21.15.64:0/1121084468 conn(0x7f7a540fc7a0 msgr2=0x7f7a540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1121084468 shutdown_connections 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.308+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1121084468 wait complete. 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 Processor -- start 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 -- start start 2024-09-19T05:04:44.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a54100a80 0x7f7a541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54101420 0x7f7a541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54106a80 0x7f7a5419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a54110d80 con 0x7f7a54101420 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a54110c00 con 0x7f7a54100a80 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a58d8f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a54110f00 con 0x7f7a54106a80 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a51d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54101420 0x7f7a541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a51d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54101420 0x7f7a541988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51622/0 (socket says 172.21.15.64:51622) 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a51d9b640 1 -- 172.21.15.64:0/1826519716 learned_addr learned my addr 172.21.15.64:0/1826519716 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a5259c640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a54100a80 0x7f7a541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.309+0000 7f7a52d9d640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54106a80 0x7f7a5419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a51d9b640 1 -- 172.21.15.64:0/1826519716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54106a80 msgr2=0x7f7a5419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a51d9b640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54106a80 0x7f7a5419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a51d9b640 1 -- 172.21.15.64:0/1826519716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a54100a80 msgr2=0x7f7a541983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a51d9b640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a54100a80 0x7f7a541983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a51d9b640 1 -- 172.21.15.64:0/1826519716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5419d360 con 0x7f7a54101420 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a51d9b640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54101420 0x7f7a541988f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d950 tx=0x7f7a4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48017070 con 0x7f7a54101420 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a48009c30 con 0x7f7a54101420 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800f620 con 0x7f7a54101420 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5419d650 con 0x7f7a54101420 2024-09-19T05:04:44.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.310+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541a4f30 con 0x7f7a54101420 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.311+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a54105ab0 con 0x7f7a54101420 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.312+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7a480109a0 con 0x7f7a54101420 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a437fe640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a3c0a3350 0x7f7a3c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a5259c640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a3c0a3350 0x7f7a3c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f7a480cfa40 con 0x7f7a54101420 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a437fe640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] conn(0x7f7a3c0ad000 0x7f7a3c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 --> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a48005600 con 0x7f7a3c0ad000 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f7a480cfe90 con 0x7f7a54101420 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.313+0000 7f7a52d9d640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] conn(0x7f7a3c0ad000 0x7f7a3c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.314+0000 7f7a5259c640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a3c0a3350 0x7f7a3c0a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a44031440 tx=0x7f7a44005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.314+0000 7f7a52d9d640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] conn(0x7f7a3c0ad000 0x7f7a3c0af4a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.314+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== osd.6 v2:172.21.15.64:6818/3289767656 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a48005600 con 0x7f7a3c0ad000 2024-09-19T05:04:44.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.328+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 --> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a541021c0 con 0x7f7a3c0ad000 2024-09-19T05:04:44.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.328+0000 7f7a437fe640 1 -- 172.21.15.64:0/1826519716 <== osd.6 v2:172.21.15.64:6818/3289767656 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a541021c0 con 0x7f7a3c0ad000 2024-09-19T05:04:44.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] conn(0x7f7a3c0ad000 msgr2=0x7f7a3c0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] conn(0x7f7a3c0ad000 0x7f7a3c0af4a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a3c0a3350 msgr2=0x7f7a3c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7a3c0a3350 0x7f7a3c0a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a44031440 tx=0x7f7a44005a20 comp rx=0 tx=0).stop 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54101420 msgr2=0x7f7a541988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 --2- 172.21.15.64:0/1826519716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a54101420 0x7f7a541988f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d950 tx=0x7f7a4800de10 comp rx=0 tx=0).stop 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a52d9d640 1 -- 172.21.15.64:0/1826519716 reap_dead start 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 shutdown_connections 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 >> 172.21.15.64:0/1826519716 conn(0x7f7a540fc7a0 msgr2=0x7f7a5406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 shutdown_connections 2024-09-19T05:04:44.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.329+0000 7f7a58d8f640 1 -- 172.21.15.64:0/1826519716 wait complete. 2024-09-19T05:04:44.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:44 smithi064 ceph-mon[30960]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-19T05:04:44.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:44 smithi103 ceph-mon[34446]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-19T05:04:44.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:44 smithi174 ceph-mon[34488]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-19T05:04:44.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 -- 172.21.15.64:0/1095831292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688102ef0 msgr2=0x7fe68810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 --2- 172.21.15.64:0/1095831292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688102ef0 0x7fe68810da40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe680009ca0 tx=0x7fe68002a950 comp rx=0 tx=0).stop 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 -- 172.21.15.64:0/1095831292 shutdown_connections 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 --2- 172.21.15.64:0/1095831292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688102ef0 0x7fe68810da40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 --2- 172.21.15.64:0/1095831292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe688102570 0x7fe6881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 --2- 172.21.15.64:0/1095831292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe688100e20 0x7fe688101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 -- 172.21.15.64:0/1095831292 >> 172.21.15.64:0/1095831292 conn(0x7fe6880fc7d0 msgr2=0x7fe6880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 -- 172.21.15.64:0/1095831292 shutdown_connections 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 -- 172.21.15.64:0/1095831292 wait complete. 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 Processor -- start 2024-09-19T05:04:44.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.704+0000 7fe68e26e640 1 -- start start 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68e26e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688100e20 0x7fe688198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68e26e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe688102570 0x7fe688198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68e26e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe688102ef0 0x7fe68819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68e26e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe688110e80 con 0x7fe688100e20 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68e26e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe688110d00 con 0x7fe688102ef0 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68e26e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe688111000 con 0x7fe688102570 2024-09-19T05:04:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688100e20 0x7fe688198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688100e20 0x7fe688198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51638/0 (socket says 172.21.15.64:51638) 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 -- 172.21.15.64:0/4191563587 learned_addr learned my addr 172.21.15.64:0/4191563587 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe6877fe640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe688102570 0x7fe688198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe68c80b640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe688102ef0 0x7fe68819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 -- 172.21.15.64:0/4191563587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe688102570 msgr2=0x7fe688198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe688102570 0x7fe688198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 -- 172.21.15.64:0/4191563587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe688102ef0 msgr2=0x7fe68819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe688102ef0 0x7fe68819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.705+0000 7fe687fff640 1 -- 172.21.15.64:0/4191563587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68819d3e0 con 0x7fe688100e20 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.706+0000 7fe687fff640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688100e20 0x7fe688198430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e9f0 tx=0x7fe67800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.706+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67800cd80 con 0x7fe688100e20 2024-09-19T05:04:44.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.706+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68819d6d0 con 0x7fe688100e20 2024-09-19T05:04:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.706+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6780054d0 con 0x7fe688100e20 2024-09-19T05:04:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.706+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678010490 con 0x7fe688100e20 2024-09-19T05:04:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.706+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6880fcbb0 con 0x7fe688100e20 2024-09-19T05:04:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.707+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe688062980 con 0x7fe688100e20 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.708+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe678005000 con 0x7fe688100e20 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe6857fa640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe6740a32d0 0x7fe6740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe6877fe640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe6740a32d0 0x7fe6740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fe6780d0220 con 0x7fe688100e20 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe6857fa640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] conn(0x7fe6740acf80 0x7fe6740af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 --> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6740afaf0 con 0x7fe6740acf80 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fe6780d0670 con 0x7fe688100e20 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.709+0000 7fe68c80b640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] conn(0x7fe6740acf80 0x7fe6740af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.710+0000 7fe6877fe640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe6740a32d0 0x7fe6740a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe6880fffb0 tx=0x7fe670005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.710+0000 7fe68c80b640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] conn(0x7fe6740acf80 0x7fe6740af420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.710+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== osd.0 v2:172.21.15.64:6802/198612144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6740afaf0 con 0x7fe6740acf80 2024-09-19T05:04:44.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.725+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 --> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe688062e00 con 0x7fe6740acf80 2024-09-19T05:04:44.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.725+0000 7fe6857fa640 1 -- 172.21.15.64:0/4191563587 <== osd.0 v2:172.21.15.64:6802/198612144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe688062e00 con 0x7fe6740acf80 2024-09-19T05:04:44.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.725+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] conn(0x7fe6740acf80 msgr2=0x7fe6740af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.725+0000 7fe68e26e640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] conn(0x7fe6740acf80 0x7fe6740af420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.725+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe6740a32d0 msgr2=0x7fe6740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.725+0000 7fe68e26e640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe6740a32d0 0x7fe6740a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe6880fffb0 tx=0x7fe670005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:44.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688100e20 msgr2=0x7fe688198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68e26e640 1 --2- 172.21.15.64:0/4191563587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe688100e20 0x7fe688198430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e9f0 tx=0x7fe67800edf0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68c80b640 1 -- 172.21.15.64:0/4191563587 reap_dead start 2024-09-19T05:04:44.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 shutdown_connections 2024-09-19T05:04:44.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 >> 172.21.15.64:0/4191563587 conn(0x7fe6880fc7d0 msgr2=0x7fe68806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 shutdown_connections 2024-09-19T05:04:44.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:44.726+0000 7fe68e26e640 1 -- 172.21.15.64:0/4191563587 wait complete. 2024-09-19T05:04:45.675 INFO:teuthology.orchestra.run.smithi064.stdout:34359738444 2024-09-19T05:04:45.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T05:04:46.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 -- 172.21.15.64:0/4144631178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 msgr2=0x7fca5410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:46.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 --2- 172.21.15.64:0/4144631178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca5410bb50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009900 tx=0x7fca3c02a480 comp rx=0 tx=0).stop 2024-09-19T05:04:46.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 -- 172.21.15.64:0/4144631178 shutdown_connections 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 --2- 172.21.15.64:0/4144631178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca5410c170 0x7fca5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 --2- 172.21.15.64:0/4144631178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca5410bb50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 --2- 172.21.15.64:0/4144631178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54102d40 0x7fca54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 -- 172.21.15.64:0/4144631178 >> 172.21.15.64:0/4144631178 conn(0x7fca540fc7a0 msgr2=0x7fca540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 -- 172.21.15.64:0/4144631178 shutdown_connections 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 -- 172.21.15.64:0/4144631178 wait complete. 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.190+0000 7fca590fa640 1 Processor -- start 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 -- start start 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54102d40 0x7fca541009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca54100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca5410c170 0x7fca54101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:46.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca54111e20 con 0x7fca54103660 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca54111ca0 con 0x7fca54102d40 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca590fa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca54111fa0 con 0x7fca5410c170 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca54100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca54100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51646/0 (socket says 172.21.15.64:51646) 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 -- 172.21.15.64:0/736657452 learned_addr learned my addr 172.21.15.64:0/736657452 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5359e640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca5410c170 0x7fca54101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca52d9d640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54102d40 0x7fca541009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 -- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca5410c170 msgr2=0x7fca54101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca5410c170 0x7fca54101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 -- 172.21.15.64:0/736657452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54102d40 msgr2=0x7fca541009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca54102d40 0x7fca541009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 -- 172.21.15.64:0/736657452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca541a67e0 con 0x7fca54103660 2024-09-19T05:04:46.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.191+0000 7fca5259c640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca54100ee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005940 tx=0x7fca3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.192+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c03f030 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.192+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca3c003cb0 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.192+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c035070 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.192+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca541a6a70 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.192+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541a6f50 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.193+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca54062900 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.194+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fca3c042030 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca4bfff640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fca400a32d0 0x7fca400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca52d9d640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fca400a32d0 0x7fca400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fca3c0f04c0 con 0x7fca54103660 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca4bfff640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] conn(0x7fca400acf80 0x7fca400af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:46.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 --> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca3c001d30 con 0x7fca400acf80 2024-09-19T05:04:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fca3c0f0910 con 0x7fca54103660 2024-09-19T05:04:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca52d9d640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fca400a32d0 0x7fca400a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fca44005fd0 tx=0x7fca44009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.195+0000 7fca5359e640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] conn(0x7fca400acf80 0x7fca400af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.196+0000 7fca5359e640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] conn(0x7fca400acf80 0x7fca400af420 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:46.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.197+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== osd.11 v2:172.21.15.174:6824/3718770744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca3c001d30 con 0x7fca400acf80 2024-09-19T05:04:46.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.210+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 --> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca54062d80 con 0x7fca400acf80 2024-09-19T05:04:46.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca4bfff640 1 -- 172.21.15.64:0/736657452 <== osd.11 v2:172.21.15.174:6824/3718770744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fca54062d80 con 0x7fca400acf80 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] conn(0x7fca400acf80 msgr2=0x7fca400af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] conn(0x7fca400acf80 0x7fca400af420 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fca400a32d0 msgr2=0x7fca400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fca400a32d0 0x7fca400a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fca44005fd0 tx=0x7fca44009680 comp rx=0 tx=0).stop 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 msgr2=0x7fca54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 --2- 172.21.15.64:0/736657452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca54103660 0x7fca54100ee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005940 tx=0x7fca3c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:04:46.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca5359e640 1 -- 172.21.15.64:0/736657452 reap_dead start 2024-09-19T05:04:46.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 shutdown_connections 2024-09-19T05:04:46.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.211+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 >> 172.21.15.64:0/736657452 conn(0x7fca540fc7a0 msgr2=0x7fca540fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:46.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.212+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 shutdown_connections 2024-09-19T05:04:46.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:46.212+0000 7fca590fa640 1 -- 172.21.15.64:0/736657452 wait complete. 2024-09-19T05:04:46.327 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:46 smithi064 ceph-mon[30960]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T05:04:46.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:46 smithi103 ceph-mon[34446]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T05:04:46.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:46 smithi174 ceph-mon[34488]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T05:04:47.601 INFO:teuthology.orchestra.run.smithi064.stdout:304942678037 2024-09-19T05:04:47.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T05:04:47.825 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:48.202 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:48 smithi064 ceph-mon[30960]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-19T05:04:48.230 INFO:teuthology.orchestra.run.smithi064.stdout:73014444096 2024-09-19T05:04:48.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T05:04:48.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:48 smithi103 ceph-mon[34446]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-19T05:04:48.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:48 smithi174 ceph-mon[34488]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-19T05:04:48.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 -- 172.21.15.64:0/2879460020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc10e4f0 msgr2=0x7f57cc1108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2879460020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc10e4f0 0x7f57cc1108e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009ca0 tx=0x7f57c402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 -- 172.21.15.64:0/2879460020 shutdown_connections 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2879460020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc10e4f0 0x7f57cc1108e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2879460020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc1012f0 0x7f57cc10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2879460020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57cc100940 0x7f57cc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 -- 172.21.15.64:0/2879460020 >> 172.21.15.64:0/2879460020 conn(0x7f57cc0fc7d0 msgr2=0x7f57cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 -- 172.21.15.64:0/2879460020 shutdown_connections 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.423+0000 7f57d25f9640 1 -- 172.21.15.64:0/2879460020 wait complete. 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 Processor -- start 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 -- start start 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57cc100940 0x7f57cc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc1012f0 0x7f57cc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10e4f0 0x7f57cc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc114020 con 0x7f57cc1012f0 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc113ea0 con 0x7f57cc100940 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d25f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1141a0 con 0x7f57cc10e4f0 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc1012f0 0x7f57cc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc1012f0 0x7f57cc19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51664/0 (socket says 172.21.15.64:51664) 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cb7fe640 1 -- 172.21.15.64:0/2209019928 learned_addr learned my addr 172.21.15.64:0/2209019928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cbfff640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57cc100940 0x7f57cc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57d0b96640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10e4f0 0x7f57cc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cb7fe640 1 -- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10e4f0 msgr2=0x7f57cc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cb7fe640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10e4f0 0x7f57cc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.424+0000 7f57cb7fe640 1 -- 172.21.15.64:0/2209019928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57cc100940 msgr2=0x7f57cc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57cb7fe640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57cc100940 0x7f57cc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57cb7fe640 1 -- 172.21.15.64:0/2209019928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc19f540 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57d0b96640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10e4f0 0x7f57cc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57cb7fe640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc1012f0 0x7f57cc19aad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f57c000e9f0 tx=0x7f57c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c000cd80 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57c00054d0 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc19f830 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0010490 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.425+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc1a7110 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.426+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57cc062980 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.427+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f57c0005020 con 0x7f57cc1012f0 2024-09-19T05:04:48.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.428+0000 7f57c97fa640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f57b40a32d0 0x7f57b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.428+0000 7f57cbfff640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f57b40a32d0 0x7f57b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.428+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f57c00d0560 con 0x7f57cc1012f0 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.428+0000 7f57c97fa640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] conn(0x7f57b40acf80 0x7f57b40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.428+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 --> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57b40afaf0 con 0x7f57b40acf80 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.428+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f57c00d09b0 con 0x7f57cc1012f0 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.429+0000 7f57cbfff640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f57b40a32d0 0x7f57b40a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009b60 tx=0x7f57bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.429+0000 7f57d0b96640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] conn(0x7f57b40acf80 0x7f57b40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.429+0000 7f57d0b96640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] conn(0x7f57b40acf80 0x7f57b40af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:48.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.430+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== osd.8 v2:172.21.15.174:6816/1095432672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57b40afaf0 con 0x7f57b40acf80 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.446+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 --> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f57cc062d80 con 0x7f57b40acf80 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57c97fa640 1 -- 172.21.15.64:0/2209019928 <== osd.8 v2:172.21.15.174:6816/1095432672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f57cc062d80 con 0x7f57b40acf80 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] conn(0x7f57b40acf80 msgr2=0x7f57b40af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] conn(0x7f57b40acf80 0x7f57b40af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f57b40a32d0 msgr2=0x7f57b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f57b40a32d0 0x7f57b40a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009b60 tx=0x7f57bc005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc1012f0 msgr2=0x7f57cc19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 --2- 172.21.15.64:0/2209019928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57cc1012f0 0x7f57cc19aad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f57c000e9f0 tx=0x7f57c000edf0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d0b96640 1 -- 172.21.15.64:0/2209019928 reap_dead start 2024-09-19T05:04:48.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 shutdown_connections 2024-09-19T05:04:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 >> 172.21.15.64:0/2209019928 conn(0x7f57cc0fc7d0 msgr2=0x7f57cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 shutdown_connections 2024-09-19T05:04:48.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:48.447+0000 7f57d25f9640 1 -- 172.21.15.64:0/2209019928 wait complete. 2024-09-19T05:04:48.961 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:48.961 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:48.961 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 -- 172.21.15.64:0/1756363121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358101de0 msgr2=0x7f83581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 --2- 172.21.15.64:0/1756363121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358101de0 0x7f83581021c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8348009ca0 tx=0x7f834802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 -- 172.21.15.64:0/1756363121 shutdown_connections 2024-09-19T05:04:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 --2- 172.21.15.64:0/1756363121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8358106f10 0x7f835810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 --2- 172.21.15.64:0/1756363121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8358102790 0x7f83581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 --2- 172.21.15.64:0/1756363121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358101de0 0x7f83581021c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 -- 172.21.15.64:0/1756363121 >> 172.21.15.64:0/1756363121 conn(0x7f83580fd8c0 msgr2=0x7f83580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 -- 172.21.15.64:0/1756363121 shutdown_connections 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.216+0000 7f835f033640 1 -- 172.21.15.64:0/1756363121 wait complete. 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 Processor -- start 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 -- start start 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8358101de0 0x7f83581983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358102790 0x7f8358198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8358106f10 0x7f835819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111e40 con 0x7f8358102790 2024-09-19T05:04:49.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111cc0 con 0x7f8358106f10 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f835f033640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8358111fc0 con 0x7f8358101de0 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f8357fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358102790 0x7f8358198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.217+0000 7f8357fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358102790 0x7f8358198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51684/0 (socket says 172.21.15.64:51684) 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 -- 172.21.15.64:0/1441352524 learned_addr learned my addr 172.21.15.64:0/1441352524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f835d5d0640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8358106f10 0x7f835819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f835cdcf640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8358101de0 0x7f83581983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 -- 172.21.15.64:0/1441352524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8358101de0 msgr2=0x7f83581983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8358101de0 0x7f83581983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 -- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8358106f10 msgr2=0x7f835819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8358106f10 0x7f835819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 -- 172.21.15.64:0/1441352524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835819d390 con 0x7f8358102790 2024-09-19T05:04:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8357fff640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358102790 0x7f8358198920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f834000b4a0 tx=0x7f834000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340014070 con 0x7f8358102790 2024-09-19T05:04:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8340010ac0 con 0x7f8358102790 2024-09-19T05:04:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835819d680 con 0x7f8358102790 2024-09-19T05:04:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340017c30 con 0x7f8358102790 2024-09-19T05:04:49.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.218+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581a4f60 con 0x7f8358102790 2024-09-19T05:04:49.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.219+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8358062940 con 0x7f8358102790 2024-09-19T05:04:49.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.221+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8340017070 con 0x7f8358102790 2024-09-19T05:04:49.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.221+0000 7f8355ffb640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f83440a32d0 0x7f83440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.221+0000 7f835cdcf640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f83440a32d0 0x7f83440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.221+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f83400cefe0 con 0x7f8358102790 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.222+0000 7f8355ffb640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] conn(0x7f83440acf80 0x7f83440af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.222+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 --> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83440afaf0 con 0x7f83440acf80 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.222+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f83400d6050 con 0x7f8358102790 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.222+0000 7f835cdcf640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f83440a32d0 0x7f83440a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8348009a60 tx=0x7f83480093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.222+0000 7f835d5d0640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] conn(0x7f83440acf80 0x7f83440af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.222+0000 7f835d5d0640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] conn(0x7f83440acf80 0x7f83440af420 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.223+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== osd.4 v2:172.21.15.103:6808/2896874846 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83440afaf0 con 0x7f83440acf80 2024-09-19T05:04:49.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.237+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 --> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8358062dc0 con 0x7f83440acf80 2024-09-19T05:04:49.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.237+0000 7f8355ffb640 1 -- 172.21.15.64:0/1441352524 <== osd.4 v2:172.21.15.103:6808/2896874846 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8358062dc0 con 0x7f83440acf80 2024-09-19T05:04:49.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] conn(0x7f83440acf80 msgr2=0x7f83440af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] conn(0x7f83440acf80 0x7f83440af420 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f83440a32d0 msgr2=0x7f83440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f83440a32d0 0x7f83440a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8348009a60 tx=0x7f83480093f0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358102790 msgr2=0x7f8358198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 --2- 172.21.15.64:0/1441352524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8358102790 0x7f8358198920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f834000b4a0 tx=0x7f834000b960 comp rx=0 tx=0).stop 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835d5d0640 1 -- 172.21.15.64:0/1441352524 reap_dead start 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 shutdown_connections 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 >> 172.21.15.64:0/1441352524 conn(0x7f83580fd8c0 msgr2=0x7f83580fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 shutdown_connections 2024-09-19T05:04:49.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.238+0000 7f835f033640 1 -- 172.21.15.64:0/1441352524 wait complete. 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 -- 172.21.15.64:0/3054979795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102570 msgr2=0x7f75341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 --2- 172.21.15.64:0/3054979795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102570 0x7f75341029b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f751c009900 tx=0x7f751c02a480 comp rx=0 tx=0).stop 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 -- 172.21.15.64:0/3054979795 shutdown_connections 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 --2- 172.21.15.64:0/3054979795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7534102ef0 0x7f753410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 --2- 172.21.15.64:0/3054979795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102570 0x7f75341029b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 --2- 172.21.15.64:0/3054979795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7534100e20 0x7f7534101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 -- 172.21.15.64:0/3054979795 >> 172.21.15.64:0/3054979795 conn(0x7f75340fc7d0 msgr2=0x7f75340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 -- 172.21.15.64:0/3054979795 shutdown_connections 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.435+0000 7f7538eee640 1 -- 172.21.15.64:0/3054979795 wait complete. 2024-09-19T05:04:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 Processor -- start 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 -- start start 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7534100e20 0x7f75341973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7534102570 0x7f75341978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102ef0 0x7f753419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7534110e30 con 0x7f7534102ef0 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7534110cb0 con 0x7f7534102570 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7538eee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7534110fb0 con 0x7f7534100e20 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7532d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102ef0 0x7f753419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.436+0000 7f7532d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102ef0 0x7f753419bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51712/0 (socket says 172.21.15.64:51712) 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 -- 172.21.15.64:0/2776943881 learned_addr learned my addr 172.21.15.64:0/2776943881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f753259c640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7534100e20 0x7f75341973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f752bfff640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7534102570 0x7f75341978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 -- 172.21.15.64:0/2776943881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7534100e20 msgr2=0x7f75341973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7534100e20 0x7f75341973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 -- 172.21.15.64:0/2776943881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7534102570 msgr2=0x7f75341978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7534102570 0x7f75341978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 -- 172.21.15.64:0/2776943881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753419c360 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7532d9d640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102ef0 0x7f753419bc80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f752c009fd0 tx=0x7f752c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c017070 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f752c003e20 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753419c650 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c010460 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.437+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75341a3f30 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.438+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7534062980 con 0x7f7534102ef0 2024-09-19T05:04:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f752c003970 con 0x7f7534102ef0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f752b7fe640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f75200a32d0 0x7f75200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f753259c640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f75200a32d0 0x7f75200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f752c0cfbc0 con 0x7f7534102ef0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f752b7fe640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] conn(0x7f75200acf80 0x7f75200af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 --> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75200afaf0 con 0x7f75200acf80 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.440+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f752c0d0010 con 0x7f7534102ef0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.441+0000 7f752bfff640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] conn(0x7f75200acf80 0x7f75200af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.441+0000 7f753259c640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f75200a32d0 0x7f75200a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7524009bd0 tx=0x7f7524005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.441+0000 7f752bfff640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] conn(0x7f75200acf80 0x7f75200af420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.442+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== osd.9 v2:172.21.15.64:6826/1960883919 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75200afaf0 con 0x7f75200acf80 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.455+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 --> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7534062e00 con 0x7f75200acf80 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.455+0000 7f752b7fe640 1 -- 172.21.15.64:0/2776943881 <== osd.9 v2:172.21.15.64:6826/1960883919 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7534062e00 con 0x7f75200acf80 2024-09-19T05:04:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] conn(0x7f75200acf80 msgr2=0x7f75200af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] conn(0x7f75200acf80 0x7f75200af420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f75200a32d0 msgr2=0x7f75200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f75200a32d0 0x7f75200a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7524009bd0 tx=0x7f7524005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102ef0 msgr2=0x7f753419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 --2- 172.21.15.64:0/2776943881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7534102ef0 0x7f753419bc80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f752c009fd0 tx=0x7f752c00b260 comp rx=0 tx=0).stop 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7532d9d640 1 -- 172.21.15.64:0/2776943881 reap_dead start 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 shutdown_connections 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 >> 172.21.15.64:0/2776943881 conn(0x7f75340fc7d0 msgr2=0x7f75340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 shutdown_connections 2024-09-19T05:04:49.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.456+0000 7f7538eee640 1 -- 172.21.15.64:0/2776943881 wait complete. 2024-09-19T05:04:50.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 -- 172.21.15.64:0/2160227651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908102dd0 msgr2=0x7f29081031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2160227651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908102dd0 0x7f29081031b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009cd0 tx=0x7f28f802a8d0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 -- 172.21.15.64:0/2160227651 shutdown_connections 2024-09-19T05:04:50.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2160227651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2908108300 0x7f290810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2160227651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2908103780 0x7f2908107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2160227651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908102dd0 0x7f29081031b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 -- 172.21.15.64:0/2160227651 >> 172.21.15.64:0/2160227651 conn(0x7f29080fc7d0 msgr2=0x7f29080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 -- 172.21.15.64:0/2160227651 shutdown_connections 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:49.999+0000 7f290f1d6640 1 -- 172.21.15.64:0/2160227651 wait complete. 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 Processor -- start 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 -- start start 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2908102dd0 0x7f2908198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2908103780 0x7f2908198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908108300 0x7f290819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2908111f40 con 0x7f2908108300 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2908111dc0 con 0x7f2908103780 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290f1d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29081120c0 con 0x7f2908102dd0 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908108300 0x7f290819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908108300 0x7f290819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51734/0 (socket says 172.21.15.64:51734) 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 -- 172.21.15.64:0/2108096289 learned_addr learned my addr 172.21.15.64:0/2108096289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290cf72640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2908102dd0 0x7f2908198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f2907fff640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2908103780 0x7f2908198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 -- 172.21.15.64:0/2108096289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2908102dd0 msgr2=0x7f2908198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2908102dd0 0x7f2908198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 -- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2908103780 msgr2=0x7f2908198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2908103780 0x7f2908198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.000+0000 7f290d773640 1 -- 172.21.15.64:0/2108096289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290819d4f0 con 0x7f2908108300 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.001+0000 7f290d773640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908108300 0x7f290819ce10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f290000d950 tx=0x7f290000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.001+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900017030 con 0x7f2908108300 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.001+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290819d7e0 con 0x7f2908108300 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.001+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2900004020 con 0x7f2908108300 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.001+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900003b20 con 0x7f2908108300 2024-09-19T05:04:50.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.001+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29081a50c0 con 0x7f2908108300 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.002+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2908062980 con 0x7f2908108300 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.003+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f290001a070 con 0x7f2908108300 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f2905ffb640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28f40a32d0 0x7f28f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f290cf72640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28f40a32d0 0x7f28f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f29000cf280 con 0x7f2908108300 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f2905ffb640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] conn(0x7f28f40acf80 0x7f28f40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 --> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28f40afaf0 con 0x7f28f40acf80 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f29000d6050 con 0x7f2908108300 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f290cf72640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28f40a32d0 0x7f28f40a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000f80 tx=0x7f28f8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.004+0000 7f2907fff640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] conn(0x7f28f40acf80 0x7f28f40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.005+0000 7f2907fff640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] conn(0x7f28f40acf80 0x7f28f40af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.005+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== osd.1 v2:172.21.15.103:6800/1592456147 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28f40afaf0 con 0x7f28f40acf80 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.019+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 --> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2908062e00 con 0x7f28f40acf80 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.019+0000 7f2905ffb640 1 -- 172.21.15.64:0/2108096289 <== osd.1 v2:172.21.15.103:6800/1592456147 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2908062e00 con 0x7f28f40acf80 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] conn(0x7f28f40acf80 msgr2=0x7f28f40af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] conn(0x7f28f40acf80 0x7f28f40af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28f40a32d0 msgr2=0x7f28f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f28f40a32d0 0x7f28f40a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000f80 tx=0x7f28f8009360 comp rx=0 tx=0).stop 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908108300 msgr2=0x7f290819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 --2- 172.21.15.64:0/2108096289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2908108300 0x7f290819ce10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f290000d950 tx=0x7f290000de10 comp rx=0 tx=0).stop 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290d773640 1 -- 172.21.15.64:0/2108096289 reap_dead start 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 shutdown_connections 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 >> 172.21.15.64:0/2108096289 conn(0x7f29080fc7d0 msgr2=0x7f2908103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 shutdown_connections 2024-09-19T05:04:50.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.020+0000 7f290f1d6640 1 -- 172.21.15.64:0/2108096289 wait complete. 2024-09-19T05:04:50.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:50 smithi064 ceph-mon[30960]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-19T05:04:50.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:50 smithi103 ceph-mon[34446]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-19T05:04:50.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:50 smithi174 ceph-mon[34488]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 -- 172.21.15.64:0/2010266460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 msgr2=0x7f78b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 --2- 172.21.15.64:0/2010266460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b81021c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009900 tx=0x7f78b402a410 comp rx=0 tx=0).stop 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 -- 172.21.15.64:0/2010266460 shutdown_connections 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 --2- 172.21.15.64:0/2010266460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b8106f10 0x7f78b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 --2- 172.21.15.64:0/2010266460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b8102790 0x7f78b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 --2- 172.21.15.64:0/2010266460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b81021c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 -- 172.21.15.64:0/2010266460 >> 172.21.15.64:0/2010266460 conn(0x7f78b80fd8c0 msgr2=0x7f78b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 -- 172.21.15.64:0/2010266460 shutdown_connections 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.784+0000 7f78bf784640 1 -- 172.21.15.64:0/2010266460 wait complete. 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 Processor -- start 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 -- start start 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b8102790 0x7f78b819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b8106f10 0x7f78b819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8111de0 con 0x7f78b8101de0 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8111c60 con 0x7f78b8106f10 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bf784640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8111f60 con 0x7f78b8102790 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bd520640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bd520640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51746/0 (socket says 172.21.15.64:51746) 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bd520640 1 -- 172.21.15.64:0/1920946318 learned_addr learned my addr 172.21.15.64:0/1920946318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bdd21640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b8106f10 0x7f78b819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.785+0000 7f78bcd1f640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b8102790 0x7f78b819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bd520640 1 -- 172.21.15.64:0/1920946318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b8102790 msgr2=0x7f78b819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bd520640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b8102790 0x7f78b819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bd520640 1 -- 172.21.15.64:0/1920946318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b8106f10 msgr2=0x7f78b819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bd520640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b8106f10 0x7f78b819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bd520640 1 -- 172.21.15.64:0/1920946318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b819f510 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bdd21640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b8106f10 0x7f78b819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bcd1f640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b8102790 0x7f78b819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bd520640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b819a560 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b403f030 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b819f7a0 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78b40035f0 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4035070 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.786+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81a70e0 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.787+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78b8075780 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.788+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f78b4003b10 con 0x7f78b8101de0 2024-09-19T05:04:50.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78ae7fc640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a40a32d0 0x7f78a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78bcd1f640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a40a32d0 0x7f78a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f78b40f0db0 con 0x7f78b8101de0 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78ae7fc640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] conn(0x7f78a40acf80 0x7f78a40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 --> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78b4001d30 con 0x7f78a40acf80 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f78b40f1200 con 0x7f78b8101de0 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78bdd21640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] conn(0x7f78a40acf80 0x7f78a40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.789+0000 7f78bcd1f640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a40a32d0 0x7f78a40a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78b810ee50 tx=0x7f78a8005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.790+0000 7f78bdd21640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] conn(0x7f78a40acf80 0x7f78a40af420 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.790+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== osd.3 v2:172.21.15.64:6810/973141608 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78b4001d30 con 0x7f78a40acf80 2024-09-19T05:04:50.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.805+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 --> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78b81032c0 con 0x7f78a40acf80 2024-09-19T05:04:50.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78ae7fc640 1 -- 172.21.15.64:0/1920946318 <== osd.3 v2:172.21.15.64:6810/973141608 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78b81032c0 con 0x7f78a40acf80 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] conn(0x7f78a40acf80 msgr2=0x7f78a40af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] conn(0x7f78a40acf80 0x7f78a40af420 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a40a32d0 msgr2=0x7f78a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f78a40a32d0 0x7f78a40a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78b810ee50 tx=0x7f78a8005d90 comp rx=0 tx=0).stop 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 msgr2=0x7f78b819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 --2- 172.21.15.64:0/1920946318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78b8101de0 0x7f78b819a560 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b4004270 comp rx=0 tx=0).stop 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bdd21640 1 -- 172.21.15.64:0/1920946318 reap_dead start 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 shutdown_connections 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.806+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 >> 172.21.15.64:0/1920946318 conn(0x7f78b80fd8c0 msgr2=0x7f78b80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.807+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 shutdown_connections 2024-09-19T05:04:50.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:50.807+0000 7f78bf784640 1 -- 172.21.15.64:0/1920946318 wait complete. 2024-09-19T05:04:51.347 INFO:teuthology.orchestra.run.smithi064.stdout:128849018936 2024-09-19T05:04:51.347 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T05:04:51.946 INFO:teuthology.orchestra.run.smithi064.stdout:55834574919 2024-09-19T05:04:51.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T05:04:52.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:52 smithi064 ceph-mon[30960]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-19T05:04:52.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:52 smithi103 ceph-mon[34446]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-19T05:04:52.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:52 smithi174 ceph-mon[34488]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 8 op/s 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 -- 172.21.15.64:0/3597800226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102570 msgr2=0x7f490c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 --2- 172.21.15.64:0/3597800226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102570 0x7f490c1029b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402a480 comp rx=0 tx=0).stop 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 -- 172.21.15.64:0/3597800226 shutdown_connections 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 --2- 172.21.15.64:0/3597800226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c102ef0 0x7f490c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 --2- 172.21.15.64:0/3597800226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102570 0x7f490c1029b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 --2- 172.21.15.64:0/3597800226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490c100e20 0x7f490c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 -- 172.21.15.64:0/3597800226 >> 172.21.15.64:0/3597800226 conn(0x7f490c0fc7d0 msgr2=0x7f490c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 -- 172.21.15.64:0/3597800226 shutdown_connections 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.586+0000 7f49126ef640 1 -- 172.21.15.64:0/3597800226 wait complete. 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 Processor -- start 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 -- start start 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490c100e20 0x7f490c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c102570 0x7f490c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102ef0 0x7f490c1a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110e50 con 0x7f490c102ef0 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110cd0 con 0x7f490c100e20 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f49126ef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f490c110fd0 con 0x7f490c102570 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f4910c8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102ef0 0x7f490c1a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f4910c8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102ef0 0x7f490c1a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51768/0 (socket says 172.21.15.64:51768) 2024-09-19T05:04:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f4910c8c640 1 -- 172.21.15.64:0/1392503168 learned_addr learned my addr 172.21.15.64:0/1392503168 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.587+0000 7f490bfff640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490c100e20 0x7f490c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4910c8c640 1 -- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c102570 msgr2=0x7f490c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4910c8c640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c102570 0x7f490c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4910c8c640 1 -- 172.21.15.64:0/1392503168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490c100e20 msgr2=0x7f490c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4910c8c640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490c100e20 0x7f490c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4910c8c640 1 -- 172.21.15.64:0/1392503168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1a48a0 con 0x7f490c102ef0 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4910c8c640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102ef0 0x7f490c1a4360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f490400b970 tx=0x7f490400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904019070 con 0x7f490c102ef0 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a4a70 con 0x7f490c102ef0 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4904014440 con 0x7f490c102ef0 2024-09-19T05:04:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49040075d0 con 0x7f490c102ef0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.588+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a4fb0 con 0x7f490c102ef0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.589+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f490c062980 con 0x7f490c102ef0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.590+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f490401c070 con 0x7f490c102ef0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f4909ffb640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f48f80a32d0 0x7f48f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f490bfff640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f48f80a32d0 0x7f48f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f49040cf740 con 0x7f490c102ef0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f4909ffb640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] conn(0x7f48f80acf80 0x7f48f80af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 --> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49040047d0 con 0x7f48f80acf80 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f49040cfb90 con 0x7f490c102ef0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f490bfff640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f48f80a32d0 0x7f48f80a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00bbe0 tx=0x7f48fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.591+0000 7f4903fff640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] conn(0x7f48f80acf80 0x7f48f80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.592+0000 7f4903fff640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] conn(0x7f48f80acf80 0x7f48f80af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.593+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== osd.5 v2:172.21.15.174:6808/3607820315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49040047d0 con 0x7f48f80acf80 2024-09-19T05:04:52.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.606+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 --> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f490c062e00 con 0x7f48f80acf80 2024-09-19T05:04:52.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f4909ffb640 1 -- 172.21.15.64:0/1392503168 <== osd.5 v2:172.21.15.174:6808/3607820315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f490c062e00 con 0x7f48f80acf80 2024-09-19T05:04:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] conn(0x7f48f80acf80 msgr2=0x7f48f80af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] conn(0x7f48f80acf80 0x7f48f80af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f48f80a32d0 msgr2=0x7f48f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:52.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f48f80a32d0 0x7f48f80a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00bbe0 tx=0x7f48fc00b390 comp rx=0 tx=0).stop 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102ef0 msgr2=0x7f490c1a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 --2- 172.21.15.64:0/1392503168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490c102ef0 0x7f490c1a4360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f490400b970 tx=0x7f490400be30 comp rx=0 tx=0).stop 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f4910c8c640 1 -- 172.21.15.64:0/1392503168 reap_dead start 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 shutdown_connections 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.607+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 >> 172.21.15.64:0/1392503168 conn(0x7f490c0fc7d0 msgr2=0x7f490c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.608+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 shutdown_connections 2024-09-19T05:04:52.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.608+0000 7f49126ef640 1 -- 172.21.15.64:0/1392503168 wait complete. 2024-09-19T05:04:53.439 INFO:teuthology.orchestra.run.smithi064.stdout:253403070494 2024-09-19T05:04:53.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T05:04:53.843 INFO:teuthology.orchestra.run.smithi064.stdout:223338299428 2024-09-19T05:04:53.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T05:04:54.367 INFO:teuthology.orchestra.run.smithi064.stdout:98784247868 2024-09-19T05:04:54.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T05:04:54.390 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:54 smithi064 ceph-mon[30960]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:04:54.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:54 smithi103 ceph-mon[34446]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.803+0000 7f6c0d698640 1 -- 172.21.15.64:0/3984876691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c081027a0 msgr2=0x7f6c081067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.803+0000 7f6c0d698640 1 --2- 172.21.15.64:0/3984876691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c081027a0 0x7f6c081067f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009900 tx=0x7f6bf002a410 comp rx=0 tx=0).stop 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 -- 172.21.15.64:0/3984876691 shutdown_connections 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 --2- 172.21.15.64:0/3984876691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c08106f20 0x7f6c0810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 --2- 172.21.15.64:0/3984876691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c081027a0 0x7f6c081067f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 --2- 172.21.15.64:0/3984876691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c08101df0 0x7f6c081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 -- 172.21.15.64:0/3984876691 >> 172.21.15.64:0/3984876691 conn(0x7f6c080fd8f0 msgr2=0x7f6c080ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 -- 172.21.15.64:0/3984876691 shutdown_connections 2024-09-19T05:04:54.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 -- 172.21.15.64:0/3984876691 wait complete. 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 Processor -- start 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.804+0000 7f6c0d698640 1 -- start start 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c08101df0 0x7f6c08198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c081027a0 0x7f6c08198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c08106f20 0x7f6c0819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08111e30 con 0x7f6c08106f20 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08111cb0 con 0x7f6c081027a0 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08111fb0 con 0x7f6c08101df0 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c08106f20 0x7f6c0819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c08106f20 0x7f6c0819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51786/0 (socket says 172.21.15.64:51786) 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 -- 172.21.15.64:0/2025654752 learned_addr learned my addr 172.21.15.64:0/2025654752 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c06ffd640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c08101df0 0x7f6c08198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c067fc640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c081027a0 0x7f6c08198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 -- 172.21.15.64:0/2025654752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c08101df0 msgr2=0x7f6c08198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:54.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c08101df0 0x7f6c08198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 -- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c081027a0 msgr2=0x7f6c08198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c081027a0 0x7f6c08198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 -- 172.21.15.64:0/2025654752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0819d2b0 con 0x7f6c08106f20 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c067fc640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c081027a0 0x7f6c08198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c077fe640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c08106f20 0x7f6c0819cbd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800cce0 tx=0x7f6bf800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800f450 con 0x7f6c08106f20 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0819d5a0 con 0x7f6c08106f20 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bf800faa0 con 0x7f6c08106f20 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8018b70 con 0x7f6c08106f20 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.805+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c081a4e80 con 0x7f6c08106f20 2024-09-19T05:04:54.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.806+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c08062900 con 0x7f6c08106f20 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.808+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6bf8018d10 con 0x7f6c08106f20 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.808+0000 7f6bfffff640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bf40a32d0 0x7f6bf40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.808+0000 7f6c06ffd640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bf40a32d0 0x7f6bf40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.808+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f6bf80d01e0 con 0x7f6c08106f20 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.809+0000 7f6bfffff640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] conn(0x7f6bf40acf80 0x7f6bf40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.809+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 --> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bf8015e70 con 0x7f6bf40acf80 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.809+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f6bf80d0630 con 0x7f6c08106f20 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.809+0000 7f6c06ffd640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bf40a32d0 0x7f6bf40a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c000058c0 tx=0x7f6c0000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.809+0000 7f6c067fc640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] conn(0x7f6bf40acf80 0x7f6bf40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.809+0000 7f6c067fc640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] conn(0x7f6bf40acf80 0x7f6bf40af420 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.810+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== osd.7 v2:172.21.15.103:6816/944387500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bf8015e70 con 0x7f6bf40acf80 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.824+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 --> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c08062d00 con 0x7f6bf40acf80 2024-09-19T05:04:54.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6bfffff640 1 -- 172.21.15.64:0/2025654752 <== osd.7 v2:172.21.15.103:6816/944387500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6c08062d00 con 0x7f6bf40acf80 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] conn(0x7f6bf40acf80 msgr2=0x7f6bf40af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] conn(0x7f6bf40acf80 0x7f6bf40af420 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bf40a32d0 msgr2=0x7f6bf40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bf40a32d0 0x7f6bf40a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c000058c0 tx=0x7f6c0000a4c0 comp rx=0 tx=0).stop 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c08106f20 msgr2=0x7f6c0819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 --2- 172.21.15.64:0/2025654752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c08106f20 0x7f6c0819cbd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800cce0 tx=0x7f6bf800a620 comp rx=0 tx=0).stop 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c077fe640 1 -- 172.21.15.64:0/2025654752 reap_dead start 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 shutdown_connections 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 >> 172.21.15.64:0/2025654752 conn(0x7f6c080fd8f0 msgr2=0x7f6c080fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.825+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 shutdown_connections 2024-09-19T05:04:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:52.826+0000 7f6c0d698640 1 -- 172.21.15.64:0/2025654752 wait complete. 2024-09-19T05:04:54.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:54 smithi174 ceph-mon[34488]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:04:55.154 INFO:teuthology.orchestra.run.smithi064.stdout:146028888115 2024-09-19T05:04:55.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T05:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:55.805 INFO:teuthology.orchestra.run.smithi064.stdout:176093659180 2024-09-19T05:04:55.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T05:04:56.325 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:56 smithi064 ceph-mon[30960]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:04:56.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:56 smithi103 ceph-mon[34446]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:04:56.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:56 smithi174 ceph-mon[34488]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:04:56.720 INFO:teuthology.orchestra.run.smithi064.stdout:193273528361 2024-09-19T05:04:56.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T05:04:56.746 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:56.746 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:57.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/1213477406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102ef0 msgr2=0x7f9f6810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/1213477406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102ef0 0x7f9f6810da40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009ca0 tx=0x7f9f6002a950 comp rx=0 tx=0).stop 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/1213477406 shutdown_connections 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/1213477406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102ef0 0x7f9f6810da40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/1213477406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68102570 0x7f9f681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/1213477406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f68100e20 0x7f9f68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/1213477406 >> 172.21.15.64:0/1213477406 conn(0x7f9f680fc7d0 msgr2=0x7f9f680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/1213477406 shutdown_connections 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/1213477406 wait complete. 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.347+0000 7f9f6ffb8640 1 Processor -- start 2024-09-19T05:04:57.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 -- start start 2024-09-19T05:04:57.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68100e20 0x7f9f68199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:57.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 0x7f9f68199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:57.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f68102ef0 0x7f9f681a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:57.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68110e50 con 0x7f9f68102570 2024-09-19T05:04:57.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68110cd0 con 0x7f9f68100e20 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6ffb8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68110fd0 con 0x7f9f68102ef0 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 0x7f9f68199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 0x7f9f68199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48402/0 (socket says 172.21.15.64:48402) 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 -- 172.21.15.64:0/3776454446 learned_addr learned my addr 172.21.15.64:0/3776454446 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6dd54640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68100e20 0x7f9f68199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 -- 172.21.15.64:0/3776454446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f68102ef0 msgr2=0x7f9f681a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f68102ef0 0x7f9f681a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 -- 172.21.15.64:0/3776454446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68100e20 msgr2=0x7f9f68199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68100e20 0x7f9f68199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6d553640 1 -- 172.21.15.64:0/3776454446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f681a48e0 con 0x7f9f68102570 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.348+0000 7f9f6dd54640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68100e20 0x7f9f68199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.349+0000 7f9f6d553640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 0x7f9f68199860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c890 tx=0x7f9f5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:57.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.349+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58004220 con 0x7f9f68102570 2024-09-19T05:04:57.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.349+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f681a4ab0 con 0x7f9f68102570 2024-09-19T05:04:57.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.349+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f580043c0 con 0x7f9f68102570 2024-09-19T05:04:57.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.349+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58004a00 con 0x7f9f68102570 2024-09-19T05:04:57.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.349+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f681a4fc0 con 0x7f9f68102570 2024-09-19T05:04:57.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.350+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f681021b0 con 0x7f9f68102570 2024-09-19T05:04:57.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.351+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9f58004ba0 con 0x7f9f68102570 2024-09-19T05:04:57.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.352+0000 7f9f5effd640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9f540a32d0 0x7f9f540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:57.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.352+0000 7f9f6dd54640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9f540a32d0 0x7f9f540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:57.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.352+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f9f580cfc50 con 0x7f9f68102570 2024-09-19T05:04:57.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.352+0000 7f9f6dd54640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9f540a32d0 0x7f9f540a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009930 tx=0x7f9f64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:57.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.355+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f58014030 con 0x7f9f68102570 2024-09-19T05:04:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.535+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9f68101200 con 0x7f9f68102570 2024-09-19T05:04:57.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.536+0000 7f9f5effd640 1 -- 172.21.15.64:0/3776454446 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f68101200 con 0x7f9f68102570 2024-09-19T05:04:57.537 INFO:teuthology.orchestra.run.smithi064.stdout:73014444099 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9f540a32d0 msgr2=0x7f9f540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9f540a32d0 0x7f9f540a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009930 tx=0x7f9f64008040 comp rx=0 tx=0).stop 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 msgr2=0x7f9f68199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 0x7f9f68199860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c890 tx=0x7f9f5800cd50 comp rx=0 tx=0).stop 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 shutdown_connections 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9f540a32d0 0x7f9f540a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f68102ef0 0x7f9f681a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f68102570 0x7f9f68199860 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 --2- 172.21.15.64:0/3776454446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f68100e20 0x7f9f68199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.538+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 >> 172.21.15.64:0/3776454446 conn(0x7f9f680fc7d0 msgr2=0x7f9f680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.539+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 shutdown_connections 2024-09-19T05:04:57.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:57.539+0000 7f9f6ffb8640 1 -- 172.21.15.64:0/3776454446 wait complete. 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 -- 172.21.15.64:0/3263553874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 msgr2=0x7f44641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 --2- 172.21.15.64:0/3263553874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f44641067e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f445002a410 comp rx=0 tx=0).stop 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 -- 172.21.15.64:0/3263553874 shutdown_connections 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 --2- 172.21.15.64:0/3263553874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464106f10 0x7f446410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 --2- 172.21.15.64:0/3263553874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f44641067e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 --2- 172.21.15.64:0/3263553874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464101de0 0x7f44641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 -- 172.21.15.64:0/3263553874 >> 172.21.15.64:0/3263553874 conn(0x7f44640fd8c0 msgr2=0x7f44640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 -- 172.21.15.64:0/3263553874 shutdown_connections 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.078+0000 7f446afc3640 1 -- 172.21.15.64:0/3263553874 wait complete. 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 Processor -- start 2024-09-19T05:04:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 -- start start 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464101de0 0x7f4464198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f4464198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464106f10 0x7f446419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4464111e40 con 0x7f4464102790 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4464111cc0 con 0x7f4464101de0 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f446afc3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4464111fc0 con 0x7f4464106f10 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f4464198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f4464198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48422/0 (socket says 172.21.15.64:48422) 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 -- 172.21.15.64:0/2475963453 learned_addr learned my addr 172.21.15.64:0/2475963453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4469560640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464106f10 0x7f446419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4468d5f640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464101de0 0x7f4464198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 -- 172.21.15.64:0/2475963453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464106f10 msgr2=0x7f446419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464106f10 0x7f446419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 -- 172.21.15.64:0/2475963453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464101de0 msgr2=0x7f4464198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464101de0 0x7f4464198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4463fff640 1 -- 172.21.15.64:0/2475963453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446419d3f0 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4468d5f640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464101de0 0x7f4464198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.079+0000 7f4469560640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464106f10 0x7f446419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.080+0000 7f4463fff640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f4464198980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4450005940 tx=0x7f4450004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.080+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445003f030 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.080+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4450003aa0 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.080+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446419d680 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.080+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450035070 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.080+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641a4fc0 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.081+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44641032c0 con 0x7f4464102790 2024-09-19T05:04:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.082+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4450042030 con 0x7f4464102790 2024-09-19T05:04:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.083+0000 7f4461ffb640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44540a32d0 0x7f44540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.083+0000 7f4468d5f640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44540a32d0 0x7f44540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.083+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f44500efff0 con 0x7f4464102790 2024-09-19T05:04:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.083+0000 7f4468d5f640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44540a32d0 0x7f44540a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4458006fd0 tx=0x7f4458008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.087+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44500bcb90 con 0x7f4464102790 2024-09-19T05:04:58.254 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:58 smithi064 ceph-mon[30960]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:04:58.254 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:58 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3776454446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:04:58.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 -- 172.21.15.64:0/364434018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750102d30 msgr2=0x7f3750103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 --2- 172.21.15.64:0/364434018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750102d30 0x7f3750103110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3738009900 tx=0x7f373802a480 comp rx=0 tx=0).stop 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 -- 172.21.15.64:0/364434018 shutdown_connections 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 --2- 172.21.15.64:0/364434018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375010c160 0x7f375010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 --2- 172.21.15.64:0/364434018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750103650 0x7f375010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 --2- 172.21.15.64:0/364434018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750102d30 0x7f3750103110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.252+0000 7f3755599640 1 -- 172.21.15.64:0/364434018 >> 172.21.15.64:0/364434018 conn(0x7f37500fc790 msgr2=0x7f37500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.253+0000 7f3755599640 1 -- 172.21.15.64:0/364434018 shutdown_connections 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.253+0000 7f3755599640 1 -- 172.21.15.64:0/364434018 wait complete. 2024-09-19T05:04:58.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.253+0000 7f3755599640 1 Processor -- start 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 -- start start 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750102d30 0x7f3750100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 0x7f3750100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375010c160 0x7f3750101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3750111e60 con 0x7f3750103650 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3750111ce0 con 0x7f375010c160 2024-09-19T05:04:58.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f3755599640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3750111fe0 con 0x7f3750102d30 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f374e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 0x7f3750100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f374effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750102d30 0x7f3750100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f374e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 0x7f3750100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48434/0 (socket says 172.21.15.64:48434) 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f374e7fc640 1 -- 172.21.15.64:0/2219783666 learned_addr learned my addr 172.21.15.64:0/2219783666 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.254+0000 7f374f7fe640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375010c160 0x7f3750101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374e7fc640 1 -- 172.21.15.64:0/2219783666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750102d30 msgr2=0x7f3750100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374e7fc640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750102d30 0x7f3750100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374e7fc640 1 -- 172.21.15.64:0/2219783666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375010c160 msgr2=0x7f3750101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374e7fc640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375010c160 0x7f3750101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374e7fc640 1 -- 172.21.15.64:0/2219783666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37501a67d0 con 0x7f3750103650 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374effd640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750102d30 0x7f3750100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f374e7fc640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 0x7f3750100ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f374000c970 tx=0x7f374000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740007bb0 con 0x7f3750103650 2024-09-19T05:04:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3740007d50 con 0x7f3750103650 2024-09-19T05:04:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37501a6ac0 con 0x7f3750103650 2024-09-19T05:04:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.255+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740018780 con 0x7f3750103650 2024-09-19T05:04:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.256+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a6fb0 con 0x7f3750103650 2024-09-19T05:04:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.257+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3750103110 con 0x7f3750103650 2024-09-19T05:04:58.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.258+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f374000f450 con 0x7f3750103650 2024-09-19T05:04:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.259+0000 7f3747fff640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f373c0a32d0 0x7f373c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.259+0000 7f374effd640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f373c0a32d0 0x7f373c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.259+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f37400cfae0 con 0x7f3750103650 2024-09-19T05:04:58.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.259+0000 7f374effd640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f373c0a32d0 0x7f373c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3738005ab0 tx=0x7f3738005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.263+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3740014030 con 0x7f3750103650 2024-09-19T05:04:58.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.268+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f446410a120 con 0x7f4464102790 2024-09-19T05:04:58.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.269+0000 7f4461ffb640 1 -- 172.21.15.64:0/2475963453 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44500bc490 con 0x7f4464102790 2024-09-19T05:04:58.270 INFO:teuthology.orchestra.run.smithi064.stdout:34359738446 2024-09-19T05:04:58.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44540a32d0 msgr2=0x7f44540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44540a32d0 0x7f44540a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4458006fd0 tx=0x7f4458008040 comp rx=0 tx=0).stop 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 msgr2=0x7f4464198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f4464198980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4450005940 tx=0x7f4450004270 comp rx=0 tx=0).stop 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 shutdown_connections 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44540a32d0 0x7f44540a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4464106f10 0x7f446419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464102790 0x7f4464198980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 --2- 172.21.15.64:0/2475963453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4464101de0 0x7f4464198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 >> 172.21.15.64:0/2475963453 conn(0x7f44640fd8c0 msgr2=0x7f44640fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 shutdown_connections 2024-09-19T05:04:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.271+0000 7f446afc3640 1 -- 172.21.15.64:0/2475963453 wait complete. 2024-09-19T05:04:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.449+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f37501071d0 con 0x7f3750103650 2024-09-19T05:04:58.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.450+0000 7f3747fff640 1 -- 172.21.15.64:0/2219783666 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f374009c680 con 0x7f3750103650 2024-09-19T05:04:58.451 INFO:teuthology.orchestra.run.smithi064.stdout:304942678039 2024-09-19T05:04:58.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f373c0a32d0 msgr2=0x7f373c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f373c0a32d0 0x7f373c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3738005ab0 tx=0x7f3738005a20 comp rx=0 tx=0).stop 2024-09-19T05:04:58.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 msgr2=0x7f3750100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 0x7f3750100ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f374000c970 tx=0x7f374000ce30 comp rx=0 tx=0).stop 2024-09-19T05:04:58.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 shutdown_connections 2024-09-19T05:04:58.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f373c0a32d0 0x7f373c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375010c160 0x7f3750101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3750103650 0x7f3750100ed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 --2- 172.21.15.64:0/2219783666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3750102d30 0x7f3750100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 >> 172.21.15.64:0/2219783666 conn(0x7f37500fc790 msgr2=0x7f37500fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:58.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 shutdown_connections 2024-09-19T05:04:58.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.453+0000 7f3755599640 1 -- 172.21.15.64:0/2219783666 wait complete. 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 -- 172.21.15.64:0/2990668570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850108290 msgr2=0x7fd85010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 --2- 172.21.15.64:0/2990668570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850108290 0x7fd85010ec30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd848009cd0 tx=0x7fd84802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 -- 172.21.15.64:0/2990668570 shutdown_connections 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 --2- 172.21.15.64:0/2990668570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850108290 0x7fd85010ec30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 --2- 172.21.15.64:0/2990668570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd850103740 0x7fd850107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 --2- 172.21.15.64:0/2990668570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd850102d90 0x7fd850103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 -- 172.21.15.64:0/2990668570 >> 172.21.15.64:0/2990668570 conn(0x7fd8500fc7d0 msgr2=0x7fd8500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 -- 172.21.15.64:0/2990668570 shutdown_connections 2024-09-19T05:04:58.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.437+0000 7fd857d9d640 1 -- 172.21.15.64:0/2990668570 wait complete. 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 Processor -- start 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 -- start start 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850102d90 0x7fd850198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd850103740 0x7fd8501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd850108290 0x7fd85019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd850111f40 con 0x7fd850102d90 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd850111dc0 con 0x7fd850108290 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd857d9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501120c0 con 0x7fd850103740 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd855b39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850102d90 0x7fd850198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd855b39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850102d90 0x7fd850198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48454/0 (socket says 172.21.15.64:48454) 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd855b39640 1 -- 172.21.15.64:0/3940492420 learned_addr learned my addr 172.21.15.64:0/3940492420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:04:58.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.438+0000 7fd855338640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd850103740 0x7fd8501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd85633a640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd850108290 0x7fd85019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd855b39640 1 -- 172.21.15.64:0/3940492420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd850103740 msgr2=0x7fd8501989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd855b39640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd850103740 0x7fd8501989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd855b39640 1 -- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd850108290 msgr2=0x7fd85019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd855b39640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd850108290 0x7fd85019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd855b39640 1 -- 172.21.15.64:0/3940492420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85019d430 con 0x7fd850102d90 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd85633a640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd850108290 0x7fd85019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd855b39640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850102d90 0x7fd850198480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00d950 tx=0x7fd84c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c017070 con 0x7fd850102d90 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85019d720 con 0x7fd850102d90 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd84c009c30 con 0x7fd850102d90 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00f620 con 0x7fd850102d90 2024-09-19T05:04:58.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.439+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8501a5000 con 0x7fd850102d90 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.440+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd850062980 con 0x7fd850102d90 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.442+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd84c0109a0 con 0x7fd850102d90 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.442+0000 7fd846ffd640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd83c0a32d0 0x7fd83c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.442+0000 7fd855338640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd83c0a32d0 0x7fd83c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.442+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fd84c0cfac0 con 0x7fd850102d90 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.443+0000 7fd846ffd640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] conn(0x7fd83c0acf80 0x7fd83c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.443+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 --> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd83c0afaf0 con 0x7fd83c0acf80 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.443+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fd84c0cff10 con 0x7fd850102d90 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.443+0000 7fd855338640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd83c0a32d0 0x7fd83c0a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8500fee10 tx=0x7fd84000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.443+0000 7fd85633a640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] conn(0x7fd83c0acf80 0x7fd83c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.443+0000 7fd85633a640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] conn(0x7fd83c0acf80 0x7fd83c0af420 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:58.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.444+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== osd.10 v2:172.21.15.103:6824/1109323153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd83c0afaf0 con 0x7fd83c0acf80 2024-09-19T05:04:58.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.460+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 --> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd850062d80 con 0x7fd83c0acf80 2024-09-19T05:04:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.460+0000 7fd846ffd640 1 -- 172.21.15.64:0/3940492420 <== osd.10 v2:172.21.15.103:6824/1109323153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd850062d80 con 0x7fd83c0acf80 2024-09-19T05:04:58.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] conn(0x7fd83c0acf80 msgr2=0x7fd83c0af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] conn(0x7fd83c0acf80 0x7fd83c0af420 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd83c0a32d0 msgr2=0x7fd83c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd83c0a32d0 0x7fd83c0a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8500fee10 tx=0x7fd84000b390 comp rx=0 tx=0).stop 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850102d90 msgr2=0x7fd850198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 --2- 172.21.15.64:0/3940492420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd850102d90 0x7fd850198480 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00d950 tx=0x7fd84c00de10 comp rx=0 tx=0).stop 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd85633a640 1 -- 172.21.15.64:0/3940492420 reap_dead start 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 shutdown_connections 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 >> 172.21.15.64:0/3940492420 conn(0x7fd8500fc7d0 msgr2=0x7fd850105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 shutdown_connections 2024-09-19T05:04:58.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:04:58.461+0000 7fd857d9d640 1 -- 172.21.15.64:0/3940492420 wait complete. 2024-09-19T05:04:58.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:58 smithi103 ceph-mon[34446]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:04:58.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:58 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3776454446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:04:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:58 smithi174 ceph-mon[34488]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:04:58.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:58 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3776454446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:04:59.303 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:59 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2475963453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:04:59.304 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:04:59 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2219783666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:04:59.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738446 for osd.0 2024-09-19T05:04:59.342 DEBUG:teuthology.parallel:result is None 2024-09-19T05:04:59.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:59 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2475963453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:04:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:04:59 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2219783666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:04:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:59 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2475963453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:04:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:04:59 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2219783666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:04:59.787 INFO:teuthology.orchestra.run.smithi064.stdout:274877906970 2024-09-19T05:04:59.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T05:05:00.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444096 got 73014444099 for osd.2 2024-09-19T05:05:00.176 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:00.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:00 smithi064 ceph-mon[30960]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:05:00.428 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:00 smithi174 ceph-mon[34488]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:05:00.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:00 smithi103 ceph-mon[34446]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:05:00.787 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678037 got 304942678039 for osd.11 2024-09-19T05:05:00.787 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:00.812 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:00.813 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:00.845 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:01.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.503+0000 7f2326c35640 1 -- 172.21.15.64:0/3457164067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102570 msgr2=0x7f23201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.504+0000 7f2326c35640 1 --2- 172.21.15.64:0/3457164067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102570 0x7f23201029b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f230c009900 tx=0x7f230c02a480 comp rx=0 tx=0).stop 2024-09-19T05:05:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.504+0000 7f2326c35640 1 -- 172.21.15.64:0/3457164067 shutdown_connections 2024-09-19T05:05:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.504+0000 7f2326c35640 1 --2- 172.21.15.64:0/3457164067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320102ef0 0x7f232010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.504+0000 7f2326c35640 1 --2- 172.21.15.64:0/3457164067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102570 0x7f23201029b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.504+0000 7f2326c35640 1 --2- 172.21.15.64:0/3457164067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320100e20 0x7f2320101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.504+0000 7f2326c35640 1 -- 172.21.15.64:0/3457164067 >> 172.21.15.64:0/3457164067 conn(0x7f23200fc7d0 msgr2=0x7f23200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:01.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.505+0000 7f2326c35640 1 -- 172.21.15.64:0/3457164067 shutdown_connections 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.505+0000 7f2326c35640 1 -- 172.21.15.64:0/3457164067 wait complete. 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.505+0000 7f2326c35640 1 Processor -- start 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.505+0000 7f2326c35640 1 -- start start 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f2326c35640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 0x7f2320197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f2326c35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320102570 0x7f2320197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f2326c35640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320102ef0 0x7f232019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f2326c35640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2320110e50 con 0x7f2320100e20 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f2326c35640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2320110cd0 con 0x7f2320102570 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f2326c35640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2320110fd0 con 0x7f2320102ef0 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 0x7f2320197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 0x7f2320197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48476/0 (socket says 172.21.15.64:48476) 2024-09-19T05:05:01.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 -- 172.21.15.64:0/849909361 learned_addr learned my addr 172.21.15.64:0/849909361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f231ffff640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320102570 0x7f2320197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23251d2640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320102ef0 0x7f232019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 -- 172.21.15.64:0/849909361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320102ef0 msgr2=0x7f232019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320102ef0 0x7f232019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 -- 172.21.15.64:0/849909361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320102570 msgr2=0x7f2320197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320102570 0x7f2320197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 -- 172.21.15.64:0/849909361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232019c300 con 0x7f2320100e20 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f23249d1640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 0x7f2320197350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f231400e9f0 tx=0x7f231400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.506+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231400cd80 con 0x7f2320100e20 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.507+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232019c5f0 con 0x7f2320100e20 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.507+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23140108e0 con 0x7f2320100e20 2024-09-19T05:05:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.507+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314018920 con 0x7f2320100e20 2024-09-19T05:05:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.507+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2320069f70 con 0x7f2320100e20 2024-09-19T05:05:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.508+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23201021b0 con 0x7f2320100e20 2024-09-19T05:05:01.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.509+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2314010a90 con 0x7f2320100e20 2024-09-19T05:05:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.510+0000 7f231dffb640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23100a32d0 0x7f23100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.510+0000 7f231ffff640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23100a32d0 0x7f23100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.510+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f23140d0320 con 0x7f2320100e20 2024-09-19T05:05:01.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.511+0000 7f231ffff640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23100a32d0 0x7f23100a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f230c009230 tx=0x7f230c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:01.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.513+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f231409cec0 con 0x7f2320100e20 2024-09-19T05:05:01.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.690+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2320101200 con 0x7f2320100e20 2024-09-19T05:05:01.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.690+0000 7f231dffb640 1 -- 172.21.15.64:0/849909361 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2320101200 con 0x7f2320100e20 2024-09-19T05:05:01.692 INFO:teuthology.orchestra.run.smithi064.stdout:98784247870 2024-09-19T05:05:01.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23100a32d0 msgr2=0x7f23100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23100a32d0 0x7f23100a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f230c009230 tx=0x7f230c0058d0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 msgr2=0x7f2320197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 0x7f2320197350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f231400e9f0 tx=0x7f231400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 shutdown_connections 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f23100a32d0 0x7f23100a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320102ef0 0x7f232019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320102570 0x7f2320197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 --2- 172.21.15.64:0/849909361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320100e20 0x7f2320197350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.692+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 >> 172.21.15.64:0/849909361 conn(0x7f23200fc7d0 msgr2=0x7f23200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.693+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 shutdown_connections 2024-09-19T05:05:01.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.693+0000 7f2326c35640 1 -- 172.21.15.64:0/849909361 wait complete. 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/2869230023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f810e500 msgr2=0x7fd0f8110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/2869230023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f810e500 0x7fd0f8110910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009ca0 tx=0x7fd0f002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/2869230023 shutdown_connections 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/2869230023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f810e500 0x7fd0f8110910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/2869230023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f8101440 0x7fd0f810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/2869230023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0f8100b20 0x7fd0f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/2869230023 >> 172.21.15.64:0/2869230023 conn(0x7fd0f80fc7d0 msgr2=0x7fd0f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/2869230023 shutdown_connections 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/2869230023 wait complete. 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 Processor -- start 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.753+0000 7fd0ff0ac640 1 -- start start 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0f8100b20 0x7fd0f819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 0x7fd0f819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f810e500 0x7fd0f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8114100 con 0x7fd0f8101440 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8113f80 con 0x7fd0f8100b20 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8114280 con 0x7fd0f810e500 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 0x7fd0f819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 0x7fd0f819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48490/0 (socket says 172.21.15.64:48490) 2024-09-19T05:05:01.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 -- 172.21.15.64:0/3680275087 learned_addr learned my addr 172.21.15.64:0/3680275087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:01.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0fce48640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0f8100b20 0x7fd0f819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0fd649640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f810e500 0x7fd0f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 -- 172.21.15.64:0/3680275087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f810e500 msgr2=0x7fd0f819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f810e500 0x7fd0f819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 -- 172.21.15.64:0/3680275087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0f8100b20 msgr2=0x7fd0f819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0f8100b20 0x7fd0f819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 -- 172.21.15.64:0/3680275087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f819f5d0 con 0x7fd0f8101440 2024-09-19T05:05:01.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0fd649640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f810e500 0x7fd0f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:01.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f7fff640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 0x7fd0f819ab60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c970 tx=0x7fd0e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:01.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004220 con 0x7fd0f8101440 2024-09-19T05:05:01.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e40043c0 con 0x7fd0f8101440 2024-09-19T05:05:01.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f819f8c0 con 0x7fd0f8101440 2024-09-19T05:05:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.754+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004a00 con 0x7fd0f8101440 2024-09-19T05:05:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.755+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f81a71a0 con 0x7fd0f8101440 2024-09-19T05:05:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.756+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f8100f00 con 0x7fd0f8101440 2024-09-19T05:05:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.757+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd0e40079b0 con 0x7fd0f8101440 2024-09-19T05:05:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.758+0000 7fd0f5ffb640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd0e80a32d0 0x7fd0e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:01.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.758+0000 7fd0fce48640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd0e80a32d0 0x7fd0e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.758+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fd0e40cf7f0 con 0x7fd0f8101440 2024-09-19T05:05:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.758+0000 7fd0fce48640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd0e80a32d0 0x7fd0e80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009bd0 tx=0x7fd0ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:01.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.762+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0e4014030 con 0x7fd0f8101440 2024-09-19T05:05:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:01.941 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:01.942 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:01.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.942+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd0f8109600 con 0x7fd0f8101440 2024-09-19T05:05:01.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.943+0000 7fd0f5ffb640 1 -- 172.21.15.64:0/3680275087 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0e409c390 con 0x7fd0f8101440 2024-09-19T05:05:01.944 INFO:teuthology.orchestra.run.smithi064.stdout:55834574922 2024-09-19T05:05:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd0e80a32d0 msgr2=0x7fd0e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd0e80a32d0 0x7fd0e80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009bd0 tx=0x7fd0ec005f90 comp rx=0 tx=0).stop 2024-09-19T05:05:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 msgr2=0x7fd0f819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 0x7fd0f819ab60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c970 tx=0x7fd0e400ce30 comp rx=0 tx=0).stop 2024-09-19T05:05:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 shutdown_connections 2024-09-19T05:05:01.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd0e80a32d0 0x7fd0e80a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f810e500 0x7fd0f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f8101440 0x7fd0f819ab60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 --2- 172.21.15.64:0/3680275087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0f8100b20 0x7fd0f819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.946+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 >> 172.21.15.64:0/3680275087 conn(0x7fd0f80fc7d0 msgr2=0x7fd0f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.947+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 shutdown_connections 2024-09-19T05:05:01.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:01.947+0000 7fd0ff0ac640 1 -- 172.21.15.64:0/3680275087 wait complete. 2024-09-19T05:05:02.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:02 smithi064 ceph-mon[30960]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:05:02.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:02 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/849909361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:05:02.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:02 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3680275087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:05:02.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:02 smithi103 ceph-mon[34446]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:05:02.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:02 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/849909361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:05:02.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:02 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3680275087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:05:02.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:02 smithi174 ceph-mon[34488]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:05:02.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:02 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/849909361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:05:02.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:02 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3680275087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:05:02.851 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247868 got 98784247870 for osd.3 2024-09-19T05:05:02.851 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:03.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 -- 172.21.15.64:0/315406975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201082a0 msgr2=0x7f0b2010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 --2- 172.21.15.64:0/315406975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201082a0 0x7f0b2010ec40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009930 tx=0x7f0b1802a420 comp rx=0 tx=0).stop 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 -- 172.21.15.64:0/315406975 shutdown_connections 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 --2- 172.21.15.64:0/315406975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201082a0 0x7f0b2010ec40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 --2- 172.21.15.64:0/315406975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b201036e0 0x7f0b20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 --2- 172.21.15.64:0/315406975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b20102d30 0x7f0b20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 -- 172.21.15.64:0/315406975 >> 172.21.15.64:0/315406975 conn(0x7f0b200fc7d0 msgr2=0x7f0b200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 -- 172.21.15.64:0/315406975 shutdown_connections 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 -- 172.21.15.64:0/315406975 wait complete. 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 Processor -- start 2024-09-19T05:05:03.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.030+0000 7f0b2757f640 1 -- start start 2024-09-19T05:05:03.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b20102d30 0x7f0b201984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 0x7f0b201989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b201082a0 0x7f0b2019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20111f50 con 0x7f0b201036e0 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20111dd0 con 0x7f0b201082a0 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201120d0 con 0x7f0b20102d30 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 0x7f0b201989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 0x7f0b201989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48510/0 (socket says 172.21.15.64:48510) 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 -- 172.21.15.64:0/406753122 learned_addr learned my addr 172.21.15.64:0/406753122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2531b640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b20102d30 0x7f0b201984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b25b1c640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b201082a0 0x7f0b2019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 -- 172.21.15.64:0/406753122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b20102d30 msgr2=0x7f0b201984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:03.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b20102d30 0x7f0b201984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 -- 172.21.15.64:0/406753122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b201082a0 msgr2=0x7f0b2019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b201082a0 0x7f0b2019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 -- 172.21.15.64:0/406753122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019d460 con 0x7f0b201036e0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b24b1a640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 0x7f0b201989f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d740 tx=0x7f0b1400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14017070 con 0x7f0b201036e0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b1400de50 con 0x7f0b201036e0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.031+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2019d750 con 0x7f0b201036e0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.032+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400f620 con 0x7f0b201036e0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.032+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a5030 con 0x7f0b201036e0 2024-09-19T05:05:03.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.032+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b20103110 con 0x7f0b201036e0 2024-09-19T05:05:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.034+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0b1401a070 con 0x7f0b201036e0 2024-09-19T05:05:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.035+0000 7f0b1e7fc640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0b080a3350 0x7f0b080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.035+0000 7f0b2531b640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0b080a3350 0x7f0b080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.035+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f0b140cf320 con 0x7f0b201036e0 2024-09-19T05:05:03.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.035+0000 7f0b2531b640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0b080a3350 0x7f0b080a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005fd0 tx=0x7f0b10009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:03.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.039+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b1409b870 con 0x7f0b201036e0 2024-09-19T05:05:03.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.221+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f0b2010a2e0 con 0x7f0b201036e0 2024-09-19T05:05:03.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.222+0000 7f0b1e7fc640 1 -- 172.21.15.64:0/406753122 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b1409fb70 con 0x7f0b201036e0 2024-09-19T05:05:03.223 INFO:teuthology.orchestra.run.smithi064.stdout:253403070497 2024-09-19T05:05:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0b080a3350 msgr2=0x7f0b080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0b080a3350 0x7f0b080a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005fd0 tx=0x7f0b10009680 comp rx=0 tx=0).stop 2024-09-19T05:05:03.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 msgr2=0x7f0b201989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 0x7f0b201989f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d740 tx=0x7f0b1400dc00 comp rx=0 tx=0).stop 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 shutdown_connections 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0b080a3350 0x7f0b080a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b201082a0 0x7f0b2019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b201036e0 0x7f0b201989f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 --2- 172.21.15.64:0/406753122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b20102d30 0x7f0b201984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 >> 172.21.15.64:0/406753122 conn(0x7f0b200fc7d0 msgr2=0x7f0b20105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 shutdown_connections 2024-09-19T05:05:03.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:03.224+0000 7f0b2757f640 1 -- 172.21.15.64:0/406753122 wait complete. 2024-09-19T05:05:04.187 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:04 smithi174 ceph-mon[34488]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T05:05:04.187 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:04 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/406753122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:05:04.187 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:04 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:04.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:04 smithi103 ceph-mon[34446]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T05:05:04.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:04 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/406753122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:05:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:04 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:04.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070494 got 253403070497 for osd.9 2024-09-19T05:05:04.263 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:04.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:04 smithi064 ceph-mon[30960]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T05:05:04.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:04 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/406753122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:05:04.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:04 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:04.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 -- 172.21.15.64:0/812693333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 msgr2=0x7f35ac10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 --2- 172.21.15.64:0/812693333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac10ec40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009cd0 tx=0x7f35a402ed40 comp rx=0 tx=0).stop 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 -- 172.21.15.64:0/812693333 shutdown_connections 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 --2- 172.21.15.64:0/812693333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac10ec40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 --2- 172.21.15.64:0/812693333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ac1036e0 0x7f35ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 --2- 172.21.15.64:0/812693333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35ac102d30 0x7f35ac103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 -- 172.21.15.64:0/812693333 >> 172.21.15.64:0/812693333 conn(0x7f35ac0fc7d0 msgr2=0x7f35ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 -- 172.21.15.64:0/812693333 shutdown_connections 2024-09-19T05:05:04.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.436+0000 7f35b19f7640 1 -- 172.21.15.64:0/812693333 wait complete. 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 Processor -- start 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 -- start start 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ac102d30 0x7f35ac1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35ac1036e0 0x7f35ac196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10fdd0 con 0x7f35ac1082a0 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10fc50 con 0x7f35ac1036e0 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35b19f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10ff50 con 0x7f35ac102d30 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac19acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43936/0 (socket says 172.21.15.64:43936) 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35ab7fe640 1 -- 172.21.15.64:0/463877487 learned_addr learned my addr 172.21.15.64:0/463877487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35aaffd640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ac102d30 0x7f35ac1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.437+0000 7f35aa7fc640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35ac1036e0 0x7f35ac196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35ab7fe640 1 -- 172.21.15.64:0/463877487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ac102d30 msgr2=0x7f35ac1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ac102d30 0x7f35ac1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35ab7fe640 1 -- 172.21.15.64:0/463877487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35ac1036e0 msgr2=0x7f35ac196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35ac1036e0 0x7f35ac196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35ab7fe640 1 -- 172.21.15.64:0/463877487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac19b3a0 con 0x7f35ac1082a0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35ab7fe640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac19acc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009bf0 tx=0x7f35a4035ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a403c040 con 0x7f35ac1082a0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac19b630 con 0x7f35ac1082a0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35a4047070 con 0x7f35ac1082a0 2024-09-19T05:05:04.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4037ab0 con 0x7f35ac1082a0 2024-09-19T05:05:04.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.438+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1a2f70 con 0x7f35ac1082a0 2024-09-19T05:05:04.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.439+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac06a850 con 0x7f35ac1082a0 2024-09-19T05:05:04.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.440+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f35a402f850 con 0x7f35ac1082a0 2024-09-19T05:05:04.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.441+0000 7f35b09f5640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f35940a32d0 0x7f35940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.441+0000 7f35aaffd640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f35940a32d0 0x7f35940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.441+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f35a40f4a00 con 0x7f35ac1082a0 2024-09-19T05:05:04.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.441+0000 7f35aaffd640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f35940a32d0 0x7f35940a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f359c006fd0 tx=0x7f359c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:04.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.445+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35a40f6e20 con 0x7f35ac1082a0 2024-09-19T05:05:04.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.631+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f35ac197750 con 0x7f35ac1082a0 2024-09-19T05:05:04.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.632+0000 7f35b09f5640 1 -- 172.21.15.64:0/463877487 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35a40c16b0 con 0x7f35ac1082a0 2024-09-19T05:05:04.633 INFO:teuthology.orchestra.run.smithi064.stdout:128849018939 2024-09-19T05:05:04.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f35940a32d0 msgr2=0x7f35940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f35940a32d0 0x7f35940a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f359c006fd0 tx=0x7f359c008040 comp rx=0 tx=0).stop 2024-09-19T05:05:04.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 msgr2=0x7f35ac19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac19acc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009bf0 tx=0x7f35a4035ad0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 shutdown_connections 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f35940a32d0 0x7f35940a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ac1082a0 0x7f35ac19acc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35ac1036e0 0x7f35ac196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 --2- 172.21.15.64:0/463877487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ac102d30 0x7f35ac1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 >> 172.21.15.64:0/463877487 conn(0x7f35ac0fc7d0 msgr2=0x7f35ac105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 shutdown_connections 2024-09-19T05:05:04.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.634+0000 7f35b19f7640 1 -- 172.21.15.64:0/463877487 wait complete. 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 -- 172.21.15.64:0/86074287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 msgr2=0x7fe04010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 --2- 172.21.15.64:0/86074287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04010ec70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe030009930 tx=0x7fe03002a420 comp rx=0 tx=0).stop 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 -- 172.21.15.64:0/86074287 shutdown_connections 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 --2- 172.21.15.64:0/86074287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04010ec70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 --2- 172.21.15.64:0/86074287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe040103780 0x7fe040107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 --2- 172.21.15.64:0/86074287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe040102dd0 0x7fe0401031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 -- 172.21.15.64:0/86074287 >> 172.21.15.64:0/86074287 conn(0x7fe0400fc7d0 msgr2=0x7fe0400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 -- 172.21.15.64:0/86074287 shutdown_connections 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 -- 172.21.15.64:0/86074287 wait complete. 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 Processor -- start 2024-09-19T05:05:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.798+0000 7fe04848f640 1 -- start start 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04848f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe040102dd0 0x7fe04019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04848f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe040103780 0x7fe04019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04848f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04848f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe040111f80 con 0x7fe040108300 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04848f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe040111e00 con 0x7fe040102dd0 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04848f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe040112100 con 0x7fe040103780 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43968/0 (socket says 172.21.15.64:43968) 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 -- 172.21.15.64:0/1777495468 learned_addr learned my addr 172.21.15.64:0/1777495468 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe045a2a640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe040103780 0x7fe04019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe04622b640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe040102dd0 0x7fe04019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 -- 172.21.15.64:0/1777495468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe040103780 msgr2=0x7fe04019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe040103780 0x7fe04019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 -- 172.21.15.64:0/1777495468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe040102dd0 msgr2=0x7fe04019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe040102dd0 0x7fe04019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 -- 172.21.15.64:0/1777495468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04019f630 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe046a2c640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04019ef50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe030005940 tx=0x7fe030004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03003f030 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.799+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe030003aa0 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.800+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04019f8c0 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.800+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030035070 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.800+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401a7200 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.800+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0401031b0 con 0x7fe040108300 2024-09-19T05:05:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.802+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe030042030 con 0x7fe040108300 2024-09-19T05:05:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.802+0000 7fe03f7fe640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe0280a3350 0x7fe0280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.802+0000 7fe04622b640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe0280a3350 0x7fe0280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.802+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fe0300efee0 con 0x7fe040108300 2024-09-19T05:05:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.803+0000 7fe04622b640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe0280a3350 0x7fe0280a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe038005fd0 tx=0x7fe038009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.806+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe0300bca80 con 0x7fe040108300 2024-09-19T05:05:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.986+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe04010c2a0 con 0x7fe040108300 2024-09-19T05:05:04.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.987+0000 7fe03f7fe640 1 -- 172.21.15.64:0/1777495468 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0300bc380 con 0x7fe040108300 2024-09-19T05:05:04.989 INFO:teuthology.orchestra.run.smithi064.stdout:193273528363 2024-09-19T05:05:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe0280a3350 msgr2=0x7fe0280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe0280a3350 0x7fe0280a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe038005fd0 tx=0x7fe038009760 comp rx=0 tx=0).stop 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 msgr2=0x7fe04019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04019ef50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe030005940 tx=0x7fe030004270 comp rx=0 tx=0).stop 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 shutdown_connections 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe0280a3350 0x7fe0280a5810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040108300 0x7fe04019ef50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe040103780 0x7fe04019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 --2- 172.21.15.64:0/1777495468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe040102dd0 0x7fe04019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 >> 172.21.15.64:0/1777495468 conn(0x7fe0400fc7d0 msgr2=0x7fe0400fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 shutdown_connections 2024-09-19T05:05:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:04.989+0000 7fe04848f640 1 -- 172.21.15.64:0/1777495468 wait complete. 2024-09-19T05:05:05.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:05 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/463877487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:05:05.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:05 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:05 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1777495468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:05:05.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:05 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/463877487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:05:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:05 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:05 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1777495468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:05:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:05 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/463877487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:05:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:05 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:05.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:05 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1777495468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/1971067285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc106f10 msgr2=0x7fc3dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/1971067285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc106f10 0x7fc3dc10ea90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009ca0 tx=0x7fc3d402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/1971067285 shutdown_connections 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/1971067285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc106f10 0x7fc3dc10ea90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/1971067285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3dc102790 0x7fc3dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/1971067285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc101de0 0x7fc3dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/1971067285 >> 172.21.15.64:0/1971067285 conn(0x7fc3dc0fd8c0 msgr2=0x7fc3dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:05.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/1971067285 shutdown_connections 2024-09-19T05:05:05.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.743+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/1971067285 wait complete. 2024-09-19T05:05:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 Processor -- start 2024-09-19T05:05:05.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 -- start start 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 0x7fc3dc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc102790 0x7fc3dc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3dc106f10 0x7fc3dc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc111e60 con 0x7fc3dc101de0 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc111ce0 con 0x7fc3dc106f10 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e2ce6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc111fe0 con 0x7fc3dc102790 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.744+0000 7fc3e0a82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 0x7fc3dc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 0x7fc3dc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43998/0 (socket says 172.21.15.64:43998) 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 -- 172.21.15.64:0/2558611345 learned_addr learned my addr 172.21.15.64:0/2558611345 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e1283640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3dc106f10 0x7fc3dc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3dbfff640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc102790 0x7fc3dc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:05.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 -- 172.21.15.64:0/2558611345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc102790 msgr2=0x7fc3dc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc102790 0x7fc3dc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 -- 172.21.15.64:0/2558611345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3dc106f10 msgr2=0x7fc3dc19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3dc106f10 0x7fc3dc19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 -- 172.21.15.64:0/2558611345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc19f510 con 0x7fc3dc101de0 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e0a82640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 0x7fc3dc19a560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b450 tx=0x7fc3d000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0015020 con 0x7fc3dc101de0 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3d0011d40 con 0x7fc3dc101de0 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000f340 con 0x7fc3dc101de0 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.745+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc19f800 con 0x7fc3dc101de0 2024-09-19T05:05:05.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.746+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a70e0 con 0x7fc3dc101de0 2024-09-19T05:05:05.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.747+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc1021c0 con 0x7fc3dc101de0 2024-09-19T05:05:05.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.748+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc3d0018020 con 0x7fc3dc101de0 2024-09-19T05:05:05.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.749+0000 7fc3d9ffb640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc3cc0a32d0 0x7fc3cc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:05.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.749+0000 7fc3dbfff640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc3cc0a32d0 0x7fc3cc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:05.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.749+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fc3d00cf4d0 con 0x7fc3dc101de0 2024-09-19T05:05:05.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.749+0000 7fc3dbfff640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc3cc0a32d0 0x7fc3cc0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0fe1b0 tx=0x7fc3c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:05.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.753+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3d001f020 con 0x7fc3dc101de0 2024-09-19T05:05:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.931+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc3dc1a7310 con 0x7fc3dc101de0 2024-09-19T05:05:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.931+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/2558611345 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc3d000d040 con 0x7fc3dc101de0 2024-09-19T05:05:05.933 INFO:teuthology.orchestra.run.smithi064.stdout:146028888118 2024-09-19T05:05:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc3cc0a32d0 msgr2=0x7fc3cc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc3cc0a32d0 0x7fc3cc0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0fe1b0 tx=0x7fc3c8005f90 comp rx=0 tx=0).stop 2024-09-19T05:05:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 msgr2=0x7fc3dc19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 0x7fc3dc19a560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b450 tx=0x7fc3d000b910 comp rx=0 tx=0).stop 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 shutdown_connections 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fc3cc0a32d0 0x7fc3cc0a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3dc106f10 0x7fc3dc19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc102790 0x7fc3dc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 --2- 172.21.15.64:0/2558611345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3dc101de0 0x7fc3dc19a560 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 >> 172.21.15.64:0/2558611345 conn(0x7fc3dc0fd8c0 msgr2=0x7fc3dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 shutdown_connections 2024-09-19T05:05:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:05.934+0000 7fc3e2ce6640 1 -- 172.21.15.64:0/2558611345 wait complete. 2024-09-19T05:05:06.134 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528361 got 193273528363 for osd.7 2024-09-19T05:05:06.135 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:06.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:06 smithi064 ceph-mon[30960]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-19T05:05:06.368 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:06 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2558611345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:05:06.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:06 smithi103 ceph-mon[34446]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-19T05:05:06.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:06 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2558611345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:05:06.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:06 smithi174 ceph-mon[34488]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-19T05:05:06.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:06 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2558611345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 -- 172.21.15.64:0/3964266488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 msgr2=0x7efce0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 --2- 172.21.15.64:0/3964266488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0103120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009900 tx=0x7efcdc02a480 comp rx=0 tx=0).stop 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 -- 172.21.15.64:0/3964266488 shutdown_connections 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 --2- 172.21.15.64:0/3964266488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce010c170 0x7efce010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 --2- 172.21.15.64:0/3964266488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efce0103660 0x7efce010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 --2- 172.21.15.64:0/3964266488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0103120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 -- 172.21.15.64:0/3964266488 >> 172.21.15.64:0/3964266488 conn(0x7efce00fc7a0 msgr2=0x7efce00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:06.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 -- 172.21.15.64:0/3964266488 shutdown_connections 2024-09-19T05:05:06.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.927+0000 7efce86ed640 1 -- 172.21.15.64:0/3964266488 wait complete. 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 Processor -- start 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 -- start start 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce0103660 0x7efce01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efce010c170 0x7efce019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111e20 con 0x7efce0102d40 2024-09-19T05:05:06.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111ca0 con 0x7efce0103660 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.928+0000 7efce86ed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111fa0 con 0x7efce010c170 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44034/0 (socket says 172.21.15.64:44034) 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 -- 172.21.15.64:0/1332630179 learned_addr learned my addr 172.21.15.64:0/1332630179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce5c88640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce0103660 0x7efce01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6c8a640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efce010c170 0x7efce019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 -- 172.21.15.64:0/1332630179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efce010c170 msgr2=0x7efce019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efce010c170 0x7efce019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 -- 172.21.15.64:0/1332630179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce0103660 msgr2=0x7efce01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce0103660 0x7efce01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 -- 172.21.15.64:0/1332630179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce019d320 con 0x7efce0102d40 2024-09-19T05:05:06.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce6489640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0198370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005940 tx=0x7efcdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc03f030 con 0x7efce0102d40 2024-09-19T05:05:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc003aa0 con 0x7efce0102d40 2024-09-19T05:05:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce019d5b0 con 0x7efce0102d40 2024-09-19T05:05:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc035070 con 0x7efce0102d40 2024-09-19T05:05:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.929+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a4ef0 con 0x7efce0102d40 2024-09-19T05:05:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.930+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce0103120 con 0x7efce0102d40 2024-09-19T05:05:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.932+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efcdc042030 con 0x7efce0102d40 2024-09-19T05:05:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.932+0000 7efcd77fe640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7efccc0a32d0 0x7efccc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.932+0000 7efce5c88640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7efccc0a32d0 0x7efccc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.932+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7efcdc0efc60 con 0x7efce0102d40 2024-09-19T05:05:06.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.933+0000 7efce5c88640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7efccc0a32d0 0x7efccc0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efce010d610 tx=0x7efcd000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.937+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efcdc0bc1b0 con 0x7efce0102d40 2024-09-19T05:05:07.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.112+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7efce01071e0 con 0x7efce0102d40 2024-09-19T05:05:07.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.112+0000 7efcd77fe640 1 -- 172.21.15.64:0/1332630179 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efcdc0c04b0 con 0x7efce0102d40 2024-09-19T05:05:07.114 INFO:teuthology.orchestra.run.smithi064.stdout:223338299431 2024-09-19T05:05:07.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7efccc0a32d0 msgr2=0x7efccc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7efccc0a32d0 0x7efccc0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efce010d610 tx=0x7efcd000a250 comp rx=0 tx=0).stop 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 msgr2=0x7efce0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0198370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efcdc005940 tx=0x7efcdc004270 comp rx=0 tx=0).stop 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 shutdown_connections 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7efccc0a32d0 0x7efccc0a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efce010c170 0x7efce019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce0103660 0x7efce01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 --2- 172.21.15.64:0/1332630179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efce0102d40 0x7efce0198370 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 >> 172.21.15.64:0/1332630179 conn(0x7efce00fc7a0 msgr2=0x7efce00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 shutdown_connections 2024-09-19T05:05:07.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:07.115+0000 7efce86ed640 1 -- 172.21.15.64:0/1332630179 wait complete. 2024-09-19T05:05:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:07 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2005784603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:05:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:07 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1332630179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:05:07.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:07 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2005784603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:05:07.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:07 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1332630179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:05:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:07 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2005784603' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:05:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:07 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1332630179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:05:07.908 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299428 got 223338299431 for osd.8 2024-09-19T05:05:07.908 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:07.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 -- 172.21.15.64:0/1849868854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 msgr2=0x7fb2c810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/1849868854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c810e790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009900 tx=0x7fb2b802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:07.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 -- 172.21.15.64:0/1849868854 shutdown_connections 2024-09-19T05:05:07.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/1849868854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c810e790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/1849868854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c81037a0 0x7fb2c810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/1849868854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c8102df0 0x7fb2c81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 -- 172.21.15.64:0/1849868854 >> 172.21.15.64:0/1849868854 conn(0x7fb2c80fc7d0 msgr2=0x7fb2c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 -- 172.21.15.64:0/1849868854 shutdown_connections 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 -- 172.21.15.64:0/1849868854 wait complete. 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.342+0000 7fb2cd72b640 1 Processor -- start 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 -- start start 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c8102df0 0x7fb2c81a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c81037a0 0x7fb2c81a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8111fc0 con 0x7fb2c810c2f0 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8111e40 con 0x7fb2c8102df0 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2cd72b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8112140 con 0x7fb2c81037a0 2024-09-19T05:05:07.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44018/0 (socket says 172.21.15.64:44018) 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 -- 172.21.15.64:0/2005784603 learned_addr learned my addr 172.21.15.64:0/2005784603 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c67fc640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c81037a0 0x7fb2c81a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c6ffd640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c8102df0 0x7fb2c81a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 -- 172.21.15.64:0/2005784603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c81037a0 msgr2=0x7fb2c81a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c81037a0 0x7fb2c81a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 -- 172.21.15.64:0/2005784603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c8102df0 msgr2=0x7fb2c81a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c8102df0 0x7fb2c81a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 -- 172.21.15.64:0/2005784603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c819ad20 con 0x7fb2c810c2f0 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.343+0000 7fb2c77fe640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c819a5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8004770 tx=0x7fb2b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.344+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b803f030 con 0x7fb2c810c2f0 2024-09-19T05:05:07.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.344+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c819afb0 con 0x7fb2c810c2f0 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.344+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b80318e0 con 0x7fb2c810c2f0 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.344+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8035070 con 0x7fb2c810c2f0 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.344+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a7300 con 0x7fb2c810c2f0 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stdout:274877906972 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.345+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c81031d0 con 0x7fb2c810c2f0 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.346+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb2b8042030 con 0x7fb2c810c2f0 2024-09-19T05:05:07.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.347+0000 7fb2bffff640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2b40a32d0 0x7fb2b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.347+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fb2b80f0010 con 0x7fb2c810c2f0 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.347+0000 7fb2c6ffd640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2b40a32d0 0x7fb2b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.347+0000 7fb2c6ffd640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2b40a32d0 0x7fb2b40a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005fd0 tx=0x7fb2c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.351+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2b80bc600 con 0x7fb2c810c2f0 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.530+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb2c81a74d0 con 0x7fb2c810c2f0 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.531+0000 7fb2bffff640 1 -- 172.21.15.64:0/2005784603 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb2b80c0900 con 0x7fb2c810c2f0 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.532+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2b40a32d0 msgr2=0x7fb2b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.532+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2b40a32d0 0x7fb2b40a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005fd0 tx=0x7fb2c0009500 comp rx=0 tx=0).stop 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.532+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 msgr2=0x7fb2c819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.532+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c819a5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8004770 tx=0x7fb2b8004270 comp rx=0 tx=0).stop 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 shutdown_connections 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2b40a32d0 0x7fb2b40a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c810c2f0 0x7fb2c819a5f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c81037a0 0x7fb2c81a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 --2- 172.21.15.64:0/2005784603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c8102df0 0x7fb2c81a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 >> 172.21.15.64:0/2005784603 conn(0x7fb2c80fc7d0 msgr2=0x7fb2c80fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 shutdown_connections 2024-09-19T05:05:07.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:06.533+0000 7fb2cd72b640 1 -- 172.21.15.64:0/2005784603 wait complete. 2024-09-19T05:05:08.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:08 smithi064 ceph-mon[30960]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:08.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:08 smithi103 ceph-mon[34446]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2266508319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41036e0 msgr2=0x7f1cf4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2266508319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41036e0 0x7f1cf4107b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009930 tx=0x7f1ce802a420 comp rx=0 tx=0).stop 2024-09-19T05:05:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2266508319 shutdown_connections 2024-09-19T05:05:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2266508319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cf41082a0 0x7f1cf410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2266508319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41036e0 0x7f1cf4107b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2266508319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2266508319 >> 172.21.15.64:0/2266508319 conn(0x7f1cf40fc7d0 msgr2=0x7f1cf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2266508319 shutdown_connections 2024-09-19T05:05:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2266508319 wait complete. 2024-09-19T05:05:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 Processor -- start 2024-09-19T05:05:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.470+0000 7f1cfa58d640 1 -- start start 2024-09-19T05:05:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cfa58d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cfa58d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cf41036e0 0x7f1cf41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cfa58d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41082a0 0x7f1cf419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cfa58d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4111f70 con 0x7f1cf41082a0 2024-09-19T05:05:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cfa58d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4111df0 con 0x7f1cf4102d30 2024-09-19T05:05:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cfa58d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41120f0 con 0x7f1cf41036e0 2024-09-19T05:05:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf8b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41082a0 0x7f1cf419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf41a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48198/0 (socket says 172.21.15.64:48198) 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 -- 172.21.15.64:0/2270413428 learned_addr learned my addr 172.21.15.64:0/2270413428 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 -- 172.21.15.64:0/2270413428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cf41036e0 msgr2=0x7f1cf41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cf41036e0 0x7f1cf41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 -- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41082a0 msgr2=0x7f1cf419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41082a0 0x7f1cf419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf3fff640 1 -- 172.21.15.64:0/2270413428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf419aeb0 con 0x7f1cf4102d30 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.471+0000 7f1cf8b2a640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41082a0 0x7f1cf419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.472+0000 7f1cf3fff640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf41a0570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9f0 tx=0x7f1ce400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.472+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400cd80 con 0x7f1cf4102d30 2024-09-19T05:05:08.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.472+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ce40108e0 con 0x7f1cf4102d30 2024-09-19T05:05:08.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.472+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf419b1a0 con 0x7f1cf4102d30 2024-09-19T05:05:08.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.472+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4018920 con 0x7f1cf4102d30 2024-09-19T05:05:08.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.472+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a74f0 con 0x7f1cf4102d30 2024-09-19T05:05:08.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.473+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf406a850 con 0x7f1cf4102d30 2024-09-19T05:05:08.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.474+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1ce4010a90 con 0x7f1cf4102d30 2024-09-19T05:05:08.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.475+0000 7f1cf17fa640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.475+0000 7f1cf37fe640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.475+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f1ce40cff60 con 0x7f1cf4102d30 2024-09-19T05:05:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.475+0000 7f1cf37fe640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80096c0 tx=0x7f1ce8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.478+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ce409cb00 con 0x7f1cf4102d30 2024-09-19T05:05:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:08 smithi174 ceph-mon[34488]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.657+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1cf419bb70 con 0x7f1cf4102d30 2024-09-19T05:05:08.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.658+0000 7f1cf17fa640 1 -- 172.21.15.64:0/2270413428 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ce409c400 con 0x7f1cf4102d30 2024-09-19T05:05:08.659 INFO:teuthology.orchestra.run.smithi064.stdout:176093659185 2024-09-19T05:05:08.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1cdc0a3350 msgr2=0x7f1cdc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80096c0 tx=0x7f1ce8031400 comp rx=0 tx=0).stop 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 msgr2=0x7f1cf41a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf41a0570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9f0 tx=0x7f1ce400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 shutdown_connections 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f1cdc0a3350 0x7f1cdc0a5810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cf41082a0 0x7f1cf419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cf41036e0 0x7f1cf41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 --2- 172.21.15.64:0/2270413428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cf4102d30 0x7f1cf41a0570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 >> 172.21.15.64:0/2270413428 conn(0x7f1cf40fc7d0 msgr2=0x7f1cf410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 shutdown_connections 2024-09-19T05:05:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:08.660+0000 7f1cfa58d640 1 -- 172.21.15.64:0/2270413428 wait complete. 2024-09-19T05:05:08.880 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906972 for osd.10 2024-09-19T05:05:08.880 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:09.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:09 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2270413428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:05:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:09 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2270413428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:05:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:09 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2270413428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:05:09.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659180 got 176093659185 for osd.6 2024-09-19T05:05:09.553 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:09.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574922 for osd.1 2024-09-19T05:05:09.898 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:10.419 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018936 got 128849018939 for osd.4 2024-09-19T05:05:10.420 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:10.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:10 smithi103 ceph-mon[34446]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:10 smithi174 ceph-mon[34488]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:10 smithi064 ceph-mon[30960]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:11.014 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888115 got 146028888118 for osd.5 2024-09-19T05:05:11.014 DEBUG:teuthology.parallel:result is None 2024-09-19T05:05:11.015 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:05:11.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:05:11.666 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 -- 172.21.15.64:0/3103622545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102ef0 msgr2=0x7f2c1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/3103622545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102ef0 0x7f2c1810da40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009ca0 tx=0x7f2c1002a970 comp rx=0 tx=0).stop 2024-09-19T05:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 -- 172.21.15.64:0/3103622545 shutdown_connections 2024-09-19T05:05:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/3103622545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102ef0 0x7f2c1810da40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/3103622545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102570 0x7f2c181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/3103622545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18100e20 0x7f2c18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 -- 172.21.15.64:0/3103622545 >> 172.21.15.64:0/3103622545 conn(0x7f2c180fc7d0 msgr2=0x7f2c180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 -- 172.21.15.64:0/3103622545 shutdown_connections 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.258+0000 7f2c1e59f640 1 -- 172.21.15.64:0/3103622545 wait complete. 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 Processor -- start 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 -- start start 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18110e70 con 0x7f2c18102570 2024-09-19T05:05:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18110cf0 con 0x7f2c18102ef0 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1e59f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18110ff0 con 0x7f2c18100e20 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44068/0 (socket says 172.21.15.64:44068) 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 -- 172.21.15.64:0/375464718 learned_addr learned my addr 172.21.15.64:0/375464718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c1cb3c640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 -- 172.21.15.64:0/375464718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18100e20 msgr2=0x7f2c180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 -- 172.21.15.64:0/375464718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 msgr2=0x7f2c180fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 -- 172.21.15.64:0/375464718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181a6980 con 0x7f2c18102570 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.259+0000 7f2c0ffff640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000d950 tx=0x7f2c0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.260+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00017030 con 0x7f2c18102570 2024-09-19T05:05:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.260+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181a6c70 con 0x7f2c18102570 2024-09-19T05:05:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.260+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00004020 con 0x7f2c18102570 2024-09-19T05:05:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.260+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000047a0 con 0x7f2c18102570 2024-09-19T05:05:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.260+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a71b0 con 0x7f2c18102570 2024-09-19T05:05:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.261+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c18101200 con 0x7f2c18102570 2024-09-19T05:05:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.262+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2c000041d0 con 0x7f2c18102570 2024-09-19T05:05:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.262+0000 7f2c15ffb640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2c040a32d0 0x7f2c040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.263+0000 7f2c17fff640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2c040a32d0 0x7f2c040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.263+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f2c000cf510 con 0x7f2c18102570 2024-09-19T05:05:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.263+0000 7f2c17fff640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2c040a32d0 0x7f2c040a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c080096f0 tx=0x7f2c08006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.267+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2c0009c0b0 con 0x7f2c18102570 2024-09-19T05:05:12.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:12 smithi064 ceph-mon[30960]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:12.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:12 smithi103 ceph-mon[34446]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:12.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.472+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c1810a110 con 0x7f2c040a32d0 2024-09-19T05:05:12.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.479+0000 7f2c15ffb640 1 -- 172.21.15.64:0/375464718 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258718 (secure 0 0 0) 0x7f2c080572b0 con 0x7f2c040a32d0 2024-09-19T05:05:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:12.485 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:05:12.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2c040a32d0 msgr2=0x7f2c040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:12.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2c040a32d0 0x7f2c040a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c080096f0 tx=0x7f2c08006c40 comp rx=0 tx=0).stop 2024-09-19T05:05:12.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 msgr2=0x7f2c180fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:12.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000d950 tx=0x7f2c0000de10 comp rx=0 tx=0).stop 2024-09-19T05:05:12.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 shutdown_connections 2024-09-19T05:05:12.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f2c040a32d0 0x7f2c040a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102ef0 0x7f2c180fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c18102570 0x7f2c180fd1c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 --2- 172.21.15.64:0/375464718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c18100e20 0x7f2c180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:12.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 >> 172.21.15.64:0/375464718 conn(0x7f2c180fc7d0 msgr2=0x7f2c1806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:12.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 shutdown_connections 2024-09-19T05:05:12.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:12.482+0000 7f2c1e59f640 1 -- 172.21.15.64:0/375464718 wait complete. 2024-09-19T05:05:12.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:12 smithi174 ceph-mon[34488]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:13.136 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T05:05:12.395195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":472,"num_read_kb":357,"num_write":347,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365736,"kb_used_data":12312,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576088,"statfs":{"total":1151940427776,"available":1151565914112,"internally_reserved":0,"allocated":12607488,"data_stored":7036953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045762+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:04:20.045762+0000","last_peered":"2024-09-19T05:04:20.045762+0000","last_clean":"2024-09-19T05:04:20.045762+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:04:20.045762+0000","last_undegraded":"2024-09-19T05:04:20.045762+0000","last_fullsized":"2024-09-19T05:04:20.045762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042748+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:04:20.042748+0000","last_peered":"2024-09-19T05:04:20.042748+0000","last_clean":"2024-09-19T05:04:20.042748+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:04:20.042748+0000","last_undegraded":"2024-09-19T05:04:20.042748+0000","last_fullsized":"2024-09-19T05:04:20.042748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042862+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:04:20.042862+0000","last_peered":"2024-09-19T05:04:20.042862+0000","last_clean":"2024-09-19T05:04:20.042862+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:04:20.042862+0000","last_undegraded":"2024-09-19T05:04:20.042862+0000","last_fullsized":"2024-09-19T05:04:20.042862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218922+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:04:20.218922+0000","last_peered":"2024-09-19T05:04:20.218922+0000","last_clean":"2024-09-19T05:04:20.218922+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:04:20.218922+0000","last_undegraded":"2024-09-19T05:04:20.218922+0000","last_fullsized":"2024-09-19T05:04:20.218922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044933+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:04:20.044933+0000","last_peered":"2024-09-19T05:04:20.044933+0000","last_clean":"2024-09-19T05:04:20.044933+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:04:20.044933+0000","last_undegraded":"2024-09-19T05:04:20.044933+0000","last_fullsized":"2024-09-19T05:04:20.044933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046644+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:04:20.046644+0000","last_peered":"2024-09-19T05:04:20.046644+0000","last_clean":"2024-09-19T05:04:20.046644+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:04:20.046644+0000","last_undegraded":"2024-09-19T05:04:20.046644+0000","last_fullsized":"2024-09-19T05:04:20.046644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041281+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:04:20.041281+0000","last_peered":"2024-09-19T05:04:20.041281+0000","last_clean":"2024-09-19T05:04:20.041281+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:04:20.041281+0000","last_undegraded":"2024-09-19T05:04:20.041281+0000","last_fullsized":"2024-09-19T05:04:20.041281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045982+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:04:20.045982+0000","last_peered":"2024-09-19T05:04:20.045982+0000","last_clean":"2024-09-19T05:04:20.045982+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:04:20.045982+0000","last_undegraded":"2024-09-19T05:04:20.045982+0000","last_fullsized":"2024-09-19T05:04:20.045982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039683+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:04:20.039683+0000","last_peered":"2024-09-19T05:04:20.039683+0000","last_clean":"2024-09-19T05:04:20.039683+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:04:20.039683+0000","last_undegraded":"2024-09-19T05:04:20.039683+0000","last_fullsized":"2024-09-19T05:04:20.039683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043775+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:04:20.043775+0000","last_peered":"2024-09-19T05:04:20.043775+0000","last_clean":"2024-09-19T05:04:20.043775+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:04:20.043775+0000","last_undegraded":"2024-09-19T05:04:20.043775+0000","last_fullsized":"2024-09-19T05:04:20.043775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041842+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:04:20.041842+0000","last_peered":"2024-09-19T05:04:20.041842+0000","last_clean":"2024-09-19T05:04:20.041842+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:04:20.041842+0000","last_undegraded":"2024-09-19T05:04:20.041842+0000","last_fullsized":"2024-09-19T05:04:20.041842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219054+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:04:20.219054+0000","last_peered":"2024-09-19T05:04:20.219054+0000","last_clean":"2024-09-19T05:04:20.219054+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:04:20.219054+0000","last_undegraded":"2024-09-19T05:04:20.219054+0000","last_fullsized":"2024-09-19T05:04:20.219054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038234+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:04:20.038234+0000","last_peered":"2024-09-19T05:04:20.038234+0000","last_clean":"2024-09-19T05:04:20.038234+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:04:20.038234+0000","last_undegraded":"2024-09-19T05:04:20.038234+0000","last_fullsized":"2024-09-19T05:04:20.038234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042859+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:04:20.042859+0000","last_peered":"2024-09-19T05:04:20.042859+0000","last_clean":"2024-09-19T05:04:20.042859+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:04:20.042859+0000","last_undegraded":"2024-09-19T05:04:20.042859+0000","last_fullsized":"2024-09-19T05:04:20.042859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042616+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:04:20.042616+0000","last_peered":"2024-09-19T05:04:20.042616+0000","last_clean":"2024-09-19T05:04:20.042616+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:04:20.042616+0000","last_undegraded":"2024-09-19T05:04:20.042616+0000","last_fullsized":"2024-09-19T05:04:20.042616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039034+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:04:20.039034+0000","last_peered":"2024-09-19T05:04:20.039034+0000","last_clean":"2024-09-19T05:04:20.039034+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:04:20.039034+0000","last_undegraded":"2024-09-19T05:04:20.039034+0000","last_fullsized":"2024-09-19T05:04:20.039034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045863+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:04:20.045863+0000","last_peered":"2024-09-19T05:04:20.045863+0000","last_clean":"2024-09-19T05:04:20.045863+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:04:20.045863+0000","last_undegraded":"2024-09-19T05:04:20.045863+0000","last_fullsized":"2024-09-19T05:04:20.045863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040764+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:04:20.040764+0000","last_peered":"2024-09-19T05:04:20.040764+0000","last_clean":"2024-09-19T05:04:20.040764+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:04:20.040764+0000","last_undegraded":"2024-09-19T05:04:20.040764+0000","last_fullsized":"2024-09-19T05:04:20.040764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045627+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:04:20.045627+0000","last_peered":"2024-09-19T05:04:20.045627+0000","last_clean":"2024-09-19T05:04:20.045627+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:04:20.045627+0000","last_undegraded":"2024-09-19T05:04:20.045627+0000","last_fullsized":"2024-09-19T05:04:20.045627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045190+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:04:20.045190+0000","last_peered":"2024-09-19T05:04:20.045190+0000","last_clean":"2024-09-19T05:04:20.045190+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:04:20.045190+0000","last_undegraded":"2024-09-19T05:04:20.045190+0000","last_fullsized":"2024-09-19T05:04:20.045190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038280+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:04:20.038280+0000","last_peered":"2024-09-19T05:04:20.038280+0000","last_clean":"2024-09-19T05:04:20.038280+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:04:20.038280+0000","last_undegraded":"2024-09-19T05:04:20.038280+0000","last_fullsized":"2024-09-19T05:04:20.038280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039404+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:04:20.039404+0000","last_peered":"2024-09-19T05:04:20.039404+0000","last_clean":"2024-09-19T05:04:20.039404+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:04:20.039404+0000","last_undegraded":"2024-09-19T05:04:20.039404+0000","last_fullsized":"2024-09-19T05:04:20.039404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046472+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:04:20.046472+0000","last_peered":"2024-09-19T05:04:20.046472+0000","last_clean":"2024-09-19T05:04:20.046472+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:04:20.046472+0000","last_undegraded":"2024-09-19T05:04:20.046472+0000","last_fullsized":"2024-09-19T05:04:20.046472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040224+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:04:20.040224+0000","last_peered":"2024-09-19T05:04:20.040224+0000","last_clean":"2024-09-19T05:04:20.040224+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:04:20.040224+0000","last_undegraded":"2024-09-19T05:04:20.040224+0000","last_fullsized":"2024-09-19T05:04:20.040224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220980+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:04:20.220980+0000","last_peered":"2024-09-19T05:04:20.220980+0000","last_clean":"2024-09-19T05:04:20.220980+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:04:20.220980+0000","last_undegraded":"2024-09-19T05:04:20.220980+0000","last_fullsized":"2024-09-19T05:04:20.220980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044307+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:04:20.044307+0000","last_peered":"2024-09-19T05:04:20.044307+0000","last_clean":"2024-09-19T05:04:20.044307+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:04:20.044307+0000","last_undegraded":"2024-09-19T05:04:20.044307+0000","last_fullsized":"2024-09-19T05:04:20.044307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046219+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:04:20.046219+0000","last_peered":"2024-09-19T05:04:20.046219+0000","last_clean":"2024-09-19T05:04:20.046219+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:04:20.046219+0000","last_undegraded":"2024-09-19T05:04:20.046219+0000","last_fullsized":"2024-09-19T05:04:20.046219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038227+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:04:20.038227+0000","last_peered":"2024-09-19T05:04:20.038227+0000","last_clean":"2024-09-19T05:04:20.038227+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:04:20.038227+0000","last_undegraded":"2024-09-19T05:04:20.038227+0000","last_fullsized":"2024-09-19T05:04:20.038227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042890+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:04:20.042890+0000","last_peered":"2024-09-19T05:04:20.042890+0000","last_clean":"2024-09-19T05:04:20.042890+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:04:20.042890+0000","last_undegraded":"2024-09-19T05:04:20.042890+0000","last_fullsized":"2024-09-19T05:04:20.042890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044794+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:04:20.044794+0000","last_peered":"2024-09-19T05:04:20.044794+0000","last_clean":"2024-09-19T05:04:20.044794+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:04:20.044794+0000","last_undegraded":"2024-09-19T05:04:20.044794+0000","last_fullsized":"2024-09-19T05:04:20.044794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044665+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:04:20.044665+0000","last_peered":"2024-09-19T05:04:20.044665+0000","last_clean":"2024-09-19T05:04:20.044665+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:04:20.044665+0000","last_undegraded":"2024-09-19T05:04:20.044665+0000","last_fullsized":"2024-09-19T05:04:20.044665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045176+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:04:20.045176+0000","last_peered":"2024-09-19T05:04:20.045176+0000","last_clean":"2024-09-19T05:04:20.045176+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:04:20.045176+0000","last_undegraded":"2024-09-19T05:04:20.045176+0000","last_fullsized":"2024-09-19T05:04:20.045176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045192+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:04:20.045192+0000","last_peered":"2024-09-19T05:04:20.045192+0000","last_clean":"2024-09-19T05:04:20.045192+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:04:20.045192+0000","last_undegraded":"2024-09-19T05:04:20.045192+0000","last_fullsized":"2024-09-19T05:04:20.045192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":82,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:07.091041+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:05:07.091041+0000","last_peered":"2024-09-19T05:05:07.091041+0000","last_clean":"2024-09-19T05:05:07.091041+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:05:07.091041+0000","last_undegraded":"2024-09-19T05:05:07.091041+0000","last_fullsized":"2024-09-19T05:05:07.091041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220294+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:04:20.220294+0000","last_peered":"2024-09-19T05:04:20.220294+0000","last_clean":"2024-09-19T05:04:20.220294+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:04:20.220294+0000","last_undegraded":"2024-09-19T05:04:20.220294+0000","last_fullsized":"2024-09-19T05:04:20.220294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043292+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:04:20.043292+0000","last_peered":"2024-09-19T05:04:20.043292+0000","last_clean":"2024-09-19T05:04:20.043292+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:04:20.043292+0000","last_undegraded":"2024-09-19T05:04:20.043292+0000","last_fullsized":"2024-09-19T05:04:20.043292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037995+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:04:20.037995+0000","last_peered":"2024-09-19T05:04:20.037995+0000","last_clean":"2024-09-19T05:04:20.037995+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:04:20.037995+0000","last_undegraded":"2024-09-19T05:04:20.037995+0000","last_fullsized":"2024-09-19T05:04:20.037995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046054+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:04:20.046054+0000","last_peered":"2024-09-19T05:04:20.046054+0000","last_clean":"2024-09-19T05:04:20.046054+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:04:20.046054+0000","last_undegraded":"2024-09-19T05:04:20.046054+0000","last_fullsized":"2024-09-19T05:04:20.046054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040479+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:04:20.040479+0000","last_peered":"2024-09-19T05:04:20.040479+0000","last_clean":"2024-09-19T05:04:20.040479+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:04:20.040479+0000","last_undegraded":"2024-09-19T05:04:20.040479+0000","last_fullsized":"2024-09-19T05:04:20.040479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041190+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:04:20.041190+0000","last_peered":"2024-09-19T05:04:20.041190+0000","last_clean":"2024-09-19T05:04:20.041190+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:04:20.041190+0000","last_undegraded":"2024-09-19T05:04:20.041190+0000","last_fullsized":"2024-09-19T05:04:20.041190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":76,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:07.090319+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:05:07.090319+0000","last_peered":"2024-09-19T05:05:07.090319+0000","last_clean":"2024-09-19T05:05:07.090319+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:05:07.090319+0000","last_undegraded":"2024-09-19T05:05:07.090319+0000","last_fullsized":"2024-09-19T05:05:07.090319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039611+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:04:20.039611+0000","last_peered":"2024-09-19T05:04:20.039611+0000","last_clean":"2024-09-19T05:04:20.039611+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:04:20.039611+0000","last_undegraded":"2024-09-19T05:04:20.039611+0000","last_fullsized":"2024-09-19T05:04:20.039611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.223302+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:04:20.223302+0000","last_peered":"2024-09-19T05:04:20.223302+0000","last_clean":"2024-09-19T05:04:20.223302+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:04:20.223302+0000","last_undegraded":"2024-09-19T05:04:20.223302+0000","last_fullsized":"2024-09-19T05:04:20.223302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044799+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:04:20.044799+0000","last_peered":"2024-09-19T05:04:20.044799+0000","last_clean":"2024-09-19T05:04:20.044799+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:04:20.044799+0000","last_undegraded":"2024-09-19T05:04:20.044799+0000","last_fullsized":"2024-09-19T05:04:20.044799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040554+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:04:20.040554+0000","last_peered":"2024-09-19T05:04:20.040554+0000","last_clean":"2024-09-19T05:04:20.040554+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:04:20.040554+0000","last_undegraded":"2024-09-19T05:04:20.040554+0000","last_fullsized":"2024-09-19T05:04:20.040554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039165+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:04:20.039165+0000","last_peered":"2024-09-19T05:04:20.039165+0000","last_clean":"2024-09-19T05:04:20.039165+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:04:20.039165+0000","last_undegraded":"2024-09-19T05:04:20.039165+0000","last_fullsized":"2024-09-19T05:04:20.039165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":164,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:08.098498+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:05:08.098498+0000","last_peered":"2024-09-19T05:05:08.098498+0000","last_clean":"2024-09-19T05:05:08.098498+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:05:08.098498+0000","last_undegraded":"2024-09-19T05:05:08.098498+0000","last_fullsized":"2024-09-19T05:05:08.098498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":116,"num_write":32,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044465+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:04:20.044465+0000","last_peered":"2024-09-19T05:04:20.044465+0000","last_clean":"2024-09-19T05:04:20.044465+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:04:20.044465+0000","last_undegraded":"2024-09-19T05:04:20.044465+0000","last_fullsized":"2024-09-19T05:04:20.044465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.058790+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:04:20.058790+0000","last_peered":"2024-09-19T05:04:20.058790+0000","last_clean":"2024-09-19T05:04:20.058790+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:04:20.058790+0000","last_undegraded":"2024-09-19T05:04:20.058790+0000","last_fullsized":"2024-09-19T05:04:20.058790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038290+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:04:20.038290+0000","last_peered":"2024-09-19T05:04:20.038290+0000","last_clean":"2024-09-19T05:04:20.038290+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:04:20.038290+0000","last_undegraded":"2024-09-19T05:04:20.038290+0000","last_fullsized":"2024-09-19T05:04:20.038290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040388+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:04:20.040388+0000","last_peered":"2024-09-19T05:04:20.040388+0000","last_clean":"2024-09-19T05:04:20.040388+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:04:20.040388+0000","last_undegraded":"2024-09-19T05:04:20.040388+0000","last_fullsized":"2024-09-19T05:04:20.040388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041839+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:04:20.041839+0000","last_peered":"2024-09-19T05:04:20.041839+0000","last_clean":"2024-09-19T05:04:20.041839+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:04:20.041839+0000","last_undegraded":"2024-09-19T05:04:20.041839+0000","last_fullsized":"2024-09-19T05:04:20.041839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'3","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:08.105699+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:05:08.105699+0000","last_peered":"2024-09-19T05:05:08.105699+0000","last_clean":"2024-09-19T05:05:08.105699+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:05:08.105699+0000","last_undegraded":"2024-09-19T05:05:08.105699+0000","last_fullsized":"2024-09-19T05:05:08.105699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041986+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:04:20.041986+0000","last_peered":"2024-09-19T05:04:20.041986+0000","last_clean":"2024-09-19T05:04:20.041986+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:04:20.041986+0000","last_undegraded":"2024-09-19T05:04:20.041986+0000","last_fullsized":"2024-09-19T05:04:20.041986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044042+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:04:20.044042+0000","last_peered":"2024-09-19T05:04:20.044042+0000","last_clean":"2024-09-19T05:04:20.044042+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:04:20.044042+0000","last_undegraded":"2024-09-19T05:04:20.044042+0000","last_fullsized":"2024-09-19T05:04:20.044042+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220988+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:04:20.220988+0000","last_peered":"2024-09-19T05:04:20.220988+0000","last_clean":"2024-09-19T05:04:20.220988+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:04:20.220988+0000","last_undegraded":"2024-09-19T05:04:20.220988+0000","last_fullsized":"2024-09-19T05:04:20.220988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044689+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:04:20.044689+0000","last_peered":"2024-09-19T05:04:20.044689+0000","last_clean":"2024-09-19T05:04:20.044689+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:04:20.044689+0000","last_undegraded":"2024-09-19T05:04:20.044689+0000","last_fullsized":"2024-09-19T05:04:20.044689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045469+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:04:20.045469+0000","last_peered":"2024-09-19T05:04:20.045469+0000","last_clean":"2024-09-19T05:04:20.045469+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:04:20.045469+0000","last_undegraded":"2024-09-19T05:04:20.045469+0000","last_fullsized":"2024-09-19T05:04:20.045469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038412+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:04:20.038412+0000","last_peered":"2024-09-19T05:04:20.038412+0000","last_clean":"2024-09-19T05:04:20.038412+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:04:20.038412+0000","last_undegraded":"2024-09-19T05:04:20.038412+0000","last_fullsized":"2024-09-19T05:04:20.038412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041819+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:04:20.041819+0000","last_peered":"2024-09-19T05:04:20.041819+0000","last_clean":"2024-09-19T05:04:20.041819+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:04:20.041819+0000","last_undegraded":"2024-09-19T05:04:20.041819+0000","last_fullsized":"2024-09-19T05:04:20.041819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040558+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:04:20.040558+0000","last_peered":"2024-09-19T05:04:20.040558+0000","last_clean":"2024-09-19T05:04:20.040558+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:04:20.040558+0000","last_undegraded":"2024-09-19T05:04:20.040558+0000","last_fullsized":"2024-09-19T05:04:20.040558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041588+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:04:20.041588+0000","last_peered":"2024-09-19T05:04:20.041588+0000","last_clean":"2024-09-19T05:04:20.041588+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:04:20.041588+0000","last_undegraded":"2024-09-19T05:04:20.041588+0000","last_fullsized":"2024-09-19T05:04:20.041588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044395+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:04:20.044395+0000","last_peered":"2024-09-19T05:04:20.044395+0000","last_clean":"2024-09-19T05:04:20.044395+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:04:20.044395+0000","last_undegraded":"2024-09-19T05:04:20.044395+0000","last_fullsized":"2024-09-19T05:04:20.044395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041954+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:04:20.041954+0000","last_peered":"2024-09-19T05:04:20.041954+0000","last_clean":"2024-09-19T05:04:20.041954+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:04:20.041954+0000","last_undegraded":"2024-09-19T05:04:20.041954+0000","last_fullsized":"2024-09-19T05:04:20.041954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":343,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042103+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:04:20.042103+0000","last_peered":"2024-09-19T05:04:20.042103+0000","last_clean":"2024-09-19T05:04:20.042103+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:04:20.042103+0000","last_undegraded":"2024-09-19T05:04:20.042103+0000","last_fullsized":"2024-09-19T05:04:20.042103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218980+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:04:20.218980+0000","last_peered":"2024-09-19T05:04:20.218980+0000","last_clean":"2024-09-19T05:04:20.218980+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:04:20.218980+0000","last_undegraded":"2024-09-19T05:04:20.218980+0000","last_fullsized":"2024-09-19T05:04:20.218980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046490+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:04:20.046490+0000","last_peered":"2024-09-19T05:04:20.046490+0000","last_clean":"2024-09-19T05:04:20.046490+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:04:20.046490+0000","last_undegraded":"2024-09-19T05:04:20.046490+0000","last_fullsized":"2024-09-19T05:04:20.046490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.048907+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:04:20.048907+0000","last_peered":"2024-09-19T05:04:20.048907+0000","last_clean":"2024-09-19T05:04:20.048907+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:04:20.048907+0000","last_undegraded":"2024-09-19T05:04:20.048907+0000","last_fullsized":"2024-09-19T05:04:20.048907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037984+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:04:20.037984+0000","last_peered":"2024-09-19T05:04:20.037984+0000","last_clean":"2024-09-19T05:04:20.037984+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:04:20.037984+0000","last_undegraded":"2024-09-19T05:04:20.037984+0000","last_fullsized":"2024-09-19T05:04:20.037984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:09.106659+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:05:09.106659+0000","last_peered":"2024-09-19T05:05:09.106659+0000","last_clean":"2024-09-19T05:05:09.106659+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:05:09.106659+0000","last_undegraded":"2024-09-19T05:05:09.106659+0000","last_fullsized":"2024-09-19T05:05:09.106659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040362+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:04:20.040362+0000","last_peered":"2024-09-19T05:04:20.040362+0000","last_clean":"2024-09-19T05:04:20.040362+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:04:20.040362+0000","last_undegraded":"2024-09-19T05:04:20.040362+0000","last_fullsized":"2024-09-19T05:04:20.040362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041750+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:04:20.041750+0000","last_peered":"2024-09-19T05:04:20.041750+0000","last_clean":"2024-09-19T05:04:20.041750+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:04:20.041750+0000","last_undegraded":"2024-09-19T05:04:20.041750+0000","last_fullsized":"2024-09-19T05:04:20.041750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043344+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:04:20.043344+0000","last_peered":"2024-09-19T05:04:20.043344+0000","last_clean":"2024-09-19T05:04:20.043344+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:04:20.043344+0000","last_undegraded":"2024-09-19T05:04:20.043344+0000","last_fullsized":"2024-09-19T05:04:20.043344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040735+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:04:20.040735+0000","last_peered":"2024-09-19T05:04:20.040735+0000","last_clean":"2024-09-19T05:04:20.040735+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:04:20.040735+0000","last_undegraded":"2024-09-19T05:04:20.040735+0000","last_fullsized":"2024-09-19T05:04:20.040735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043291+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:04:20.043291+0000","last_peered":"2024-09-19T05:04:20.043291+0000","last_clean":"2024-09-19T05:04:20.043291+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:04:20.043291+0000","last_undegraded":"2024-09-19T05:04:20.043291+0000","last_fullsized":"2024-09-19T05:04:20.043291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044730+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:04:20.044730+0000","last_peered":"2024-09-19T05:04:20.044730+0000","last_clean":"2024-09-19T05:04:20.044730+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:04:20.044730+0000","last_undegraded":"2024-09-19T05:04:20.044730+0000","last_fullsized":"2024-09-19T05:04:20.044730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038415+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:04:20.038415+0000","last_peered":"2024-09-19T05:04:20.038415+0000","last_clean":"2024-09-19T05:04:20.038415+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:04:20.038415+0000","last_undegraded":"2024-09-19T05:04:20.038415+0000","last_fullsized":"2024-09-19T05:04:20.038415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:31.976954+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:04:31.976954+0000","last_peered":"2024-09-19T05:04:31.976954+0000","last_clean":"2024-09-19T05:04:31.976954+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:04:31.976954+0000","last_undegraded":"2024-09-19T05:04:31.976954+0000","last_fullsized":"2024-09-19T05:04:31.976954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046734+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:04:20.046734+0000","last_peered":"2024-09-19T05:04:20.046734+0000","last_clean":"2024-09-19T05:04:20.046734+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:04:20.046734+0000","last_undegraded":"2024-09-19T05:04:20.046734+0000","last_fullsized":"2024-09-19T05:04:20.046734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219379+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:04:20.219379+0000","last_peered":"2024-09-19T05:04:20.219379+0000","last_clean":"2024-09-19T05:04:20.219379+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:04:20.219379+0000","last_undegraded":"2024-09-19T05:04:20.219379+0000","last_fullsized":"2024-09-19T05:04:20.219379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.047866+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:04:20.047866+0000","last_peered":"2024-09-19T05:04:20.047866+0000","last_clean":"2024-09-19T05:04:20.047866+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:04:20.047866+0000","last_undegraded":"2024-09-19T05:04:20.047866+0000","last_fullsized":"2024-09-19T05:04:20.047866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039180+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:04:20.039180+0000","last_peered":"2024-09-19T05:04:20.039180+0000","last_clean":"2024-09-19T05:04:20.039180+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:04:20.039180+0000","last_undegraded":"2024-09-19T05:04:20.039180+0000","last_fullsized":"2024-09-19T05:04:20.039180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043811+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:04:20.043811+0000","last_peered":"2024-09-19T05:04:20.043811+0000","last_clean":"2024-09-19T05:04:20.043811+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:04:20.043811+0000","last_undegraded":"2024-09-19T05:04:20.043811+0000","last_fullsized":"2024-09-19T05:04:20.043811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046197+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:04:20.046197+0000","last_peered":"2024-09-19T05:04:20.046197+0000","last_clean":"2024-09-19T05:04:20.046197+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:04:20.046197+0000","last_undegraded":"2024-09-19T05:04:20.046197+0000","last_fullsized":"2024-09-19T05:04:20.046197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045294+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:04:20.045294+0000","last_peered":"2024-09-19T05:04:20.045294+0000","last_clean":"2024-09-19T05:04:20.045294+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:04:20.045294+0000","last_undegraded":"2024-09-19T05:04:20.045294+0000","last_fullsized":"2024-09-19T05:04:20.045294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044905+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:04:20.044905+0000","last_peered":"2024-09-19T05:04:20.044905+0000","last_clean":"2024-09-19T05:04:20.044905+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:04:20.044905+0000","last_undegraded":"2024-09-19T05:04:20.044905+0000","last_fullsized":"2024-09-19T05:04:20.044905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046267+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:04:20.046267+0000","last_peered":"2024-09-19T05:04:20.046267+0000","last_clean":"2024-09-19T05:04:20.046267+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:04:20.046267+0000","last_undegraded":"2024-09-19T05:04:20.046267+0000","last_fullsized":"2024-09-19T05:04:20.046267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044834+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:04:20.044834+0000","last_peered":"2024-09-19T05:04:20.044834+0000","last_clean":"2024-09-19T05:04:20.044834+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:04:20.044834+0000","last_undegraded":"2024-09-19T05:04:20.044834+0000","last_fullsized":"2024-09-19T05:04:20.044834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041613+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:04:20.041613+0000","last_peered":"2024-09-19T05:04:20.041613+0000","last_clean":"2024-09-19T05:04:20.041613+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:04:20.041613+0000","last_undegraded":"2024-09-19T05:04:20.041613+0000","last_fullsized":"2024-09-19T05:04:20.041613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040445+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:04:20.040445+0000","last_peered":"2024-09-19T05:04:20.040445+0000","last_clean":"2024-09-19T05:04:20.040445+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:04:20.040445+0000","last_undegraded":"2024-09-19T05:04:20.040445+0000","last_fullsized":"2024-09-19T05:04:20.040445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044233+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:04:20.044233+0000","last_peered":"2024-09-19T05:04:20.044233+0000","last_clean":"2024-09-19T05:04:20.044233+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:04:20.044233+0000","last_undegraded":"2024-09-19T05:04:20.044233+0000","last_fullsized":"2024-09-19T05:04:20.044233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039172+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:04:20.039172+0000","last_peered":"2024-09-19T05:04:20.039172+0000","last_clean":"2024-09-19T05:04:20.039172+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:04:20.039172+0000","last_undegraded":"2024-09-19T05:04:20.039172+0000","last_fullsized":"2024-09-19T05:04:20.039172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.052380+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:04:20.052380+0000","last_peered":"2024-09-19T05:04:20.052380+0000","last_clean":"2024-09-19T05:04:20.052380+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:04:20.052380+0000","last_undegraded":"2024-09-19T05:04:20.052380+0000","last_fullsized":"2024-09-19T05:04:20.052380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043121+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:04:20.043121+0000","last_peered":"2024-09-19T05:04:20.043121+0000","last_clean":"2024-09-19T05:04:20.043121+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:04:20.043121+0000","last_undegraded":"2024-09-19T05:04:20.043121+0000","last_fullsized":"2024-09-19T05:04:20.043121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046887+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:04:20.046887+0000","last_peered":"2024-09-19T05:04:20.046887+0000","last_clean":"2024-09-19T05:04:20.046887+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:04:20.046887+0000","last_undegraded":"2024-09-19T05:04:20.046887+0000","last_fullsized":"2024-09-19T05:04:20.046887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046641+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:04:20.046641+0000","last_peered":"2024-09-19T05:04:20.046641+0000","last_clean":"2024-09-19T05:04:20.046641+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:04:20.046641+0000","last_undegraded":"2024-09-19T05:04:20.046641+0000","last_fullsized":"2024-09-19T05:04:20.046641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041192+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:04:20.041192+0000","last_peered":"2024-09-19T05:04:20.041192+0000","last_clean":"2024-09-19T05:04:20.041192+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:04:20.041192+0000","last_undegraded":"2024-09-19T05:04:20.041192+0000","last_fullsized":"2024-09-19T05:04:20.041192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":336,"num_read_kb":238,"num_write":164,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678042,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":422800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1568768,"data_stored":1076557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":423002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":422922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528365,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1014854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32640,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712512,"statfs":{"total":95995035648,"available":95961612288,"internally_reserved":0,"allocated":1490944,"data_stored":1014538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":422828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":422702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":486203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":484494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":423662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:05:13.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:05:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 -- 172.21.15.64:0/991551582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101610 msgr2=0x7f65ec1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 --2- 172.21.15.64:0/991551582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101610 0x7f65ec1019f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009ca0 tx=0x7f65dc02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 -- 172.21.15.64:0/991551582 shutdown_connections 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 --2- 172.21.15.64:0/991551582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 0x7f65ec10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 --2- 172.21.15.64:0/991551582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ec101fc0 0x7f65ec105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 --2- 172.21.15.64:0/991551582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101610 0x7f65ec1019f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 -- 172.21.15.64:0/991551582 >> 172.21.15.64:0/991551582 conn(0x7f65ec0fc5c0 msgr2=0x7f65ec0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 -- 172.21.15.64:0/991551582 shutdown_connections 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.883+0000 7f65f37a7640 1 -- 172.21.15.64:0/991551582 wait complete. 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 Processor -- start 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 -- start start 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ec101610 0x7f65ec199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 0x7f65ec199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 0x7f65ec1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110d90 con 0x7f65ec101fc0 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110c10 con 0x7f65ec101610 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f37a7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110f10 con 0x7f65ec105e40 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 0x7f65ec199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 0x7f65ec199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44092/0 (socket says 172.21.15.64:44092) 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 -- 172.21.15.64:0/24976103 learned_addr learned my addr 172.21.15.64:0/24976103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f1543640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ec101610 0x7f65ec199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f1d44640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 0x7f65ec1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 -- 172.21.15.64:0/24976103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 msgr2=0x7f65ec1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 0x7f65ec1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 -- 172.21.15.64:0/24976103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ec101610 msgr2=0x7f65ec199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:13.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ec101610 0x7f65ec199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f0d42640 1 -- 172.21.15.64:0/24976103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec1a47c0 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.884+0000 7f65f1d44640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 0x7f65ec1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.885+0000 7f65f0d42640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 0x7f65ec199740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005fd0 tx=0x7f65e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.885+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0017070 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.885+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65e0009c30 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.885+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec1a4990 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.885+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0003d40 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.885+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a4ed0 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.886+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec1019f0 con 0x7f65ec101fc0 2024-09-19T05:05:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.887+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f65e0003ee0 con 0x7f65ec101fc0 2024-09-19T05:05:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.888+0000 7f65ea7fc640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f65d40a3350 0x7f65d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.888+0000 7f65f1543640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f65d40a3350 0x7f65d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.888+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f65e00cfe70 con 0x7f65ec101fc0 2024-09-19T05:05:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.888+0000 7f65f1543640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f65d40a3350 0x7f65d40a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005e30 tx=0x7f65dc031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:13.891+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f65e0020030 con 0x7f65ec101fc0 2024-09-19T05:05:14.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.064+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65ec062630 con 0x7f65d40a3350 2024-09-19T05:05:14.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.070+0000 7f65ea7fc640 1 -- 172.21.15.64:0/24976103 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258718 (secure 0 0 0) 0x7f65ec062630 con 0x7f65d40a3350 2024-09-19T05:05:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:14.076 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f65d40a3350 msgr2=0x7f65d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f65d40a3350 0x7f65d40a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005e30 tx=0x7f65dc031000 comp rx=0 tx=0).stop 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 msgr2=0x7f65ec199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 0x7f65ec199740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005fd0 tx=0x7f65e000dde0 comp rx=0 tx=0).stop 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 shutdown_connections 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f65d40a3350 0x7f65d40a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec105e40 0x7f65ec1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65ec101fc0 0x7f65ec199740 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 --2- 172.21.15.64:0/24976103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ec101610 0x7f65ec199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:14.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 >> 172.21.15.64:0/24976103 conn(0x7f65ec0fc5c0 msgr2=0x7f65ec1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 shutdown_connections 2024-09-19T05:05:14.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:14.074+0000 7f65f37a7640 1 -- 172.21.15.64:0/24976103 wait complete. 2024-09-19T05:05:14.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:14.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:14.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:14.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:14.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:14 smithi064 ceph-mon[30960]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:14 smithi174 ceph-mon[34488]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:14.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:14.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:14.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:14.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:14 smithi103 ceph-mon[34446]: from='mgr.14150 172.21.15.64:0/119184990' entity='mgr.x' 2024-09-19T05:05:14.736 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T05:05:12.395195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":472,"num_read_kb":357,"num_write":347,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365736,"kb_used_data":12312,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576088,"statfs":{"total":1151940427776,"available":1151565914112,"internally_reserved":0,"allocated":12607488,"data_stored":7036953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045762+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:04:20.045762+0000","last_peered":"2024-09-19T05:04:20.045762+0000","last_clean":"2024-09-19T05:04:20.045762+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:04:20.045762+0000","last_undegraded":"2024-09-19T05:04:20.045762+0000","last_fullsized":"2024-09-19T05:04:20.045762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042748+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:04:20.042748+0000","last_peered":"2024-09-19T05:04:20.042748+0000","last_clean":"2024-09-19T05:04:20.042748+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:04:20.042748+0000","last_undegraded":"2024-09-19T05:04:20.042748+0000","last_fullsized":"2024-09-19T05:04:20.042748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042862+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:04:20.042862+0000","last_peered":"2024-09-19T05:04:20.042862+0000","last_clean":"2024-09-19T05:04:20.042862+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:04:20.042862+0000","last_undegraded":"2024-09-19T05:04:20.042862+0000","last_fullsized":"2024-09-19T05:04:20.042862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218922+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:04:20.218922+0000","last_peered":"2024-09-19T05:04:20.218922+0000","last_clean":"2024-09-19T05:04:20.218922+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:04:20.218922+0000","last_undegraded":"2024-09-19T05:04:20.218922+0000","last_fullsized":"2024-09-19T05:04:20.218922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044933+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:04:20.044933+0000","last_peered":"2024-09-19T05:04:20.044933+0000","last_clean":"2024-09-19T05:04:20.044933+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:04:20.044933+0000","last_undegraded":"2024-09-19T05:04:20.044933+0000","last_fullsized":"2024-09-19T05:04:20.044933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046644+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:04:20.046644+0000","last_peered":"2024-09-19T05:04:20.046644+0000","last_clean":"2024-09-19T05:04:20.046644+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:04:20.046644+0000","last_undegraded":"2024-09-19T05:04:20.046644+0000","last_fullsized":"2024-09-19T05:04:20.046644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041281+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:04:20.041281+0000","last_peered":"2024-09-19T05:04:20.041281+0000","last_clean":"2024-09-19T05:04:20.041281+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:04:20.041281+0000","last_undegraded":"2024-09-19T05:04:20.041281+0000","last_fullsized":"2024-09-19T05:04:20.041281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045982+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:04:20.045982+0000","last_peered":"2024-09-19T05:04:20.045982+0000","last_clean":"2024-09-19T05:04:20.045982+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:04:20.045982+0000","last_undegraded":"2024-09-19T05:04:20.045982+0000","last_fullsized":"2024-09-19T05:04:20.045982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039683+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:04:20.039683+0000","last_peered":"2024-09-19T05:04:20.039683+0000","last_clean":"2024-09-19T05:04:20.039683+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:04:20.039683+0000","last_undegraded":"2024-09-19T05:04:20.039683+0000","last_fullsized":"2024-09-19T05:04:20.039683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043775+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:04:20.043775+0000","last_peered":"2024-09-19T05:04:20.043775+0000","last_clean":"2024-09-19T05:04:20.043775+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:04:20.043775+0000","last_undegraded":"2024-09-19T05:04:20.043775+0000","last_fullsized":"2024-09-19T05:04:20.043775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041842+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:04:20.041842+0000","last_peered":"2024-09-19T05:04:20.041842+0000","last_clean":"2024-09-19T05:04:20.041842+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:04:20.041842+0000","last_undegraded":"2024-09-19T05:04:20.041842+0000","last_fullsized":"2024-09-19T05:04:20.041842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219054+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:04:20.219054+0000","last_peered":"2024-09-19T05:04:20.219054+0000","last_clean":"2024-09-19T05:04:20.219054+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:04:20.219054+0000","last_undegraded":"2024-09-19T05:04:20.219054+0000","last_fullsized":"2024-09-19T05:04:20.219054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038234+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:04:20.038234+0000","last_peered":"2024-09-19T05:04:20.038234+0000","last_clean":"2024-09-19T05:04:20.038234+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:04:20.038234+0000","last_undegraded":"2024-09-19T05:04:20.038234+0000","last_fullsized":"2024-09-19T05:04:20.038234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042859+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:04:20.042859+0000","last_peered":"2024-09-19T05:04:20.042859+0000","last_clean":"2024-09-19T05:04:20.042859+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:04:20.042859+0000","last_undegraded":"2024-09-19T05:04:20.042859+0000","last_fullsized":"2024-09-19T05:04:20.042859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042616+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:04:20.042616+0000","last_peered":"2024-09-19T05:04:20.042616+0000","last_clean":"2024-09-19T05:04:20.042616+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:04:20.042616+0000","last_undegraded":"2024-09-19T05:04:20.042616+0000","last_fullsized":"2024-09-19T05:04:20.042616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039034+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:04:20.039034+0000","last_peered":"2024-09-19T05:04:20.039034+0000","last_clean":"2024-09-19T05:04:20.039034+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:04:20.039034+0000","last_undegraded":"2024-09-19T05:04:20.039034+0000","last_fullsized":"2024-09-19T05:04:20.039034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045863+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:04:20.045863+0000","last_peered":"2024-09-19T05:04:20.045863+0000","last_clean":"2024-09-19T05:04:20.045863+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:04:20.045863+0000","last_undegraded":"2024-09-19T05:04:20.045863+0000","last_fullsized":"2024-09-19T05:04:20.045863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040764+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:04:20.040764+0000","last_peered":"2024-09-19T05:04:20.040764+0000","last_clean":"2024-09-19T05:04:20.040764+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:04:20.040764+0000","last_undegraded":"2024-09-19T05:04:20.040764+0000","last_fullsized":"2024-09-19T05:04:20.040764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045627+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:04:20.045627+0000","last_peered":"2024-09-19T05:04:20.045627+0000","last_clean":"2024-09-19T05:04:20.045627+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:04:20.045627+0000","last_undegraded":"2024-09-19T05:04:20.045627+0000","last_fullsized":"2024-09-19T05:04:20.045627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045190+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:04:20.045190+0000","last_peered":"2024-09-19T05:04:20.045190+0000","last_clean":"2024-09-19T05:04:20.045190+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:04:20.045190+0000","last_undegraded":"2024-09-19T05:04:20.045190+0000","last_fullsized":"2024-09-19T05:04:20.045190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038280+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:04:20.038280+0000","last_peered":"2024-09-19T05:04:20.038280+0000","last_clean":"2024-09-19T05:04:20.038280+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:04:20.038280+0000","last_undegraded":"2024-09-19T05:04:20.038280+0000","last_fullsized":"2024-09-19T05:04:20.038280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039404+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:04:20.039404+0000","last_peered":"2024-09-19T05:04:20.039404+0000","last_clean":"2024-09-19T05:04:20.039404+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:04:20.039404+0000","last_undegraded":"2024-09-19T05:04:20.039404+0000","last_fullsized":"2024-09-19T05:04:20.039404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046472+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:04:20.046472+0000","last_peered":"2024-09-19T05:04:20.046472+0000","last_clean":"2024-09-19T05:04:20.046472+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:04:20.046472+0000","last_undegraded":"2024-09-19T05:04:20.046472+0000","last_fullsized":"2024-09-19T05:04:20.046472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040224+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:04:20.040224+0000","last_peered":"2024-09-19T05:04:20.040224+0000","last_clean":"2024-09-19T05:04:20.040224+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:04:20.040224+0000","last_undegraded":"2024-09-19T05:04:20.040224+0000","last_fullsized":"2024-09-19T05:04:20.040224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220980+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:04:20.220980+0000","last_peered":"2024-09-19T05:04:20.220980+0000","last_clean":"2024-09-19T05:04:20.220980+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:04:20.220980+0000","last_undegraded":"2024-09-19T05:04:20.220980+0000","last_fullsized":"2024-09-19T05:04:20.220980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044307+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:04:20.044307+0000","last_peered":"2024-09-19T05:04:20.044307+0000","last_clean":"2024-09-19T05:04:20.044307+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:04:20.044307+0000","last_undegraded":"2024-09-19T05:04:20.044307+0000","last_fullsized":"2024-09-19T05:04:20.044307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046219+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:04:20.046219+0000","last_peered":"2024-09-19T05:04:20.046219+0000","last_clean":"2024-09-19T05:04:20.046219+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:04:20.046219+0000","last_undegraded":"2024-09-19T05:04:20.046219+0000","last_fullsized":"2024-09-19T05:04:20.046219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038227+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:04:20.038227+0000","last_peered":"2024-09-19T05:04:20.038227+0000","last_clean":"2024-09-19T05:04:20.038227+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:04:20.038227+0000","last_undegraded":"2024-09-19T05:04:20.038227+0000","last_fullsized":"2024-09-19T05:04:20.038227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042890+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:04:20.042890+0000","last_peered":"2024-09-19T05:04:20.042890+0000","last_clean":"2024-09-19T05:04:20.042890+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:04:20.042890+0000","last_undegraded":"2024-09-19T05:04:20.042890+0000","last_fullsized":"2024-09-19T05:04:20.042890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044794+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:04:20.044794+0000","last_peered":"2024-09-19T05:04:20.044794+0000","last_clean":"2024-09-19T05:04:20.044794+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:04:20.044794+0000","last_undegraded":"2024-09-19T05:04:20.044794+0000","last_fullsized":"2024-09-19T05:04:20.044794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044665+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:04:20.044665+0000","last_peered":"2024-09-19T05:04:20.044665+0000","last_clean":"2024-09-19T05:04:20.044665+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:04:20.044665+0000","last_undegraded":"2024-09-19T05:04:20.044665+0000","last_fullsized":"2024-09-19T05:04:20.044665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045176+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:04:20.045176+0000","last_peered":"2024-09-19T05:04:20.045176+0000","last_clean":"2024-09-19T05:04:20.045176+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:04:20.045176+0000","last_undegraded":"2024-09-19T05:04:20.045176+0000","last_fullsized":"2024-09-19T05:04:20.045176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045192+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:04:20.045192+0000","last_peered":"2024-09-19T05:04:20.045192+0000","last_clean":"2024-09-19T05:04:20.045192+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:04:20.045192+0000","last_undegraded":"2024-09-19T05:04:20.045192+0000","last_fullsized":"2024-09-19T05:04:20.045192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":82,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:07.091041+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:05:07.091041+0000","last_peered":"2024-09-19T05:05:07.091041+0000","last_clean":"2024-09-19T05:05:07.091041+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:05:07.091041+0000","last_undegraded":"2024-09-19T05:05:07.091041+0000","last_fullsized":"2024-09-19T05:05:07.091041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220294+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:04:20.220294+0000","last_peered":"2024-09-19T05:04:20.220294+0000","last_clean":"2024-09-19T05:04:20.220294+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:04:20.220294+0000","last_undegraded":"2024-09-19T05:04:20.220294+0000","last_fullsized":"2024-09-19T05:04:20.220294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043292+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:04:20.043292+0000","last_peered":"2024-09-19T05:04:20.043292+0000","last_clean":"2024-09-19T05:04:20.043292+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:04:20.043292+0000","last_undegraded":"2024-09-19T05:04:20.043292+0000","last_fullsized":"2024-09-19T05:04:20.043292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037995+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:04:20.037995+0000","last_peered":"2024-09-19T05:04:20.037995+0000","last_clean":"2024-09-19T05:04:20.037995+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:04:20.037995+0000","last_undegraded":"2024-09-19T05:04:20.037995+0000","last_fullsized":"2024-09-19T05:04:20.037995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046054+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:04:20.046054+0000","last_peered":"2024-09-19T05:04:20.046054+0000","last_clean":"2024-09-19T05:04:20.046054+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:04:20.046054+0000","last_undegraded":"2024-09-19T05:04:20.046054+0000","last_fullsized":"2024-09-19T05:04:20.046054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040479+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:04:20.040479+0000","last_peered":"2024-09-19T05:04:20.040479+0000","last_clean":"2024-09-19T05:04:20.040479+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:04:20.040479+0000","last_undegraded":"2024-09-19T05:04:20.040479+0000","last_fullsized":"2024-09-19T05:04:20.040479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041190+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:04:20.041190+0000","last_peered":"2024-09-19T05:04:20.041190+0000","last_clean":"2024-09-19T05:04:20.041190+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:04:20.041190+0000","last_undegraded":"2024-09-19T05:04:20.041190+0000","last_fullsized":"2024-09-19T05:04:20.041190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'3","reported_seq":76,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:07.090319+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:05:07.090319+0000","last_peered":"2024-09-19T05:05:07.090319+0000","last_clean":"2024-09-19T05:05:07.090319+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:05:07.090319+0000","last_undegraded":"2024-09-19T05:05:07.090319+0000","last_fullsized":"2024-09-19T05:05:07.090319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039611+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:04:20.039611+0000","last_peered":"2024-09-19T05:04:20.039611+0000","last_clean":"2024-09-19T05:04:20.039611+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:04:20.039611+0000","last_undegraded":"2024-09-19T05:04:20.039611+0000","last_fullsized":"2024-09-19T05:04:20.039611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.223302+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:04:20.223302+0000","last_peered":"2024-09-19T05:04:20.223302+0000","last_clean":"2024-09-19T05:04:20.223302+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:04:20.223302+0000","last_undegraded":"2024-09-19T05:04:20.223302+0000","last_fullsized":"2024-09-19T05:04:20.223302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044799+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:04:20.044799+0000","last_peered":"2024-09-19T05:04:20.044799+0000","last_clean":"2024-09-19T05:04:20.044799+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:04:20.044799+0000","last_undegraded":"2024-09-19T05:04:20.044799+0000","last_fullsized":"2024-09-19T05:04:20.044799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040554+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:04:20.040554+0000","last_peered":"2024-09-19T05:04:20.040554+0000","last_clean":"2024-09-19T05:04:20.040554+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:04:20.040554+0000","last_undegraded":"2024-09-19T05:04:20.040554+0000","last_fullsized":"2024-09-19T05:04:20.040554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039165+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:04:20.039165+0000","last_peered":"2024-09-19T05:04:20.039165+0000","last_clean":"2024-09-19T05:04:20.039165+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:04:20.039165+0000","last_undegraded":"2024-09-19T05:04:20.039165+0000","last_fullsized":"2024-09-19T05:04:20.039165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":164,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:08.098498+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:05:08.098498+0000","last_peered":"2024-09-19T05:05:08.098498+0000","last_clean":"2024-09-19T05:05:08.098498+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:05:08.098498+0000","last_undegraded":"2024-09-19T05:05:08.098498+0000","last_fullsized":"2024-09-19T05:05:08.098498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":116,"num_write":32,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044465+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:04:20.044465+0000","last_peered":"2024-09-19T05:04:20.044465+0000","last_clean":"2024-09-19T05:04:20.044465+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:04:20.044465+0000","last_undegraded":"2024-09-19T05:04:20.044465+0000","last_fullsized":"2024-09-19T05:04:20.044465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.058790+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:04:20.058790+0000","last_peered":"2024-09-19T05:04:20.058790+0000","last_clean":"2024-09-19T05:04:20.058790+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:04:20.058790+0000","last_undegraded":"2024-09-19T05:04:20.058790+0000","last_fullsized":"2024-09-19T05:04:20.058790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038290+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:04:20.038290+0000","last_peered":"2024-09-19T05:04:20.038290+0000","last_clean":"2024-09-19T05:04:20.038290+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:04:20.038290+0000","last_undegraded":"2024-09-19T05:04:20.038290+0000","last_fullsized":"2024-09-19T05:04:20.038290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040388+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:04:20.040388+0000","last_peered":"2024-09-19T05:04:20.040388+0000","last_clean":"2024-09-19T05:04:20.040388+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:04:20.040388+0000","last_undegraded":"2024-09-19T05:04:20.040388+0000","last_fullsized":"2024-09-19T05:04:20.040388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041839+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:04:20.041839+0000","last_peered":"2024-09-19T05:04:20.041839+0000","last_clean":"2024-09-19T05:04:20.041839+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:04:20.041839+0000","last_undegraded":"2024-09-19T05:04:20.041839+0000","last_fullsized":"2024-09-19T05:04:20.041839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'3","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:08.105699+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:05:08.105699+0000","last_peered":"2024-09-19T05:05:08.105699+0000","last_clean":"2024-09-19T05:05:08.105699+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:05:08.105699+0000","last_undegraded":"2024-09-19T05:05:08.105699+0000","last_fullsized":"2024-09-19T05:05:08.105699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041986+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:04:20.041986+0000","last_peered":"2024-09-19T05:04:20.041986+0000","last_clean":"2024-09-19T05:04:20.041986+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:04:20.041986+0000","last_undegraded":"2024-09-19T05:04:20.041986+0000","last_fullsized":"2024-09-19T05:04:20.041986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044042+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:04:20.044042+0000","last_peered":"2024-09-19T05:04:20.044042+0000","last_clean":"2024-09-19T05:04:20.044042+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:04:20.044042+0000","last_undegraded":"2024-09-19T05:04:20.044042+0000","last_fullsized":"2024-09-19T05:04:20.044042+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.220988+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:04:20.220988+0000","last_peered":"2024-09-19T05:04:20.220988+0000","last_clean":"2024-09-19T05:04:20.220988+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:04:20.220988+0000","last_undegraded":"2024-09-19T05:04:20.220988+0000","last_fullsized":"2024-09-19T05:04:20.220988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044689+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:04:20.044689+0000","last_peered":"2024-09-19T05:04:20.044689+0000","last_clean":"2024-09-19T05:04:20.044689+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:04:20.044689+0000","last_undegraded":"2024-09-19T05:04:20.044689+0000","last_fullsized":"2024-09-19T05:04:20.044689+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045469+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:04:20.045469+0000","last_peered":"2024-09-19T05:04:20.045469+0000","last_clean":"2024-09-19T05:04:20.045469+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:04:20.045469+0000","last_undegraded":"2024-09-19T05:04:20.045469+0000","last_fullsized":"2024-09-19T05:04:20.045469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038412+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:04:20.038412+0000","last_peered":"2024-09-19T05:04:20.038412+0000","last_clean":"2024-09-19T05:04:20.038412+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:04:20.038412+0000","last_undegraded":"2024-09-19T05:04:20.038412+0000","last_fullsized":"2024-09-19T05:04:20.038412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041819+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:04:20.041819+0000","last_peered":"2024-09-19T05:04:20.041819+0000","last_clean":"2024-09-19T05:04:20.041819+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:04:20.041819+0000","last_undegraded":"2024-09-19T05:04:20.041819+0000","last_fullsized":"2024-09-19T05:04:20.041819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040558+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:04:20.040558+0000","last_peered":"2024-09-19T05:04:20.040558+0000","last_clean":"2024-09-19T05:04:20.040558+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:04:20.040558+0000","last_undegraded":"2024-09-19T05:04:20.040558+0000","last_fullsized":"2024-09-19T05:04:20.040558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041588+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:04:20.041588+0000","last_peered":"2024-09-19T05:04:20.041588+0000","last_clean":"2024-09-19T05:04:20.041588+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:04:20.041588+0000","last_undegraded":"2024-09-19T05:04:20.041588+0000","last_fullsized":"2024-09-19T05:04:20.041588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044395+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:04:20.044395+0000","last_peered":"2024-09-19T05:04:20.044395+0000","last_clean":"2024-09-19T05:04:20.044395+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:04:20.044395+0000","last_undegraded":"2024-09-19T05:04:20.044395+0000","last_fullsized":"2024-09-19T05:04:20.044395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041954+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:04:20.041954+0000","last_peered":"2024-09-19T05:04:20.041954+0000","last_clean":"2024-09-19T05:04:20.041954+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:04:20.041954+0000","last_undegraded":"2024-09-19T05:04:20.041954+0000","last_fullsized":"2024-09-19T05:04:20.041954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":343,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.042103+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:04:20.042103+0000","last_peered":"2024-09-19T05:04:20.042103+0000","last_clean":"2024-09-19T05:04:20.042103+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:04:20.042103+0000","last_undegraded":"2024-09-19T05:04:20.042103+0000","last_fullsized":"2024-09-19T05:04:20.042103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.218980+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:04:20.218980+0000","last_peered":"2024-09-19T05:04:20.218980+0000","last_clean":"2024-09-19T05:04:20.218980+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:04:20.218980+0000","last_undegraded":"2024-09-19T05:04:20.218980+0000","last_fullsized":"2024-09-19T05:04:20.218980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046490+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:04:20.046490+0000","last_peered":"2024-09-19T05:04:20.046490+0000","last_clean":"2024-09-19T05:04:20.046490+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:04:20.046490+0000","last_undegraded":"2024-09-19T05:04:20.046490+0000","last_fullsized":"2024-09-19T05:04:20.046490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.048907+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:04:20.048907+0000","last_peered":"2024-09-19T05:04:20.048907+0000","last_clean":"2024-09-19T05:04:20.048907+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:04:20.048907+0000","last_undegraded":"2024-09-19T05:04:20.048907+0000","last_fullsized":"2024-09-19T05:04:20.048907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.037984+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:04:20.037984+0000","last_peered":"2024-09-19T05:04:20.037984+0000","last_clean":"2024-09-19T05:04:20.037984+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:04:20.037984+0000","last_undegraded":"2024-09-19T05:04:20.037984+0000","last_fullsized":"2024-09-19T05:04:20.037984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'4","reported_seq":79,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:05:09.106659+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:05:09.106659+0000","last_peered":"2024-09-19T05:05:09.106659+0000","last_clean":"2024-09-19T05:05:09.106659+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:05:09.106659+0000","last_undegraded":"2024-09-19T05:05:09.106659+0000","last_fullsized":"2024-09-19T05:05:09.106659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040362+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:04:20.040362+0000","last_peered":"2024-09-19T05:04:20.040362+0000","last_clean":"2024-09-19T05:04:20.040362+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:04:20.040362+0000","last_undegraded":"2024-09-19T05:04:20.040362+0000","last_fullsized":"2024-09-19T05:04:20.040362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041750+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:04:20.041750+0000","last_peered":"2024-09-19T05:04:20.041750+0000","last_clean":"2024-09-19T05:04:20.041750+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:04:20.041750+0000","last_undegraded":"2024-09-19T05:04:20.041750+0000","last_fullsized":"2024-09-19T05:04:20.041750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043344+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:04:20.043344+0000","last_peered":"2024-09-19T05:04:20.043344+0000","last_clean":"2024-09-19T05:04:20.043344+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:04:20.043344+0000","last_undegraded":"2024-09-19T05:04:20.043344+0000","last_fullsized":"2024-09-19T05:04:20.043344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040735+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:04:20.040735+0000","last_peered":"2024-09-19T05:04:20.040735+0000","last_clean":"2024-09-19T05:04:20.040735+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:04:20.040735+0000","last_undegraded":"2024-09-19T05:04:20.040735+0000","last_fullsized":"2024-09-19T05:04:20.040735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043291+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:04:20.043291+0000","last_peered":"2024-09-19T05:04:20.043291+0000","last_clean":"2024-09-19T05:04:20.043291+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:04:20.043291+0000","last_undegraded":"2024-09-19T05:04:20.043291+0000","last_fullsized":"2024-09-19T05:04:20.043291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044730+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:04:20.044730+0000","last_peered":"2024-09-19T05:04:20.044730+0000","last_clean":"2024-09-19T05:04:20.044730+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:04:20.044730+0000","last_undegraded":"2024-09-19T05:04:20.044730+0000","last_fullsized":"2024-09-19T05:04:20.044730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.038415+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:04:20.038415+0000","last_peered":"2024-09-19T05:04:20.038415+0000","last_clean":"2024-09-19T05:04:20.038415+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:04:20.038415+0000","last_undegraded":"2024-09-19T05:04:20.038415+0000","last_fullsized":"2024-09-19T05:04:20.038415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:31.976954+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:04:31.976954+0000","last_peered":"2024-09-19T05:04:31.976954+0000","last_clean":"2024-09-19T05:04:31.976954+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:04:31.976954+0000","last_undegraded":"2024-09-19T05:04:31.976954+0000","last_fullsized":"2024-09-19T05:04:31.976954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046734+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:04:20.046734+0000","last_peered":"2024-09-19T05:04:20.046734+0000","last_clean":"2024-09-19T05:04:20.046734+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:04:20.046734+0000","last_undegraded":"2024-09-19T05:04:20.046734+0000","last_fullsized":"2024-09-19T05:04:20.046734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.219379+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:04:20.219379+0000","last_peered":"2024-09-19T05:04:20.219379+0000","last_clean":"2024-09-19T05:04:20.219379+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:04:20.219379+0000","last_undegraded":"2024-09-19T05:04:20.219379+0000","last_fullsized":"2024-09-19T05:04:20.219379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.047866+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:04:20.047866+0000","last_peered":"2024-09-19T05:04:20.047866+0000","last_clean":"2024-09-19T05:04:20.047866+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:04:20.047866+0000","last_undegraded":"2024-09-19T05:04:20.047866+0000","last_fullsized":"2024-09-19T05:04:20.047866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039180+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:04:20.039180+0000","last_peered":"2024-09-19T05:04:20.039180+0000","last_clean":"2024-09-19T05:04:20.039180+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:04:20.039180+0000","last_undegraded":"2024-09-19T05:04:20.039180+0000","last_fullsized":"2024-09-19T05:04:20.039180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043811+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:04:20.043811+0000","last_peered":"2024-09-19T05:04:20.043811+0000","last_clean":"2024-09-19T05:04:20.043811+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:04:20.043811+0000","last_undegraded":"2024-09-19T05:04:20.043811+0000","last_fullsized":"2024-09-19T05:04:20.043811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046197+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:04:20.046197+0000","last_peered":"2024-09-19T05:04:20.046197+0000","last_clean":"2024-09-19T05:04:20.046197+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:04:20.046197+0000","last_undegraded":"2024-09-19T05:04:20.046197+0000","last_fullsized":"2024-09-19T05:04:20.046197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.045294+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:04:20.045294+0000","last_peered":"2024-09-19T05:04:20.045294+0000","last_clean":"2024-09-19T05:04:20.045294+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:04:20.045294+0000","last_undegraded":"2024-09-19T05:04:20.045294+0000","last_fullsized":"2024-09-19T05:04:20.045294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044905+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:04:20.044905+0000","last_peered":"2024-09-19T05:04:20.044905+0000","last_clean":"2024-09-19T05:04:20.044905+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:04:20.044905+0000","last_undegraded":"2024-09-19T05:04:20.044905+0000","last_fullsized":"2024-09-19T05:04:20.044905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046267+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:04:20.046267+0000","last_peered":"2024-09-19T05:04:20.046267+0000","last_clean":"2024-09-19T05:04:20.046267+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:04:20.046267+0000","last_undegraded":"2024-09-19T05:04:20.046267+0000","last_fullsized":"2024-09-19T05:04:20.046267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044834+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:04:20.044834+0000","last_peered":"2024-09-19T05:04:20.044834+0000","last_clean":"2024-09-19T05:04:20.044834+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:04:20.044834+0000","last_undegraded":"2024-09-19T05:04:20.044834+0000","last_fullsized":"2024-09-19T05:04:20.044834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041613+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:04:20.041613+0000","last_peered":"2024-09-19T05:04:20.041613+0000","last_clean":"2024-09-19T05:04:20.041613+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:04:20.041613+0000","last_undegraded":"2024-09-19T05:04:20.041613+0000","last_fullsized":"2024-09-19T05:04:20.041613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.040445+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:04:20.040445+0000","last_peered":"2024-09-19T05:04:20.040445+0000","last_clean":"2024-09-19T05:04:20.040445+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:04:20.040445+0000","last_undegraded":"2024-09-19T05:04:20.040445+0000","last_fullsized":"2024-09-19T05:04:20.040445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.044233+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:04:20.044233+0000","last_peered":"2024-09-19T05:04:20.044233+0000","last_clean":"2024-09-19T05:04:20.044233+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:04:20.044233+0000","last_undegraded":"2024-09-19T05:04:20.044233+0000","last_fullsized":"2024-09-19T05:04:20.044233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.039172+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:04:20.039172+0000","last_peered":"2024-09-19T05:04:20.039172+0000","last_clean":"2024-09-19T05:04:20.039172+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:04:20.039172+0000","last_undegraded":"2024-09-19T05:04:20.039172+0000","last_fullsized":"2024-09-19T05:04:20.039172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.052380+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:04:20.052380+0000","last_peered":"2024-09-19T05:04:20.052380+0000","last_clean":"2024-09-19T05:04:20.052380+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:04:20.052380+0000","last_undegraded":"2024-09-19T05:04:20.052380+0000","last_fullsized":"2024-09-19T05:04:20.052380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.043121+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:04:20.043121+0000","last_peered":"2024-09-19T05:04:20.043121+0000","last_clean":"2024-09-19T05:04:20.043121+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:04:20.043121+0000","last_undegraded":"2024-09-19T05:04:20.043121+0000","last_fullsized":"2024-09-19T05:04:20.043121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046887+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:04:20.046887+0000","last_peered":"2024-09-19T05:04:20.046887+0000","last_clean":"2024-09-19T05:04:20.046887+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:04:20.046887+0000","last_undegraded":"2024-09-19T05:04:20.046887+0000","last_fullsized":"2024-09-19T05:04:20.046887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.046641+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:04:20.046641+0000","last_peered":"2024-09-19T05:04:20.046641+0000","last_clean":"2024-09-19T05:04:20.046641+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:04:20.046641+0000","last_undegraded":"2024-09-19T05:04:20.046641+0000","last_fullsized":"2024-09-19T05:04:20.046641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T05:04:20.041192+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:04:20.041192+0000","last_peered":"2024-09-19T05:04:20.041192+0000","last_clean":"2024-09-19T05:04:20.041192+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:04:20.041192+0000","last_undegraded":"2024-09-19T05:04:20.041192+0000","last_fullsized":"2024-09-19T05:04:20.041192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":336,"num_read_kb":238,"num_write":164,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678042,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":422800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1568768,"data_stored":1076557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":423002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":422922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528365,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1014854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32640,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712512,"statfs":{"total":95995035648,"available":95961612288,"internally_reserved":0,"allocated":1490944,"data_stored":1014538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":422828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":422702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":486203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":484494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":423662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:05:14.741 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:05:14.741 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T05:05:14.741 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T05:05:14.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T05:05:15.002 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:15.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:15 smithi064 ceph-mon[30960]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.439+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/1410414121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4102d40 msgr2=0x7fb2c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:15.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.439+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/1410414121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4102d40 0x7fb2c4103120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009900 tx=0x7fb2b402a480 comp rx=0 tx=0).stop 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.439+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/1410414121 shutdown_connections 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.439+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/1410414121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c410c170 0x7fb2c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.439+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/1410414121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c4103660 0x7fb2c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.439+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/1410414121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4102d40 0x7fb2c4103120 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.440+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/1410414121 >> 172.21.15.64:0/1410414121 conn(0x7fb2c40fc7a0 msgr2=0x7fb2c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.440+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/1410414121 shutdown_connections 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.440+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/1410414121 wait complete. 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.440+0000 7fb2ca4ca640 1 Processor -- start 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.440+0000 7fb2ca4ca640 1 -- start start 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c4102d40 0x7fb2c419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:15.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 0x7fb2c419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c410c170 0x7fb2c419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4111e60 con 0x7fb2c4103660 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4111ce0 con 0x7fb2c4102d40 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4111fe0 con 0x7fb2c410c170 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 0x7fb2c419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 0x7fb2c419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41486/0 (socket says 172.21.15.64:41486) 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 -- 172.21.15.64:0/3898582487 learned_addr learned my addr 172.21.15.64:0/3898582487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c3fff640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c4102d40 0x7fb2c419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 -- 172.21.15.64:0/3898582487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c410c170 msgr2=0x7fb2c419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c410c170 0x7fb2c419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 -- 172.21.15.64:0/3898582487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c4102d40 msgr2=0x7fb2c419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c4102d40 0x7fb2c419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 -- 172.21.15.64:0/3898582487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c419f580 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c37fe640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 0x7fb2c419ab10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c940 tx=0x7fb2b800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8004220 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c419f870 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b80043c0 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8004a00 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.441+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a7150 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.442+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c4103120 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.444+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb2b80079b0 con 0x7fb2c4103660 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.444+0000 7fb2c17fa640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2ac0a32d0 0x7fb2ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:15.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.444+0000 7fb2c3fff640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2ac0a32d0 0x7fb2ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.444+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fb2b80cfd00 con 0x7fb2c4103660 2024-09-19T05:05:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.445+0000 7fb2c3fff640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2ac0a32d0 0x7fb2ac0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4031440 tx=0x7fb2b40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.447+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2b8014030 con 0x7fb2c4103660 2024-09-19T05:05:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:15 smithi174 ceph-mon[34488]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:15.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.658+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb2c419b930 con 0x7fb2c4103660 2024-09-19T05:05:15.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.659+0000 7fb2c17fa640 1 -- 172.21.15.64:0/3898582487 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb2c419b930 con 0x7fb2c4103660 2024-09-19T05:05:15.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:15.662 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T05:05:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2ac0a32d0 msgr2=0x7fb2ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2ac0a32d0 0x7fb2ac0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4031440 tx=0x7fb2b40095f0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 msgr2=0x7fb2c419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 0x7fb2c419ab10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c940 tx=0x7fb2b800ce00 comp rx=0 tx=0).stop 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 shutdown_connections 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb2ac0a32d0 0x7fb2ac0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c410c170 0x7fb2c419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2c4103660 0x7fb2c419ab10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 --2- 172.21.15.64:0/3898582487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2c4102d40 0x7fb2c419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 >> 172.21.15.64:0/3898582487 conn(0x7fb2c40fc7a0 msgr2=0x7fb2c4101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 shutdown_connections 2024-09-19T05:05:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:15.661+0000 7fb2ca4ca640 1 -- 172.21.15.64:0/3898582487 wait complete. 2024-09-19T05:05:15.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:15 smithi103 ceph-mon[34446]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:16.300 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T05:05:16.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:16.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:16 smithi174 ceph-mon[34488]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:16.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:16 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3898582487' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:05:16.560 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:16.580 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:16 smithi064 ceph-mon[30960]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:16.580 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:16 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3898582487' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:05:16.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:16 smithi103 ceph-mon[34446]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:16.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:16 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3898582487' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:05:17.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 -- 172.21.15.64:0/4034416272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0102570 msgr2=0x7f16e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:17.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/4034416272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0102570 0x7f16e01029b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009ca0 tx=0x7f16d002a950 comp rx=0 tx=0).stop 2024-09-19T05:05:17.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 -- 172.21.15.64:0/4034416272 shutdown_connections 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/4034416272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0102ef0 0x7f16e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/4034416272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0102570 0x7f16e01029b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/4034416272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100e20 0x7f16e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 -- 172.21.15.64:0/4034416272 >> 172.21.15.64:0/4034416272 conn(0x7f16e00fc7d0 msgr2=0x7f16e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 -- 172.21.15.64:0/4034416272 shutdown_connections 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.132+0000 7f16e6fa2640 1 -- 172.21.15.64:0/4034416272 wait complete. 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 Processor -- start 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 -- start start 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 0x7f16e00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0102570 0x7f16e00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0102ef0 0x7f16e00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0110e50 con 0x7f16e0100e20 2024-09-19T05:05:17.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0110cd0 con 0x7f16e0102570 2024-09-19T05:05:17.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e6fa2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0110fd0 con 0x7f16e0102ef0 2024-09-19T05:05:17.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 0x7f16e00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:17.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 0x7f16e00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41492/0 (socket says 172.21.15.64:41492) 2024-09-19T05:05:17.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 -- 172.21.15.64:0/1445414820 learned_addr learned my addr 172.21.15.64:0/1445414820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:17.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16dffff640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0102570 0x7f16e00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 -- 172.21.15.64:0/1445414820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0102ef0 msgr2=0x7f16e00fd700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0102ef0 0x7f16e00fd700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 -- 172.21.15.64:0/1445414820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0102570 msgr2=0x7f16e00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0102570 0x7f16e00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.133+0000 7f16e4d3e640 1 -- 172.21.15.64:0/1445414820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e01a6a00 con 0x7f16e0100e20 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.134+0000 7f16e4d3e640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 0x7f16e00fcc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16d800b4a0 tx=0x7f16d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.134+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8014070 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.134+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d8010ac0 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.134+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e01a6cf0 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.134+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8017c30 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.134+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01a71d0 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.135+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e0101200 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.136+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f16d8017070 con 0x7f16e0100e20 2024-09-19T05:05:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.137+0000 7f16ddffb640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f16cc0a3350 0x7f16cc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.137+0000 7f16dffff640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f16cc0a3350 0x7f16cc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.138+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f16d80cf190 con 0x7f16e0100e20 2024-09-19T05:05:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.138+0000 7f16dffff640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f16cc0a3350 0x7f16cc0a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16e00fe710 tx=0x7f16d0004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:17.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.141+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16d809bde0 con 0x7f16e0100e20 2024-09-19T05:05:17.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.354+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16e00fe1b0 con 0x7f16e0100e20 2024-09-19T05:05:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.355+0000 7f16ddffb640 1 -- 172.21.15.64:0/1445414820 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f16d809b6e0 con 0x7f16e0100e20 2024-09-19T05:05:17.357 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:17.357 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-19T05:04:20:028478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:04:19.601259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24448,24466,24479]},"id":1}]} 2024-09-19T05:05:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f16cc0a3350 msgr2=0x7f16cc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f16cc0a3350 0x7f16cc0a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16e00fe710 tx=0x7f16d0004a60 comp rx=0 tx=0).stop 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 msgr2=0x7f16e00fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 0x7f16e00fcc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16d800b4a0 tx=0x7f16d800b960 comp rx=0 tx=0).stop 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 shutdown_connections 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f16cc0a3350 0x7f16cc0a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0102ef0 0x7f16e00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e0102570 0x7f16e00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 --2- 172.21.15.64:0/1445414820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e0100e20 0x7f16e00fcc80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 >> 172.21.15.64:0/1445414820 conn(0x7f16e00fc7d0 msgr2=0x7f16e006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 shutdown_connections 2024-09-19T05:05:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:17.357+0000 7f16e6fa2640 1 -- 172.21.15.64:0/1445414820 wait complete. 2024-09-19T05:05:18.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:18.527 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:18 smithi064 ceph-mon[30960]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:18.527 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:18 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1445414820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:18.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:18 smithi174 ceph-mon[34488]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:18.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:18 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1445414820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:18.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 -- 172.21.15.64:0/988334715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94102790 msgr2=0x7f0c941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:18.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 --2- 172.21.15.64:0/988334715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94102790 0x7f0c941067e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009900 tx=0x7f0c7c02a410 comp rx=0 tx=0).stop 2024-09-19T05:05:18.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 -- 172.21.15.64:0/988334715 shutdown_connections 2024-09-19T05:05:18.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 --2- 172.21.15.64:0/988334715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94106f10 0x7f0c9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 --2- 172.21.15.64:0/988334715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94102790 0x7f0c941067e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 --2- 172.21.15.64:0/988334715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c94101de0 0x7f0c941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 -- 172.21.15.64:0/988334715 >> 172.21.15.64:0/988334715 conn(0x7f0c940fd8c0 msgr2=0x7f0c940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:18.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 -- 172.21.15.64:0/988334715 shutdown_connections 2024-09-19T05:05:18.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.696+0000 7f0c997bb640 1 -- 172.21.15.64:0/988334715 wait complete. 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 Processor -- start 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 -- start start 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 0x7f0c941983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94102790 0x7f0c94198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c94106f10 0x7f0c9419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94111de0 con 0x7f0c94101de0 2024-09-19T05:05:18.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94111c60 con 0x7f0c94102790 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c997bb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94111f60 con 0x7f0c94106f10 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 0x7f0c941983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 0x7f0c941983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41500/0 (socket says 172.21.15.64:41500) 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 -- 172.21.15.64:0/2117600834 learned_addr learned my addr 172.21.15.64:0/2117600834 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c8bfff640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94102790 0x7f0c94198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c937fe640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c94106f10 0x7f0c9419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 -- 172.21.15.64:0/2117600834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c94106f10 msgr2=0x7f0c9419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c94106f10 0x7f0c9419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 -- 172.21.15.64:0/2117600834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94102790 msgr2=0x7f0c94198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94102790 0x7f0c94198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c92ffd640 1 -- 172.21.15.64:0/2117600834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9419d390 con 0x7f0c94101de0 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.697+0000 7f0c8bfff640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94102790 0x7f0c94198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:18.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.698+0000 7f0c92ffd640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 0x7f0c941983e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400d950 tx=0x7f0c8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.698+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84017030 con 0x7f0c94101de0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.698+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c84004020 con 0x7f0c94101de0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.698+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9419d680 con 0x7f0c94101de0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.698+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c840047a0 con 0x7f0c94101de0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.698+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c941a4f60 con 0x7f0c94101de0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.699+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c941021c0 con 0x7f0c94101de0 2024-09-19T05:05:18.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.701+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0c840041d0 con 0x7f0c94101de0 2024-09-19T05:05:18.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.701+0000 7f0c90ff9640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0c800a3350 0x7f0c800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:18.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.702+0000 7f0c8bfff640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0c800a3350 0x7f0c800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:18.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.702+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f0c840cfbd0 con 0x7f0c94101de0 2024-09-19T05:05:18.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.702+0000 7f0c8bfff640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0c800a3350 0x7f0c800a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c031440 tx=0x7f0c7c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:18.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.705+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c8409c7f0 con 0x7f0c94101de0 2024-09-19T05:05:18.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:18 smithi103 ceph-mon[34446]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:18.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:18 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1445414820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:18.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.915+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c94109b60 con 0x7f0c94101de0 2024-09-19T05:05:18.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.916+0000 7f0c90ff9640 1 -- 172.21.15.64:0/2117600834 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f0c8409c0f0 con 0x7f0c94101de0 2024-09-19T05:05:18.919 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:18.919 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-19T05:04:20:028478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:04:19.601259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24448,24466,24479]},"id":1}]} 2024-09-19T05:05:18.919 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0c800a3350 msgr2=0x7f0c800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0c800a3350 0x7f0c800a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c031440 tx=0x7f0c7c005a20 comp rx=0 tx=0).stop 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 msgr2=0x7f0c941983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 0x7f0c941983e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400d950 tx=0x7f0c8400de10 comp rx=0 tx=0).stop 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 shutdown_connections 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f0c800a3350 0x7f0c800a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c94106f10 0x7f0c9419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c94102790 0x7f0c94198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 --2- 172.21.15.64:0/2117600834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c94101de0 0x7f0c941983e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.918+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 >> 172.21.15.64:0/2117600834 conn(0x7f0c940fd8c0 msgr2=0x7f0c940fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.919+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 shutdown_connections 2024-09-19T05:05:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:18.919+0000 7f0c997bb640 1 -- 172.21.15.64:0/2117600834 wait complete. 2024-09-19T05:05:19.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:19 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2117600834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:19.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:19 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2117600834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:19.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:05:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:19 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2117600834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:20.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 -- 172.21.15.64:0/2021407320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048100a80 msgr2=0x7fa048100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:20.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/2021407320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048100a80 0x7fa048100ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa030009930 tx=0x7fa03002a420 comp rx=0 tx=0).stop 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 -- 172.21.15.64:0/2021407320 shutdown_connections 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/2021407320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa048101420 0x7fa04810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/2021407320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048100a80 0x7fa048100ee0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/2021407320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048106a80 0x7fa048106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 -- 172.21.15.64:0/2021407320 >> 172.21.15.64:0/2021407320 conn(0x7fa0480fc7a0 msgr2=0x7fa0480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 -- 172.21.15.64:0/2021407320 shutdown_connections 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 -- 172.21.15.64:0/2021407320 wait complete. 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.254+0000 7fa04d4dd640 1 Processor -- start 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 -- start start 2024-09-19T05:05:20.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048100a80 0x7fa0480fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 0x7fa0480fd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa048106a80 0x7fa0480fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa048110d80 con 0x7fa048101420 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa048110c00 con 0x7fa048106a80 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa04d4dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa048110f00 con 0x7fa048100a80 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 0x7fa0480fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 0x7fa0480fd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41518/0 (socket says 172.21.15.64:41518) 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 -- 172.21.15.64:0/3827603351 learned_addr learned my addr 172.21.15.64:0/3827603351 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0477fe640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa048106a80 0x7fa0480fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa046ffd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048100a80 0x7fa0480fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 -- 172.21.15.64:0/3827603351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048100a80 msgr2=0x7fa0480fcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:20.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048100a80 0x7fa0480fcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 -- 172.21.15.64:0/3827603351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa048106a80 msgr2=0x7fa0480fd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa048106a80 0x7fa0480fd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa0467fc640 1 -- 172.21.15.64:0/3827603351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0481a67f0 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.255+0000 7fa046ffd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048100a80 0x7fa0480fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.256+0000 7fa0467fc640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 0x7fa0480fd290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa030031440 tx=0x7fa030003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.256+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03003f070 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.256+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa030031d60 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.256+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0481a6a80 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.256+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030035030 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.256+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0481a6f60 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.257+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa048100ee0 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.258+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa030042070 con 0x7fa048101420 2024-09-19T05:05:20.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.259+0000 7fa03ffff640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fa0340a32d0 0x7fa0340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:20.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.259+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fa0300f07c0 con 0x7fa048101420 2024-09-19T05:05:20.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.259+0000 7fa046ffd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fa0340a32d0 0x7fa0340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:20.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.259+0000 7fa046ffd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fa0340a32d0 0x7fa0340a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa038005e10 tx=0x7fa038005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:20.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.262+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0300bd360 con 0x7fa048101420 2024-09-19T05:05:20.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.440+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa04810a000 con 0x7fa048101420 2024-09-19T05:05:20.443 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:20 smithi064 ceph-mon[30960]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:20.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.442+0000 7fa03ffff640 1 -- 172.21.15.64:0/3827603351 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22821 (secure 0 0 0) 0x7fa0300bcc60 con 0x7fa048101420 2024-09-19T05:05:20.445 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:20.445 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:20.028795+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:05:20.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fa0340a32d0 msgr2=0x7fa0340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fa0340a32d0 0x7fa0340a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa038005e10 tx=0x7fa038005da0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 msgr2=0x7fa0480fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 0x7fa0480fd290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa030031440 tx=0x7fa030003d50 comp rx=0 tx=0).stop 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 shutdown_connections 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fa0340a32d0 0x7fa0340a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa048106a80 0x7fa0480fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa048101420 0x7fa0480fd290 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 --2- 172.21.15.64:0/3827603351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048100a80 0x7fa0480fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:20.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 >> 172.21.15.64:0/3827603351 conn(0x7fa0480fc7a0 msgr2=0x7fa04806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:20.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 shutdown_connections 2024-09-19T05:05:20.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:20.445+0000 7fa04d4dd640 1 -- 172.21.15.64:0/3827603351 wait complete. 2024-09-19T05:05:20.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:20 smithi174 ceph-mon[34488]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:20.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:20 smithi103 ceph-mon[34446]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:21.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:21 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3827603351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:21.430 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:21 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3827603351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:21.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:21 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3827603351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 -- 172.21.15.64:0/3427357840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 msgr2=0x7f6bd01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3427357840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01029b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009900 tx=0x7f6bb802a480 comp rx=0 tx=0).stop 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 -- 172.21.15.64:0/3427357840 shutdown_connections 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3427357840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0102ef0 0x7f6bd010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3427357840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01029b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3427357840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0100e20 0x7f6bd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 -- 172.21.15.64:0/3427357840 >> 172.21.15.64:0/3427357840 conn(0x7f6bd00fc7d0 msgr2=0x7f6bd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 -- 172.21.15.64:0/3427357840 shutdown_connections 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.827+0000 7f6bd506b640 1 -- 172.21.15.64:0/3427357840 wait complete. 2024-09-19T05:05:21.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 Processor -- start 2024-09-19T05:05:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 -- start start 2024-09-19T05:05:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:21.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0111f50 con 0x7f6bd0102570 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0111dd0 con 0x7f6bd0102ef0 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bd506b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01120d0 con 0x7f6bd0100e20 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bc659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bc659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41546/0 (socket says 172.21.15.64:41546) 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bc659c640 1 -- 172.21.15.64:0/3659434922 learned_addr learned my addr 172.21.15.64:0/3659434922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bced9d640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.828+0000 7f6bcf59e640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bc659c640 1 -- 172.21.15.64:0/3659434922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 msgr2=0x7f6bd0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bc659c640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bc659c640 1 -- 172.21.15.64:0/3659434922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 msgr2=0x7f6bd019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:21.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bc659c640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bc659c640 1 -- 172.21.15.64:0/3659434922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd019d320 con 0x7f6bd0102570 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bc659c640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01988b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8006010 tx=0x7f6bb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb803f070 con 0x7f6bd0102570 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd019d5b0 con 0x7f6bd0102570 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bb8003aa0 con 0x7f6bd0102570 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8035030 con 0x7f6bd0102570 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.829+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a4ef0 con 0x7f6bd0102570 2024-09-19T05:05:21.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.830+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd01021b0 con 0x7f6bd0102570 2024-09-19T05:05:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.831+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6bb8042070 con 0x7f6bd0102570 2024-09-19T05:05:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.832+0000 7f6bccd99640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bbc0a32d0 0x7f6bbc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.832+0000 7f6bced9d640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bbc0a32d0 0x7f6bbc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.832+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f6bb80f0320 con 0x7f6bd0102570 2024-09-19T05:05:21.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.832+0000 7f6bced9d640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bbc0a32d0 0x7f6bbc0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000abe0 tx=0x7f6bc000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:21.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:21.836+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6bb80bcec0 con 0x7f6bd0102570 2024-09-19T05:05:22.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.048+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bd0109090 con 0x7f6bd0102570 2024-09-19T05:05:22.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.049+0000 7f6bccd99640 1 -- 172.21.15.64:0/3659434922 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f6bb80bc7c0 con 0x7f6bd0102570 2024-09-19T05:05:22.051 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:22.051 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-19T05:04:20:028478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:04:19.601259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24448,24466,24479]},"id":1}]} 2024-09-19T05:05:22.052 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bbc0a32d0 msgr2=0x7f6bbc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bbc0a32d0 0x7f6bbc0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000abe0 tx=0x7f6bc000a2d0 comp rx=0 tx=0).stop 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 msgr2=0x7f6bd01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01988b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8006010 tx=0x7f6bb80042f0 comp rx=0 tx=0).stop 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 shutdown_connections 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6bbc0a32d0 0x7f6bbc0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bd0102570 0x7f6bd01988b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 --2- 172.21.15.64:0/3659434922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 >> 172.21.15.64:0/3659434922 conn(0x7f6bd00fc7d0 msgr2=0x7f6bd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 shutdown_connections 2024-09-19T05:05:22.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:22.052+0000 7f6bd506b640 1 -- 172.21.15.64:0/3659434922 wait complete. 2024-09-19T05:05:22.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:22 smithi064 ceph-mon[30960]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:22.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:22 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3659434922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:22.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:22 smithi103 ceph-mon[34446]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:22.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:22 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3659434922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:22.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:04:08.534666+0000', 'modified': '2024-09-19T05:04:19.601259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24466, 'mds_2': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6835/642500144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 642500144}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 642500144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/3232384876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 3232384876}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 3232384876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/2004435266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 2004435266}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 2004435266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1599126810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1599126810}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1599126810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/3328031427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3328031427}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3328031427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.103:6839/100524955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 100524955}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 100524955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24448, 24466, 24479]} 2024-09-19T05:05:22.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T05:05:22.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:05:22.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:22 smithi174 ceph-mon[34488]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:22.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:22 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3659434922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:23.013 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:23.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 -- 172.21.15.64:0/1258729640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 msgr2=0x7f914c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 --2- 172.21.15.64:0/1258729640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c10da40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9144009900 tx=0x7f914402a480 comp rx=0 tx=0).stop 2024-09-19T05:05:23.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 -- 172.21.15.64:0/1258729640 shutdown_connections 2024-09-19T05:05:23.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 --2- 172.21.15.64:0/1258729640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c10da40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 --2- 172.21.15.64:0/1258729640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f914c102570 0x7f914c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 --2- 172.21.15.64:0/1258729640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914c100e20 0x7f914c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 -- 172.21.15.64:0/1258729640 >> 172.21.15.64:0/1258729640 conn(0x7f914c0fc7d0 msgr2=0x7f914c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 -- 172.21.15.64:0/1258729640 shutdown_connections 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.387+0000 7f915172f640 1 -- 172.21.15.64:0/1258729640 wait complete. 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 Processor -- start 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 -- start start 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914c100e20 0x7f914c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f914c102570 0x7f914c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f914c111fd0 con 0x7f914c102ef0 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f914c111e50 con 0x7f914c100e20 2024-09-19T05:05:23.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f915172f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f914c112150 con 0x7f914c102570 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f914b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f914b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41564/0 (socket says 172.21.15.64:41564) 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f914b7fe640 1 -- 172.21.15.64:0/2417914217 learned_addr learned my addr 172.21.15.64:0/2417914217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f914affd640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914c100e20 0x7f914c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.388+0000 7f914a7fc640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f914c102570 0x7f914c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f914b7fe640 1 -- 172.21.15.64:0/2417914217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f914c102570 msgr2=0x7f914c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f914b7fe640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f914c102570 0x7f914c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f914b7fe640 1 -- 172.21.15.64:0/2417914217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914c100e20 msgr2=0x7f914c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f914b7fe640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914c100e20 0x7f914c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f914b7fe640 1 -- 172.21.15.64:0/2417914217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c19d4e0 con 0x7f914c102ef0 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f914b7fe640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c19ce00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9144009230 tx=0x7f91440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:23.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f9143fff640 1 -- 172.21.15.64:0/2417914217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914403f030 con 0x7f914c102ef0 2024-09-19T05:05:23.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c19d770 con 0x7f914c102ef0 2024-09-19T05:05:23.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f9143fff640 1 -- 172.21.15.64:0/2417914217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9144003800 con 0x7f914c102ef0 2024-09-19T05:05:23.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f9143fff640 1 -- 172.21.15.64:0/2417914217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144035070 con 0x7f914c102ef0 2024-09-19T05:05:23.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.389+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1a50b0 con 0x7f914c102ef0 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.391+0000 7f9143fff640 1 -- 172.21.15.64:0/2417914217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f91440039b0 con 0x7f914c102ef0 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 Processor -- start 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 -- start start 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76bffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c001b0 0x558c76c0ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558c76b66090 con 0x558c76bff890 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c76b0da70 con 0x558c76c001b0 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9153b08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c76b62050 con 0x558c76c0efc0 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f9143fff640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f91380a32d0 0x7f91380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f914a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76bffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f914a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76bffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41572/0 (socket says 172.21.15.64:41572) 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f914a7fc640 1 -- 172.21.15.64:0/3795716619 learned_addr learned my addr 172.21.15.64:0/3795716619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.392+0000 7f914affd640 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c001b0 0x558c76c0ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914b7fe640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f91380a32d0 0x7f91380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 -- 172.21.15.64:0/3795716619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 msgr2=0x558c76c113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 -- 172.21.15.64:0/3795716619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c001b0 msgr2=0x558c76c0ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c001b0 0x558c76c0ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f9143fff640 1 -- 172.21.15.64:0/2417914217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f91440f0340 con 0x7f914c102ef0 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 -- 172.21.15.64:0/3795716619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c76b1a130 con 0x558c76bff890 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914affd640 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c001b0 0x558c76c0ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:23.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76bffc70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9134009980 tx=0x7f91340096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da63f6b648420b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:23.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f914b7fe640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f91380a32d0 0x7f91380a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f91440bc770 tx=0x7f91440c0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:23.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f91417fa640 1 -- 172.21.15.64:0/3795716619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134010660 con 0x558c76bff890 2024-09-19T05:05:23.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f91417fa640 1 -- 172.21.15.64:0/3795716619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134010cb0 con 0x558c76bff890 2024-09-19T05:05:23.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.393+0000 7f91417fa640 1 -- 172.21.15.64:0/3795716619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134019c10 con 0x558c76bff890 2024-09-19T05:05:23.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 -- 172.21.15.64:0/3795716619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 msgr2=0x558c76bffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76bffc70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9134009980 tx=0x7f91340096d0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 -- 172.21.15.64:0/3795716619 shutdown_connections 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c001b0 0x558c76c0ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 --2- 172.21.15.64:0/3795716619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76bffc70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 -- 172.21.15.64:0/3795716619 >> 172.21.15.64:0/3795716619 conn(0x558c76bfbf60 msgr2=0x558c76bfc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 -- 172.21.15.64:0/3795716619 shutdown_connections 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 -- 172.21.15.64:0/3795716619 wait complete. 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 -- start start 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.394+0000 7f9153b08740 1 Processor -- start 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c0e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c0bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 0x558c76c0c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558c76c11b90 con 0x558c76c0efc0 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c76b20f20 con 0x558c76bff890 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c76c11d10 con 0x558c76c001b0 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f9153b08740 10 client.0 _ll_register_callbacks cb 0x558c76aea030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 0x558c76c0c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 0x558c76c0c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41588/0 (socket says 172.21.15.64:41588) 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 -- 172.21.15.64:0/1275006602 learned_addr learned my addr 172.21.15.64:0/1275006602 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c0e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914affd640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c0bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 -- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 msgr2=0x558c76c0bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c0bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 -- 172.21.15.64:0/1275006602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 msgr2=0x558c76c0e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c0e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 -- 172.21.15.64:0/1275006602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c768d72b0 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 0x558c76c0c170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f913400a010 tx=0x7f913400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfc525db5164788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.395+0000 7f91327fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f91327fc640 1 -- 172.21.15.64:0/1275006602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913402d060 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f91327fc640 1 -- 172.21.15.64:0/1275006602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f9134019da0 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9134010cb0 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f91327fc640 1 -- 172.21.15.64:0/1275006602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f913401f040 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 10 client.14949 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558c76c1d550 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f91327fc640 1 -- 172.21.15.64:0/1275006602 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134010cb0 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f91327fc640 1 -- 172.21.15.64:0/1275006602 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9134025420 con 0x558c76c0efc0 2024-09-19T05:05:23.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 10 client.14949 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 10 client.14949 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 20 client.14949 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c1f8b0 0x558c76c20d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 4 client.14949 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.396+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c76b73600 con 0x558c76c1f8b0 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.397+0000 7f914affd640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c1f8b0 0x558c76c20d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.398+0000 7f91327fc640 10 client.14949 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:23.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.398+0000 7f914affd640 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c1f8b0 0x558c76c20d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a10a71a7653001a6 server_cookie=7608341051053f96 in_seq=0 out_seq=0 2024-09-19T05:05:23.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f91327fc640 1 -- 172.21.15.64:0/1275006602 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c76b73600 con 0x558c76c1f8b0 2024-09-19T05:05:23.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f91327fc640 10 client.14949 handle_command_reply: tid=0 2024-09-19T05:05:23.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 client.14949 shutdown 2024-09-19T05:05:23.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 msgr2=0x558c76c0c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 0x558c76c0c170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f913400a010 tx=0x7f913400c040 comp rx=0 tx=0).stop 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 shutdown_connections 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c1f8b0 0x558c76c20d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c0efc0 0x558c76c0c170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c0bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 --2- 172.21.15.64:0/1275006602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c0e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f91327fc640 0 client.14949 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 >> 172.21.15.64:0/1275006602 conn(0x558c76bfbf60 msgr2=0x558c76c01cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 shutdown_connections 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 1 -- 172.21.15.64:0/1275006602 wait complete. 2024-09-19T05:05:23.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.399+0000 7f9153b08740 20 client.14949 trim_cache size 0 max 16384 2024-09-19T05:05:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.417+0000 7f9153b08740 1 Processor -- start 2024-09-19T05:05:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.417+0000 7f9153b08740 1 -- start start 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f9153b08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76c1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f9153b08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c1f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f9153b08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c0efc0 0x558c76c1e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f9153b08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558c76c1f060 con 0x558c76bff890 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f9153b08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c76c1f730 con 0x558c76c0efc0 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f9153b08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c76c1fe00 con 0x558c76c001b0 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76c1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76c1f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41602/0 (socket says 172.21.15.64:41602) 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 -- 172.21.15.64:0/3803993618 learned_addr learned my addr 172.21.15.64:0/3803993618 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 -- 172.21.15.64:0/3803993618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 msgr2=0x558c76c1f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914affd640 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c1f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c1f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 -- 172.21.15.64:0/3803993618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c0efc0 msgr2=0x558c76c1e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c0efc0 0x558c76c1e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 -- 172.21.15.64:0/3803993618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c76bfffe0 con 0x558c76bff890 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.418+0000 7f914a7fc640 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76c1f1e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f913400a690 tx=0x7f91340094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c8481b70cd957 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9140ff9640 1 -- 172.21.15.64:0/3803993618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913402d040 con 0x558c76bff890 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9140ff9640 1 -- 172.21.15.64:0/3803993618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134009ad0 con 0x558c76bff890 2024-09-19T05:05:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9140ff9640 1 -- 172.21.15.64:0/3803993618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134019ba0 con 0x558c76bff890 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 -- 172.21.15.64:0/3803993618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 msgr2=0x558c76c1f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76c1f1e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f913400a690 tx=0x7f91340094f0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 -- 172.21.15.64:0/3803993618 shutdown_connections 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76c0efc0 0x558c76c1e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c001b0 0x558c76c1f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 --2- 172.21.15.64:0/3803993618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76bff890 0x558c76c1f1e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 -- 172.21.15.64:0/3803993618 >> 172.21.15.64:0/3803993618 conn(0x558c76bfbf60 msgr2=0x558c76bfc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 -- 172.21.15.64:0/3803993618 shutdown_connections 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.419+0000 7f9153b08740 1 -- 172.21.15.64:0/3803993618 wait complete. 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 -- start start 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 Processor -- start 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c26dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 0x558c76c27300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c22340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558c76c20ba0 con 0x558c76c001b0 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c76c204d0 con 0x558c76bff890 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c76c21270 con 0x558c76c0efc0 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f9153b08740 10 client.0 _ll_register_callbacks cb 0x7f913400a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 0x558c76c27300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 0x558c76c27300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41604/0 (socket says 172.21.15.64:41604) 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 -- 172.21.15.64:0/1767611432 learned_addr learned my addr 172.21.15.64:0/1767611432 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c22340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c26dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 -- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 msgr2=0x558c76c22340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c22340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 -- 172.21.15.64:0/1767611432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 msgr2=0x558c76c26dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c26dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914affd640 1 -- 172.21.15.64:0/1767611432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c76c22980 con 0x558c76c001b0 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.420+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c26dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f914affd640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 0x558c76c27300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f913c000c00 tx=0x7f913c006ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3639543302a92aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f91327fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f91327fc640 1 -- 172.21.15.64:0/1767611432 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c018c80 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f91327fc640 1 -- 172.21.15.64:0/1767611432 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f913c0059c0 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9134009ad0 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f91327fc640 1 -- 172.21.15.64:0/1767611432 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f913c018e20 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 10 client.14955 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558c76c26510 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f91327fc640 1 -- 172.21.15.64:0/1767611432 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134009ad0 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f91327fc640 1 -- 172.21.15.64:0/1767611432 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f913c027470 con 0x558c76c001b0 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 10 client.14955 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 10 client.14955 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 20 client.14955 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:23.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.421+0000 7f9153b08740 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c8e520 0x558c76c8d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f9153b08740 4 client.14955 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "status"}) -- 0x558c76b6db10 con 0x558c76c8e520 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c8e520 0x558c76c8d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f914a7fc640 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c8e520 0x558c76c8d230 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7df1ae7b144f6cd server_cookie=2b56e471fe079f4 in_seq=0 out_seq=0 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f91327fc640 10 client.14955 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f91327fc640 1 -- 172.21.15.64:0/1767611432 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558c76b6db10 con 0x558c76c8e520 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f91327fc640 10 client.14955 handle_command_reply: tid=0 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f9153b08740 1 client.14955 shutdown 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 msgr2=0x558c76c27300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.422+0000 7f9153b08740 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 0x558c76c27300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f913c000c00 tx=0x7f913c006ec0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 shutdown_connections 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x558c76c8e520 0x558c76c8d230 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c76c0efc0 0x558c76c22340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c76c001b0 0x558c76c27300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 --2- 172.21.15.64:0/1767611432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c76bff890 0x558c76c26dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 >> 172.21.15.64:0/1767611432 conn(0x558c76bfbf60 msgr2=0x558c76c1e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f91327fc640 0 client.14955 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 shutdown_connections 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 1 -- 172.21.15.64:0/1767611432 wait complete. 2024-09-19T05:05:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f9153b08740 20 client.14955 trim_cache size 0 max 16384 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24448, 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 66.412617675999996, 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-09-19T05:05:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 78, 2024-09-19T05:05:23.432 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 98.899656671000002 2024-09-19T05:05:23.432 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f91380a32d0 msgr2=0x7f91380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f91380a32d0 0x7f91380a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f91440bc770 tx=0x7f91440c0040 comp rx=0 tx=0).stop 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 msgr2=0x7f914c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c19ce00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9144009230 tx=0x7f91440042f0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 shutdown_connections 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f91380a32d0 0x7f91380a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f914c102ef0 0x7f914c19ce00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f914c102570 0x7f914c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 --2- 172.21.15.64:0/2417914217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914c100e20 0x7f914c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 >> 172.21.15.64:0/2417914217 conn(0x7f914c0fc7d0 msgr2=0x7f914c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:23.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.423+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 shutdown_connections 2024-09-19T05:05:23.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:23.424+0000 7f915172f640 1 -- 172.21.15.64:0/2417914217 wait complete. 2024-09-19T05:05:24.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:05:24.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:24 smithi064 ceph-mon[30960]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:24.424 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:24.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:24 smithi103 ceph-mon[34446]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:24.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:24 smithi174 ceph-mon[34488]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.851+0000 7f583c255640 1 -- 172.21.15.64:0/4070148773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102570 msgr2=0x7f58341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.851+0000 7f583c255640 1 --2- 172.21.15.64:0/4070148773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102570 0x7f58341029b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5828009900 tx=0x7f582802a480 comp rx=0 tx=0).stop 2024-09-19T05:05:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 -- 172.21.15.64:0/4070148773 shutdown_connections 2024-09-19T05:05:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 --2- 172.21.15.64:0/4070148773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102ef0 0x7f583410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 --2- 172.21.15.64:0/4070148773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102570 0x7f58341029b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 --2- 172.21.15.64:0/4070148773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834100e20 0x7f5834101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 -- 172.21.15.64:0/4070148773 >> 172.21.15.64:0/4070148773 conn(0x7f58340fc7d0 msgr2=0x7f58340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 -- 172.21.15.64:0/4070148773 shutdown_connections 2024-09-19T05:05:24.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 -- 172.21.15.64:0/4070148773 wait complete. 2024-09-19T05:05:24.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 Processor -- start 2024-09-19T05:05:24.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 -- start start 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.852+0000 7f583c255640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834100e20 0x7f58341985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102570 0x7f5834198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 0x7f583419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110ed0 con 0x7f5834102ef0 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110d50 con 0x7f5834102570 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5834111050 con 0x7f5834100e20 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 0x7f583419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 0x7f583419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55638/0 (socket says 172.21.15.64:55638) 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 -- 172.21.15.64:0/3683249930 learned_addr learned my addr 172.21.15.64:0/3683249930 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f58397f0640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102570 0x7f5834198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 -- 172.21.15.64:0/3683249930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834100e20 msgr2=0x7f58341985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834100e20 0x7f58341985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 -- 172.21.15.64:0/3683249930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102570 msgr2=0x7f5834198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102570 0x7f5834198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 -- 172.21.15.64:0/3683249930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419d550 con 0x7f5834102ef0 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f58397f0640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102570 0x7f5834198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583a7f2640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 0x7f583419ce70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f582400b970 tx=0x7f582400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f5832ffd640 1 -- 172.21.15.64:0/3683249930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824019070 con 0x7f5834102ef0 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f5832ffd640 1 -- 172.21.15.64:0/3683249930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5824014440 con 0x7f5834102ef0 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419d840 con 0x7f5834102ef0 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f5832ffd640 1 -- 172.21.15.64:0/3683249930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58240075d0 con 0x7f5834102ef0 2024-09-19T05:05:24.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.853+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a5120 con 0x7f5834102ef0 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.856+0000 7f583e62e740 1 Processor -- start 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.856+0000 7f5832ffd640 1 -- 172.21.15.64:0/3683249930 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f582401c070 con 0x7f5834102ef0 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.856+0000 7f583e62e740 1 -- start start 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.856+0000 7f583e62e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a90c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f583e62e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 0x55c262a9fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f583e62e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a9ffc0 0x55c262aa23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f583e62e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c2629f7090 con 0x55c262a911d0 2024-09-19T05:05:24.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f583e62e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c26299ea70 con 0x55c262a908b0 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f583e62e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2629f3050 con 0x55c262a9ffc0 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 0x55c262a9fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 0x55c262a9fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55646/0 (socket says 172.21.15.64:55646) 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 -- 172.21.15.64:0/2948648079 learned_addr learned my addr 172.21.15.64:0/2948648079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f58397f0640 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a90c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5832ffd640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f581c0a32d0 0x7f581c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f58397f0640 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a9ffc0 0x55c262aa23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 -- 172.21.15.64:0/2948648079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a9ffc0 msgr2=0x55c262aa23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a9ffc0 0x55c262aa23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f583a7f2640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f581c0a32d0 0x7f581c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 -- 172.21.15.64:0/2948648079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 msgr2=0x55c262a90c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a90c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 -- 172.21.15.64:0/2948648079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2629ab130 con 0x55c262a911d0 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f58397f0640 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a9ffc0 0x55c262aa23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5832ffd640 1 -- 172.21.15.64:0/3683249930 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f58240cf020 con 0x7f5834102ef0 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.857+0000 7f5839ff1640 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 0x55c262a9fa80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f582c009820 tx=0x7f582c009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8b400fae0c87c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f581bfff640 1 -- 172.21.15.64:0/2948648079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c015020 con 0x55c262a911d0 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f581bfff640 1 -- 172.21.15.64:0/2948648079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c011420 con 0x55c262a911d0 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f581bfff640 1 -- 172.21.15.64:0/2948648079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0105e0 con 0x55c262a911d0 2024-09-19T05:05:24.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583a7f2640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f581c0a32d0 0x7f581c0a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5824065fe0 tx=0x7f582406e450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 -- 172.21.15.64:0/2948648079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 msgr2=0x55c262a9fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 0x55c262a9fa80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f582c009820 tx=0x7f582c009ce0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 -- 172.21.15.64:0/2948648079 shutdown_connections 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a9ffc0 0x55c262aa23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a911d0 0x55c262a9fa80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 --2- 172.21.15.64:0/2948648079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a90c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.858+0000 7f583e62e740 1 -- 172.21.15.64:0/2948648079 >> 172.21.15.64:0/2948648079 conn(0x55c262a8cf60 msgr2=0x55c262a8d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 -- 172.21.15.64:0/2948648079 shutdown_connections 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 -- 172.21.15.64:0/2948648079 wait complete. 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 -- start start 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 Processor -- start 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a9f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 0x55c262a9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c262aa2b90 con 0x55c262a9ffc0 2024-09-19T05:05:24.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2629b1f20 con 0x55c262a908b0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c262aa2d10 con 0x55c262a911d0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.859+0000 7f583e62e740 10 client.0 _ll_register_callbacks cb 0x55c26297b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 0x55c262a9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 0x55c262a9d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55652/0 (socket says 172.21.15.64:55652) 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 -- 172.21.15.64:0/3185413034 learned_addr learned my addr 172.21.15.64:0/3185413034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a9f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f5839ff1640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 -- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 msgr2=0x55c262a9cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 -- 172.21.15.64:0/3185413034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 msgr2=0x55c262a9f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a9f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 -- 172.21.15.64:0/3185413034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2627682b0 con 0x55c262a9ffc0 2024-09-19T05:05:24.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58397f0640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 0x55c262a9d0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5828009340 tx=0x7f58280092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39989d81beeb990 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58197fa640 1 -- 172.21.15.64:0/3185413034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828035070 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58197fa640 1 -- 172.21.15.64:0/3185413034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f582803eb40 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c011420 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f58197fa640 1 -- 172.21.15.64:0/3185413034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f582803ed40 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f583e62e740 10 client.14967 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.860+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c262aae410 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f58197fa640 1 -- 172.21.15.64:0/3185413034 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c011420 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f58197fa640 1 -- 172.21.15.64:0/3185413034 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f582804b420 con 0x55c262a9ffc0 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f583e62e740 10 client.14967 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f583e62e740 10 client.14967 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f583e62e740 20 client.14967 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:24.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f583e62e740 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262ab09f0 0x55c262ab1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f583e62e740 4 client.14967 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c262a04600 con 0x55c262ab09f0 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.861+0000 7f5839ff1640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262ab09f0 0x55c262ab1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.862+0000 7f5839ff1640 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262ab09f0 0x55c262ab1e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1bbb68ceeb0b5da server_cookie=8237bd982a2635f in_seq=0 out_seq=0 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.862+0000 7f58197fa640 10 client.14967 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f58197fa640 1 -- 172.21.15.64:0/3185413034 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c262a04600 con 0x55c262ab09f0 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f58197fa640 10 client.14967 handle_command_reply: tid=0 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 client.14967 shutdown 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 msgr2=0x55c262a9d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 0x55c262a9d0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5828009340 tx=0x7f58280092d0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 shutdown_connections 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262ab09f0 0x55c262ab1e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a9ffc0 0x55c262a9d0a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 --2- 172.21.15.64:0/3185413034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a908b0 0x55c262a9f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f58197fa640 0 client.14967 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 >> 172.21.15.64:0/3185413034 conn(0x55c262a8cf60 msgr2=0x55c262a92cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 shutdown_connections 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 1 -- 172.21.15.64:0/3185413034 wait complete. 2024-09-19T05:05:24.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.863+0000 7f583e62e740 20 client.14967 trim_cache size 0 max 16384 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 Processor -- start 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 -- start start 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262ab1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262aba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c262ab01a0 con 0x55c262a908b0 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c262ab0870 con 0x55c262a9ffc0 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f583e62e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c262ab0f40 con 0x55c262a911d0 2024-09-19T05:05:24.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262ab1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262ab1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55654/0 (socket says 172.21.15.64:55654) 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 -- 172.21.15.64:0/3463917680 learned_addr learned my addr 172.21.15.64:0/3463917680 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f5839ff1640 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 -- 172.21.15.64:0/3463917680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 msgr2=0x55c262a9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 -- 172.21.15.64:0/3463917680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 msgr2=0x55c262aba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262aba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.881+0000 7f58397f0640 1 -- 172.21.15.64:0/3463917680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c262a9fb80 con 0x55c262a908b0 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f58397f0640 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262ab1e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58280096c0 tx=0x7f5828005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615bf65333a3591a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f581b7fe640 1 -- 172.21.15.64:0/3463917680 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828035040 con 0x55c262a908b0 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f581b7fe640 1 -- 172.21.15.64:0/3463917680 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582803eb40 con 0x55c262a908b0 2024-09-19T05:05:24.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f581b7fe640 1 -- 172.21.15.64:0/3463917680 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828004040 con 0x55c262a908b0 2024-09-19T05:05:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 -- 172.21.15.64:0/3463917680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 msgr2=0x55c262ab1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262ab1e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58280096c0 tx=0x7f5828005940 comp rx=0 tx=0).stop 2024-09-19T05:05:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 -- 172.21.15.64:0/3463917680 shutdown_connections 2024-09-19T05:05:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262aba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262a9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 --2- 172.21.15.64:0/3463917680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262ab1e60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.882+0000 7f583e62e740 1 -- 172.21.15.64:0/3463917680 >> 172.21.15.64:0/3463917680 conn(0x55c262a8cf60 msgr2=0x55c262a9b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 -- 172.21.15.64:0/3463917680 shutdown_connections 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 -- 172.21.15.64:0/3463917680 wait complete. 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 -- start start 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 Processor -- start 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262aba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262ab3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262ab3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c262ab1ce0 con 0x55c262a908b0 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c262ab1610 con 0x55c262a9ffc0 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c262ab23b0 con 0x55c262a911d0 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f583e62e740 10 client.0 _ll_register_callbacks cb 0x7f5828009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:24.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f58397f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262aba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.883+0000 7f58397f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262aba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55656/0 (socket says 172.21.15.64:55656) 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 -- 172.21.15.64:0/830337532 learned_addr learned my addr 172.21.15.64:0/830337532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262ab3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f5839ff1640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262ab3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 -- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 msgr2=0x55c262ab3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262ab3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 -- 172.21.15.64:0/830337532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 msgr2=0x55c262ab3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262ab3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 -- 172.21.15.64:0/830337532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c262ab4610 con 0x55c262a908b0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58397f0640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262aba6e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f582803e630 tx=0x7f582803e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c8adcdaaf61e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58197fa640 1 -- 172.21.15.64:0/830337532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582805a040 con 0x55c262a908b0 2024-09-19T05:05:24.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58197fa640 1 -- 172.21.15.64:0/830337532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f5828031c50 con 0x55c262a908b0 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582803eb40 con 0x55c262a908b0 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f58197fa640 1 -- 172.21.15.64:0/830337532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5828067070 con 0x55c262a908b0 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f583e62e740 10 client.14973 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.884+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c262b1b100 con 0x55c262a908b0 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f58197fa640 1 -- 172.21.15.64:0/830337532 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582803eb40 con 0x55c262a908b0 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f58197fa640 1 -- 172.21.15.64:0/830337532 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5828060070 con 0x55c262a908b0 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f583e62e740 10 client.14973 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f583e62e740 10 client.14973 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f583e62e740 20 client.14973 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f583e62e740 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262b1f7b0 0x55c262b1e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f583e62e740 4 client.14973 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:05:24.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "status"}) -- 0x55c2629feb10 con 0x55c262b1f7b0 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.885+0000 7f5839ff1640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262b1f7b0 0x55c262b1e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f5839ff1640 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262b1f7b0 0x55c262b1e4c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cda98d493652958c server_cookie=8c46be61bfe1481d in_seq=0 out_seq=0 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f58197fa640 10 client.14973 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f58197fa640 1 -- 172.21.15.64:0/830337532 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c2629feb10 con 0x55c262b1f7b0 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f58197fa640 10 client.14973 handle_command_reply: tid=0 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 client.14973 shutdown 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 msgr2=0x55c262aba6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262aba6e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f582803e630 tx=0x7f582803e690 comp rx=0 tx=0).stop 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 shutdown_connections 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c262a9ffc0 0x55c262ab3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c262a911d0 0x55c262ab3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c262b1f7b0 0x55c262b1e4c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 --2- 172.21.15.64:0/830337532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c262a908b0 0x55c262aba6e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f58197fa640 0 client.14973 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 >> 172.21.15.64:0/830337532 conn(0x55c262a8cf60 msgr2=0x55c262a97a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 shutdown_connections 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 1 -- 172.21.15.64:0/830337532 wait complete. 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.886+0000 7f583e62e740 20 client.14973 trim_cache size 0 max 16384 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24466, 2024-09-19T05:05:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 68.882924478999996, 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 78, 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 95.352544077999994 2024-09-19T05:05:24.894 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f581c0a32d0 msgr2=0x7f581c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f581c0a32d0 0x7f581c0a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5824065fe0 tx=0x7f582406e450 comp rx=0 tx=0).stop 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 msgr2=0x7f583419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 0x7f583419ce70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f582400b970 tx=0x7f582400be30 comp rx=0 tx=0).stop 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 shutdown_connections 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f581c0a32d0 0x7f581c0a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5834102ef0 0x7f583419ce70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834102570 0x7f5834198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 --2- 172.21.15.64:0/3683249930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834100e20 0x7f58341985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 >> 172.21.15.64:0/3683249930 conn(0x7f58340fc7d0 msgr2=0x7f583406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 shutdown_connections 2024-09-19T05:05:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:24.887+0000 7f583c255640 1 -- 172.21.15.64:0/3683249930 wait complete. 2024-09-19T05:05:25.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:05:25.803 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 -- 172.21.15.64:0/2155675410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101fc0 msgr2=0x7fdac8105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2155675410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101fc0 0x7fdac8105710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab402a410 comp rx=0 tx=0).stop 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 -- 172.21.15.64:0/2155675410 shutdown_connections 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2155675410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8105e40 0x7fdac810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2155675410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101fc0 0x7fdac8105710 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2155675410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101610 0x7fdac81019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 -- 172.21.15.64:0/2155675410 >> 172.21.15.64:0/2155675410 conn(0x7fdac80fc5c0 msgr2=0x7fdac80fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 -- 172.21.15.64:0/2155675410 shutdown_connections 2024-09-19T05:05:26.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.335+0000 7fdacebb1640 1 -- 172.21.15.64:0/2155675410 wait complete. 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 Processor -- start 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 -- start start 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 0x7fdac8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101fc0 0x7fdac8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8105e40 0x7fdac819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111f00 con 0x7fdac8101610 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111d80 con 0x7fdac8101fc0 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacebb1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8112080 con 0x7fdac8105e40 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 0x7fdac8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 0x7fdac8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55672/0 (socket says 172.21.15.64:55672) 2024-09-19T05:05:26.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 -- 172.21.15.64:0/2324877850 learned_addr learned my addr 172.21.15.64:0/2324877850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101fc0 0x7fdac8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8105e40 0x7fdac819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 -- 172.21.15.64:0/2324877850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8105e40 msgr2=0x7fdac819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8105e40 0x7fdac819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 -- 172.21.15.64:0/2324877850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101fc0 msgr2=0x7fdac8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101fc0 0x7fdac8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.336+0000 7fdacc94d640 1 -- 172.21.15.64:0/2324877850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac819d510 con 0x7fdac8101610 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.337+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 0x7fdac8198560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00b4a0 tx=0x7fdabc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.337+0000 7fdac5ffb640 1 -- 172.21.15.64:0/2324877850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc014070 con 0x7fdac8101610 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.337+0000 7fdac5ffb640 1 -- 172.21.15.64:0/2324877850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc010ac0 con 0x7fdac8101610 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.337+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac819d800 con 0x7fdac8101610 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.337+0000 7fdac5ffb640 1 -- 172.21.15.64:0/2324877850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc017c30 con 0x7fdac8101610 2024-09-19T05:05:26.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.337+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a50e0 con 0x7fdac8101610 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.339+0000 7fdac5ffb640 1 -- 172.21.15.64:0/2324877850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdabc017070 con 0x7fdac8101610 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 Processor -- start 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 -- start start 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdac5ffb640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdab80a3350 0x7fdab80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e6077ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60789a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e6078c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e606e1090 con 0x562e60789fc0 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e60688a70 con 0x562e6077a8b0 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562e606dd050 con 0x562e6077b1d0 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdab80a3350 0x7fdab80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdac5ffb640 1 -- 172.21.15.64:0/2324877850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fdabc0cf000 con 0x7fdac8101610 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdacd14e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e6078c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdacd14e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e6078c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55678/0 (socket says 172.21.15.64:55678) 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdacc94d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e6077ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdacd14e640 1 -- 172.21.15.64:0/63157554 learned_addr learned my addr 172.21.15.64:0/63157554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdac7fff640 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60789a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.340+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdab80a3350 0x7fdab80a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c830 tx=0x7fdac000cc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacc94d640 1 -- 172.21.15.64:0/63157554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 msgr2=0x562e60789a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacc94d640 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60789a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacc94d640 1 -- 172.21.15.64:0/63157554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 msgr2=0x562e6078c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacc94d640 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e6078c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacc94d640 1 -- 172.21.15.64:0/63157554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e60695130 con 0x562e6077a8b0 2024-09-19T05:05:26.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacd14e640 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e6078c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdacc94d640 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e6077ac90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0ccb40 tx=0x7fdabc09afb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54747db6063e5ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdab2ffd640 1 -- 172.21.15.64:0/63157554 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e070 con 0x562e6077a8b0 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdab2ffd640 1 -- 172.21.15.64:0/63157554 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc09fa00 con 0x562e6077a8b0 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.341+0000 7fdab2ffd640 1 -- 172.21.15.64:0/63157554 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0a3e30 con 0x562e6077a8b0 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 -- 172.21.15.64:0/63157554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 msgr2=0x562e6077ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e6077ac90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0ccb40 tx=0x7fdabc09afb0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 -- 172.21.15.64:0/63157554 shutdown_connections 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e6078c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60789a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/63157554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e6077ac90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 -- 172.21.15.64:0/63157554 >> 172.21.15.64:0/63157554 conn(0x562e60776f60 msgr2=0x562e60777340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 -- 172.21.15.64:0/63157554 shutdown_connections 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 -- 172.21.15.64:0/63157554 wait complete. 2024-09-19T05:05:26.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 -- start start 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.342+0000 7fdad0f8a740 1 Processor -- start 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60786bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e60787130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e6078cb90 con 0x562e60789fc0 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e6069bf20 con 0x562e6077a8b0 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562e6078cd10 con 0x562e6077b1d0 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 10 client.0 _ll_register_callbacks cb 0x562e60665030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e60787130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e60787130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55684/0 (socket says 172.21.15.64:55684) 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 -- 172.21.15.64:0/3066627193 learned_addr learned my addr 172.21.15.64:0/3066627193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:26.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacc94d640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdac7fff640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60786bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 -- 172.21.15.64:0/3066627193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 msgr2=0x562e60786bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60786bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 -- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 msgr2=0x562e607898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 -- 172.21.15.64:0/3066627193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e604522b0 con 0x562e60789fc0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdac7fff640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60786bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdacd14e640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e60787130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009d60 tx=0x7fdac0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb93ed7c8b4cbccc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdaa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdaa7fff640 1 -- 172.21.15.64:0/3066627193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac002b060 con 0x562e60789fc0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdaa7fff640 1 -- 172.21.15.64:0/3066627193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fdac001a7e0 con 0x562e60789fc0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.343+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc09fa00 con 0x562e60789fc0 2024-09-19T05:05:26.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdaa7fff640 1 -- 172.21.15.64:0/3066627193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdac0015040 con 0x562e60789fc0 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 10 client.14985 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e60798540 con 0x562e60789fc0 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdaa7fff640 1 -- 172.21.15.64:0/3066627193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc09fa00 con 0x562e60789fc0 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdaa7fff640 1 -- 172.21.15.64:0/3066627193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdac0026b80 con 0x562e60789fc0 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 10 client.14985 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 20 client.14985 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e6079bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 4 client.14985 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:26.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e606ee600 con 0x562e6079ab60 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.344+0000 7fdacc94d640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e6079bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.345+0000 7fdacc94d640 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e6079bfd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e07c151e62f31c server_cookie=b1ad8c9fab23080c in_seq=0 out_seq=0 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.345+0000 7fdaa7fff640 10 client.14985 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.345+0000 7fdaa7fff640 1 -- 172.21.15.64:0/3066627193 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e606ee600 con 0x562e6079ab60 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.345+0000 7fdaa7fff640 10 client.14985 handle_command_reply: tid=0 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 client.14985 shutdown 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 msgr2=0x562e60787130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e60787130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009d60 tx=0x7fdac0018000 comp rx=0 tx=0).stop 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 shutdown_connections 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e6079bfd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdaa7fff640 0 client.14985 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e60789fc0 0x562e60787130 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077b1d0 0x562e60786bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/3066627193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 >> 172.21.15.64:0/3066627193 conn(0x562e60776f60 msgr2=0x562e6077ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 shutdown_connections 2024-09-19T05:05:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 1 -- 172.21.15.64:0/3066627193 wait complete. 2024-09-19T05:05:26.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.346+0000 7fdad0f8a740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T05:05:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 Processor -- start 2024-09-19T05:05:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 -- start start 2024-09-19T05:05:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077a8b0 0x562e607854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e60785a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e60789fc0 0x562e607a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e6079a310 con 0x562e6077b1d0 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e6079a9e0 con 0x562e60789fc0 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562e6079b0b0 con 0x562e6077a8b0 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e60785a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e60785a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55692/0 (socket says 172.21.15.64:55692) 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 -- 172.21.15.64:0/2569236684 learned_addr learned my addr 172.21.15.64:0/2569236684 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e60789fc0 0x562e607a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 -- 172.21.15.64:0/2569236684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077a8b0 msgr2=0x562e607854c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077a8b0 0x562e607854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077a8b0 0x562e607854c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 -- 172.21.15.64:0/2569236684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e60789fc0 msgr2=0x562e607a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e60789fc0 0x562e607a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdac7fff640 1 -- 172.21.15.64:0/2569236684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e60777ec0 con 0x562e6077b1d0 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.364+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e60789fc0 0x562e607a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.365+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e60785a00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab4031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c7346aaf68f682 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.365+0000 7fdab27fc640 1 -- 172.21.15.64:0/2569236684 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4035040 con 0x562e6077b1d0 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.365+0000 7fdab27fc640 1 -- 172.21.15.64:0/2569236684 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab40317e0 con 0x562e6077b1d0 2024-09-19T05:05:26.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.365+0000 7fdab27fc640 1 -- 172.21.15.64:0/2569236684 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4003de0 con 0x562e6077b1d0 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.365+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2569236684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 msgr2=0x562e60785a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.365+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e60785a00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab4031ea0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2569236684 shutdown_connections 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e60789fc0 0x562e607a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e60785a00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2569236684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e6077a8b0 0x562e607854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2569236684 >> 172.21.15.64:0/2569236684 conn(0x562e60776f60 msgr2=0x562e60786130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2569236684 shutdown_connections 2024-09-19T05:05:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2569236684 wait complete. 2024-09-19T05:05:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- start start 2024-09-19T05:05:26.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 Processor -- start 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607a4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e6079dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e60789fc0 0x562e6079e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562e6079be50 con 0x562e6077b1d0 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e6079b780 con 0x562e6077a8b0 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562e6079c520 con 0x562e60789fc0 2024-09-19T05:05:26.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.366+0000 7fdad0f8a740 10 client.0 _ll_register_callbacks cb 0x7fdac0007c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e6079dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e6079dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55704/0 (socket says 172.21.15.64:55704) 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 -- 172.21.15.64:0/2308628540 learned_addr learned my addr 172.21.15.64:0/2308628540 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e60789fc0 0x562e6079e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 -- 172.21.15.64:0/2308628540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e60789fc0 msgr2=0x562e6079e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e60789fc0 0x562e6079e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 -- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 msgr2=0x562e607a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 -- 172.21.15.64:0/2308628540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6079ea00 con 0x562e6077b1d0 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdacd14e640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e60789fc0 0x562e6079e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdac7fff640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e6079dcc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdab402a450 tx=0x7fdab40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e87c53ccd2fa8ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdaa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdaa7fff640 1 -- 172.21.15.64:0/2308628540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4035030 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab40317e0 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdaa7fff640 1 -- 172.21.15.64:0/2308628540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fdab4003d60 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.367+0000 7fdaa7fff640 1 -- 172.21.15.64:0/2308628540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdab40037b0 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 10 client.14991 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e60805270 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdaa7fff640 1 -- 172.21.15.64:0/2308628540 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab40317e0 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdaa7fff640 1 -- 172.21.15.64:0/2308628540 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdab404eba0 con 0x562e6077b1d0 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 10 client.14991 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 10 client.14991 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 20 client.14991 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e60808580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 4 client.14991 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "status"}) -- 0x562e606e8b10 con 0x562e6079ab60 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.368+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e60808580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdaa7fff640 10 client.14991 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdacc94d640 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e60808580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0b030adbc159ed server_cookie=419f0166d5af2509 in_seq=0 out_seq=0 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdaa7fff640 1 -- 172.21.15.64:0/2308628540 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562e606e8b10 con 0x562e6079ab60 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdaa7fff640 10 client.14991 handle_command_reply: tid=0 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 client.14991 shutdown 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 msgr2=0x562e6079dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e6079dcc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdab402a450 tx=0x7fdab40316d0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 shutdown_connections 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562e60789fc0 0x562e6079e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562e6079ab60 0x562e60808580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e6077b1d0 0x562e6079dcc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 --2- 172.21.15.64:0/2308628540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e6077a8b0 0x562e607a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 >> 172.21.15.64:0/2308628540 conn(0x562e60776f60 msgr2=0x562e60781a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdaa7fff640 0 client.14991 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 shutdown_connections 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 1 -- 172.21.15.64:0/2308628540 wait complete. 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.369+0000 7fdad0f8a740 20 client.14991 trim_cache size 0 max 16384 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:26.378 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24479, 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 77.400955945000007, 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-09-19T05:05:26.379 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 78, 2024-09-19T05:05:26.380 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 87.904691853000003 2024-09-19T05:05:26.380 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdab80a3350 msgr2=0x7fdab80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdab80a3350 0x7fdab80a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c830 tx=0x7fdac000cc30 comp rx=0 tx=0).stop 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 msgr2=0x7fdac8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 0x7fdac8198560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00b4a0 tx=0x7fdabc00b960 comp rx=0 tx=0).stop 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 shutdown_connections 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fdab80a3350 0x7fdab80a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8105e40 0x7fdac819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101fc0 0x7fdac8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 --2- 172.21.15.64:0/2324877850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101610 0x7fdac8198560 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 >> 172.21.15.64:0/2324877850 conn(0x7fdac80fc5c0 msgr2=0x7fdac81041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 shutdown_connections 2024-09-19T05:05:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:26.370+0000 7fdacebb1640 1 -- 172.21.15.64:0/2324877850 wait complete. 2024-09-19T05:05:26.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:26 smithi064 ceph-mon[30960]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:26.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:26 smithi103 ceph-mon[34446]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:26.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:26 smithi174 ceph-mon[34488]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:27.003 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T05:05:27.003 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T05:05:27.015 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:05:27.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T05:05:27.273 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 -- 172.21.15.64:0/560871876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54100761a0 msgr2=0x7f5410074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 --2- 172.21.15.64:0/560871876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54100761a0 0x7f5410074600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5404009900 tx=0x7f540402a410 comp rx=0 tx=0).stop 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 -- 172.21.15.64:0/560871876 shutdown_connections 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 --2- 172.21.15.64:0/560871876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5410109a70 0x7f541010bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 --2- 172.21.15.64:0/560871876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5410074b40 0x7f5410074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 --2- 172.21.15.64:0/560871876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54100761a0 0x7f5410074600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 -- 172.21.15.64:0/560871876 >> 172.21.15.64:0/560871876 conn(0x7f54100fd390 msgr2=0x7f54100ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 -- 172.21.15.64:0/560871876 shutdown_connections 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.750+0000 7f5414fc0640 1 -- 172.21.15.64:0/560871876 wait complete. 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 Processor -- start 2024-09-19T05:05:27.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 -- start start 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 0x7f541019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54100761a0 0x7f541019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5410109a70 0x7f54101a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f541010fa00 con 0x7f5410074b40 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f541010f880 con 0x7f5410109a70 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f5414fc0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f541010fb80 con 0x7f54100761a0 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f540e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 0x7f541019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f540e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 0x7f541019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55710/0 (socket says 172.21.15.64:55710) 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f540e59c640 1 -- 172.21.15.64:0/4138613687 learned_addr learned my addr 172.21.15.64:0/4138613687 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.751+0000 7f540ed9d640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5410109a70 0x7f54101a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540dd9b640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54100761a0 0x7f541019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540e59c640 1 -- 172.21.15.64:0/4138613687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54100761a0 msgr2=0x7f541019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:27.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540e59c640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54100761a0 0x7f541019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540e59c640 1 -- 172.21.15.64:0/4138613687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5410109a70 msgr2=0x7f54101a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540e59c640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5410109a70 0x7f54101a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540e59c640 1 -- 172.21.15.64:0/4138613687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54101a1da0 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540dd9b640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54100761a0 0x7f541019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f540e59c640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 0x7f541019a170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5404005940 tx=0x7f5404004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540403f030 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101a2030 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5404003cb0 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404035070 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.752+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101a25e0 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.753+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5410108880 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.754+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5404042030 con 0x7f5410074b40 2024-09-19T05:05:27.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.755+0000 7f53ff7fe640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f53f80a3300 0x7f53f80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:27.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.755+0000 7f540dd9b640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f53f80a3300 0x7f53f80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:27.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.755+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f54040f0810 con 0x7f5410074b40 2024-09-19T05:05:27.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.756+0000 7f540dd9b640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f53f80a3300 0x7f53f80a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5400006fd0 tx=0x7f5400008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:27.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.758+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f54040bd3b0 con 0x7f5410074b40 2024-09-19T05:05:27.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.953+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5410062630 con 0x7f53f80a3300 2024-09-19T05:05:27.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.954+0000 7f53ff7fe640 1 -- 172.21.15.64:0/4138613687 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5410062630 con 0x7f53f80a3300 2024-09-19T05:05:27.956 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2024-09-19T05:05:27.956 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2024-09-19T05:05:27.956 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f53f80a3300 msgr2=0x7f53f80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f53f80a3300 0x7f53f80a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5400006fd0 tx=0x7f5400008040 comp rx=0 tx=0).stop 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 msgr2=0x7f541019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 0x7f541019a170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5404005940 tx=0x7f5404004430 comp rx=0 tx=0).stop 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 shutdown_connections 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f53f80a3300 0x7f53f80a57c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5410109a70 0x7f54101a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54100761a0 0x7f541019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 --2- 172.21.15.64:0/4138613687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5410074b40 0x7f541019a170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:27.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.956+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 >> 172.21.15.64:0/4138613687 conn(0x7f54100fd390 msgr2=0x7f5410105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:27.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.957+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 shutdown_connections 2024-09-19T05:05:27.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:27.957+0000 7f5414fc0640 1 -- 172.21.15.64:0/4138613687 wait complete. 2024-09-19T05:05:28.540 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:28 smithi064 ceph-mon[30960]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:28.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T05:05:28.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:28 smithi103 ceph-mon[34446]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:28.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:28 smithi174 ceph-mon[34488]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:28.823 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 -- 172.21.15.64:0/758843528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 msgr2=0x7f4538102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 --2- 172.21.15.64:0/758843528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538102000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 -- 172.21.15.64:0/758843528 shutdown_connections 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 --2- 172.21.15.64:0/758843528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45381037c0 0x7f453810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 --2- 172.21.15.64:0/758843528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4538102e00 0x7f4538103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 --2- 172.21.15.64:0/758843528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538102000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 -- 172.21.15.64:0/758843528 >> 172.21.15.64:0/758843528 conn(0x7f45380fd390 msgr2=0x7f45380ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 -- 172.21.15.64:0/758843528 shutdown_connections 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 -- 172.21.15.64:0/758843528 wait complete. 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 Processor -- start 2024-09-19T05:05:29.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.283+0000 7f453ddf5640 1 -- start start 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f453ddf5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f453ddf5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4538102e00 0x7f45381983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f453ddf5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45381037c0 0x7f453819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f453ddf5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f453810d780 con 0x7f4538101c00 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f453ddf5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f453810d600 con 0x7f4538102e00 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f453ddf5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f453810d900 con 0x7f45381037c0 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55730/0 (socket says 172.21.15.64:55730) 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 -- 172.21.15.64:0/1962805417 learned_addr learned my addr 172.21.15.64:0/1962805417 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f4537fff640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45381037c0 0x7f453819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f452effd640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4538102e00 0x7f45381983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 -- 172.21.15.64:0/1962805417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45381037c0 msgr2=0x7f453819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:29.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45381037c0 0x7f453819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 -- 172.21.15.64:0/1962805417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4538102e00 msgr2=0x7f45381983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4538102e00 0x7f45381983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 -- 172.21.15.64:0/1962805417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453819fa90 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f452effd640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4538102e00 0x7f45381983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.284+0000 7f45377fe640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538197e60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f45280314e0 tx=0x7f4528003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.285+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452803f070 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.285+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4528031e40 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.285+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453819fd20 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.285+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528035030 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.285+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45381a01a0 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.286+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45381090a0 con 0x7f4538101c00 2024-09-19T05:05:29.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.287+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4528042070 con 0x7f4538101c00 2024-09-19T05:05:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.288+0000 7f45357fa640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f45240a3380 0x7f45240a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.288+0000 7f452effd640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f45240a3380 0x7f45240a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.288+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f45280f03d0 con 0x7f4538101c00 2024-09-19T05:05:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.288+0000 7f452effd640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f45240a3380 0x7f45240a5840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4538199380 tx=0x7f4520005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:29.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.292+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45280bcf70 con 0x7f4538101c00 2024-09-19T05:05:29.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.480+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4538062630 con 0x7f45240a3380 2024-09-19T05:05:29.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:29 smithi064 ceph-mon[30960]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:29.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.492+0000 7f45357fa640 1 -- 172.21.15.64:0/1962805417 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f4538062630 con 0x7f45240a3380 2024-09-19T05:05:29.493 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T05:05:29.493 INFO:teuthology.orchestra.run.smithi064.stdout:mds.a smithi064 running (94s) 16s ago 93s 14.3M - 19.1.1-361-g2f599a46 d2baa855c067 4e6b160f1010 2024-09-19T05:05:29.493 INFO:teuthology.orchestra.run.smithi064.stdout:mds.b smithi103 running (91s) 24s ago 90s 26.5M - 19.1.1-361-g2f599a46 d2baa855c067 bad5c262f740 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.c smithi174 running (105s) 24s ago 104s 19.1M - 19.1.1-361-g2f599a46 d2baa855c067 932430102e2c 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.d smithi064 running (112s) 16s ago 112s 13.5M - 19.1.1-361-g2f599a46 d2baa855c067 27295cfa4a3b 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.e smithi103 running (110s) 24s ago 109s 18.3M - 19.1.1-361-g2f599a46 d2baa855c067 24163245ae70 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.f smithi174 running (107s) 24s ago 107s 18.2M - 19.1.1-361-g2f599a46 d2baa855c067 15f2a2c6a0bd 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.g smithi064 running (97s) 16s ago 97s 15.7M - 19.1.1-361-g2f599a46 d2baa855c067 648d649d97a1 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.h smithi103 running (102s) 24s ago 102s 16.2M - 19.1.1-361-g2f599a46 d2baa855c067 d9d7bc7fde38 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.i smithi174 running (100s) 24s ago 99s 18.2M - 19.1.1-361-g2f599a46 d2baa855c067 7b1dee635a8b 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.j smithi064 running (116s) 16s ago 115s 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 34fcded5e193 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.k smithi103 running (119s) 24s ago 118s 16.8M - 19.1.1-361-g2f599a46 d2baa855c067 dd2d8a6ada26 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mds.l smithi174 running (2m) 24s ago 2m 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 2ebaa3649d6d 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.x smithi064 *:9283,8765 running (8m) 16s ago 8m 509M - 19.1.1-361-g2f599a46 d2baa855c067 da6dc243a6d3 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.y smithi103 *:8443,8765 running (7m) 24s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 445aed7eeb37 2024-09-19T05:05:29.494 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.z smithi174 *:8443,8765 running (7m) 24s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 a3c9f68e21d3 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:mon.a smithi064 running (8m) 16s ago 9m 51.5M 2048M 19.1.1-361-g2f599a46 d2baa855c067 32af6c18b4dd 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:mon.b smithi103 running (7m) 24s ago 7m 39.6M 2048M 19.1.1-361-g2f599a46 d2baa855c067 42021eb22ec0 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:mon.c smithi174 running (7m) 24s ago 7m 45.7M 2048M 19.1.1-361-g2f599a46 d2baa855c067 fdede484ad09 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (6m) 16s ago 6m 55.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c06d67b8a2f0 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi103 running (6m) 24s ago 6m 58.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 bcdd96ab11c2 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi174 running (6m) 24s ago 6m 49.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b9d498faffd6 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi064 running (5m) 16s ago 5m 57.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 75fc0eb4b007 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi103 running (5m) 24s ago 5m 54.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 74e1822cc9bf 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi174 running (4m) 24s ago 4m 52.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 6978c7c41eb0 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 smithi064 running (4m) 16s ago 4m 44.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b122aa8d0e1e 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 smithi103 running (3m) 24s ago 3m 46.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 1c5b8b16dbda 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 smithi174 running (3m) 24s ago 3m 52.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 4d688e126865 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 smithi064 running (3m) 16s ago 3m 51.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 05b35c0193da 2024-09-19T05:05:29.495 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 smithi103 running (2m) 24s ago 2m 47.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b3bfce76667e 2024-09-19T05:05:29.496 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 smithi174 running (2m) 24s ago 2m 43.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 dbfe52337151 2024-09-19T05:05:29.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f45240a3380 msgr2=0x7f45240a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:29.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f45240a3380 0x7f45240a5840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4538199380 tx=0x7f4520005e50 comp rx=0 tx=0).stop 2024-09-19T05:05:29.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 msgr2=0x7f4538197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:29.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538197e60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f45280314e0 tx=0x7f4528003c80 comp rx=0 tx=0).stop 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 shutdown_connections 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f45240a3380 0x7f45240a5840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45381037c0 0x7f453819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4538102e00 0x7f45381983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 --2- 172.21.15.64:0/1962805417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4538101c00 0x7f4538197e60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 >> 172.21.15.64:0/1962805417 conn(0x7f45380fd390 msgr2=0x7f4538106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 shutdown_connections 2024-09-19T05:05:29.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:29.494+0000 7f453ddf5640 1 -- 172.21.15.64:0/1962805417 wait complete. 2024-09-19T05:05:29.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:29 smithi103 ceph-mon[34446]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:29.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:29 smithi174 ceph-mon[34488]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:30.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T05:05:30.350 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:30.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:30 smithi064 ceph-mon[30960]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:30.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:30 smithi174 ceph-mon[34488]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:30.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:30 smithi103 ceph-mon[34446]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 -- 172.21.15.64:0/1988450360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468104f90 msgr2=0x7f0468107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 --2- 172.21.15.64:0/1988450360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468104f90 0x7f0468107380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f045c009900 tx=0x7f045c02a410 comp rx=0 tx=0).stop 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 -- 172.21.15.64:0/1988450360 shutdown_connections 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 --2- 172.21.15.64:0/1988450360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04681078c0 0x7f0468109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 --2- 172.21.15.64:0/1988450360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468104f90 0x7f0468107380 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 --2- 172.21.15.64:0/1988450360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0468102660 0x7f0468104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 -- 172.21.15.64:0/1988450360 >> 172.21.15.64:0/1988450360 conn(0x7f04680fc280 msgr2=0x7f04680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 -- 172.21.15.64:0/1988450360 shutdown_connections 2024-09-19T05:05:30.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.864+0000 7f046d475640 1 -- 172.21.15.64:0/1988450360 wait complete. 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.865+0000 7f046d475640 1 Processor -- start 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.865+0000 7f046d475640 1 -- start start 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.865+0000 7f046d475640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 0x7f0468197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0468104f90 0x7f0468198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04681078c0 0x7f046819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f046810d740 con 0x7f0468102660 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f046810d5c0 con 0x7f0468104f90 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f046810d8c0 con 0x7f04681078c0 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 0x7f0468197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 0x7f0468197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55754/0 (socket says 172.21.15.64:55754) 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 -- 172.21.15.64:0/3071184324 learned_addr learned my addr 172.21.15.64:0/3071184324 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:30.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 -- 172.21.15.64:0/3071184324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04681078c0 msgr2=0x7f046819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04681078c0 0x7f046819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f04667fc640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0468104f90 0x7f0468198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 -- 172.21.15.64:0/3071184324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0468104f90 msgr2=0x7f0468198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0468104f90 0x7f0468198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 -- 172.21.15.64:0/3071184324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046819fa20 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f0466ffd640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 0x7f0468197df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f045800c920 tx=0x7f045800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458007c70 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0458007e10 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046819fd10 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458018820 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.866+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a01a0 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.867+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04680fdd70 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.869+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f04580189c0 con 0x7f0468102660 2024-09-19T05:05:30.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.869+0000 7f044ffff640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04500a3300 0x7f04500a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.869+0000 7f04667fc640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04500a3300 0x7f04500a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.869+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f04580cff10 con 0x7f0468102660 2024-09-19T05:05:30.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.870+0000 7f04667fc640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04500a3300 0x7f04500a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f045c02a930 tx=0x7f045c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:30.872+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0458014030 con 0x7f0468102660 2024-09-19T05:05:31.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.058+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0468062630 con 0x7f04500a3300 2024-09-19T05:05:31.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.066+0000 7f044ffff640 1 -- 172.21.15.64:0/3071184324 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0468062630 con 0x7f04500a3300 2024-09-19T05:05:31.067 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T05:05:31.067 INFO:teuthology.orchestra.run.smithi064.stdout:mds.all 12/12 26s ago 2m smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T05:05:31.067 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 3/3 26s ago 7m smithi064=x;smithi103=y;smithi174=z;count:3 2024-09-19T05:05:31.067 INFO:teuthology.orchestra.run.smithi064.stdout:mon 3/3 26s ago 7m smithi064:172.21.15.64=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c;count:3 2024-09-19T05:05:31.068 INFO:teuthology.orchestra.run.smithi064.stdout:osd 12 26s ago - 2024-09-19T05:05:31.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04500a3300 msgr2=0x7f04500a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:31.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04500a3300 0x7f04500a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f045c02a930 tx=0x7f045c005fd0 comp rx=0 tx=0).stop 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 msgr2=0x7f0468197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 0x7f0468197df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f045800c920 tx=0x7f045800cde0 comp rx=0 tx=0).stop 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 shutdown_connections 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f04500a3300 0x7f04500a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04681078c0 0x7f046819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0468104f90 0x7f0468198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 --2- 172.21.15.64:0/3071184324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0468102660 0x7f0468197df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 >> 172.21.15.64:0/3071184324 conn(0x7f04680fc280 msgr2=0x7f0468103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 shutdown_connections 2024-09-19T05:05:31.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:31.068+0000 7f046d475640 1 -- 172.21.15.64:0/3071184324 wait complete. 2024-09-19T05:05:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:31 smithi064 ceph-mon[30960]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:31.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T05:05:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:31 smithi103 ceph-mon[34446]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:31 smithi174 ceph-mon[34488]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:32.383 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:32 smithi064 ceph-mon[30960]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:32.383 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:32 smithi064 ceph-mon[30960]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.381+0000 7f99f9bdb640 1 -- 172.21.15.64:0/1326240492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f40ffed0 msgr2=0x7f99f41002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.381+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/1326240492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f40ffed0 0x7f99f41002d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009930 tx=0x7f99e402a420 comp rx=0 tx=0).stop 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 -- 172.21.15.64:0/1326240492 shutdown_connections 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/1326240492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99f4103870 0x7f99f410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/1326240492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99f4102eb0 0x7f99f4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/1326240492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f40ffed0 0x7f99f41002d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 -- 172.21.15.64:0/1326240492 >> 172.21.15.64:0/1326240492 conn(0x7f99f40fba40 msgr2=0x7f99f40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 -- 172.21.15.64:0/1326240492 shutdown_connections 2024-09-19T05:05:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.382+0000 7f99f9bdb640 1 -- 172.21.15.64:0/1326240492 wait complete. 2024-09-19T05:05:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 Processor -- start 2024-09-19T05:05:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 -- start start 2024-09-19T05:05:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99f40ffed0 0x7f99f4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 0x7f99f4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99f4103870 0x7f99f419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410d7c0 con 0x7f99f4102eb0 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410d640 con 0x7f99f40ffed0 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f9bdb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410d940 con 0x7f99f4103870 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 0x7f99f4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 0x7f99f4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55762/0 (socket says 172.21.15.64:55762) 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f2ffd640 1 -- 172.21.15.64:0/4244548963 learned_addr learned my addr 172.21.15.64:0/4244548963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.383+0000 7f99f37fe640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99f40ffed0 0x7f99f4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f3fff640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99f4103870 0x7f99f419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f2ffd640 1 -- 172.21.15.64:0/4244548963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99f4103870 msgr2=0x7f99f419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f2ffd640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99f4103870 0x7f99f419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f2ffd640 1 -- 172.21.15.64:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99f40ffed0 msgr2=0x7f99f4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f2ffd640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99f40ffed0 0x7f99f4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f2ffd640 1 -- 172.21.15.64:0/4244548963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f419fb80 con 0x7f99f4102eb0 2024-09-19T05:05:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f2ffd640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 0x7f99f4198490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b4a0 tx=0x7f99dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc014070 con 0x7f99f4102eb0 2024-09-19T05:05:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99dc010ac0 con 0x7f99f4102eb0 2024-09-19T05:05:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f419fe70 con 0x7f99f4102eb0 2024-09-19T05:05:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc017c30 con 0x7f99f4102eb0 2024-09-19T05:05:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.384+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a0350 con 0x7f99f4102eb0 2024-09-19T05:05:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.386+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f41091f0 con 0x7f99f4102eb0 2024-09-19T05:05:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.387+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f99dc017070 con 0x7f99f4102eb0 2024-09-19T05:05:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.388+0000 7f99f0ff9640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f99e00a3300 0x7f99e00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.388+0000 7f99f37fe640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f99e00a3300 0x7f99e00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.388+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f99dc0ced60 con 0x7f99f4102eb0 2024-09-19T05:05:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.388+0000 7f99f37fe640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f99e00a3300 0x7f99e00a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99e4004770 tx=0x7f99e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.392+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f99dc09ba10 con 0x7f99f4102eb0 2024-09-19T05:05:32.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.575+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f99f4062630 con 0x7f99e00a3300 2024-09-19T05:05:32.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.576+0000 7f99f0ff9640 1 -- 172.21.15.64:0/4244548963 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f99f4062630 con 0x7f99e00a3300 2024-09-19T05:05:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2024-09-19T05:05:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2024-09-19T05:05:32.578 INFO:teuthology.orchestra.run.smithi064.stdout:smithi103 172.21.15.103 2024-09-19T05:05:32.578 INFO:teuthology.orchestra.run.smithi064.stdout:smithi174 172.21.15.174 2024-09-19T05:05:32.578 INFO:teuthology.orchestra.run.smithi064.stdout:3 hosts in cluster 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f99e00a3300 msgr2=0x7f99e00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f99e00a3300 0x7f99e00a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99e4004770 tx=0x7f99e4000bc0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 msgr2=0x7f99f4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 0x7f99f4198490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b4a0 tx=0x7f99dc00b960 comp rx=0 tx=0).stop 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 shutdown_connections 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f99e00a3300 0x7f99e00a57c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99f4103870 0x7f99f419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f4102eb0 0x7f99f4198490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 --2- 172.21.15.64:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99f40ffed0 0x7f99f4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.578+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 >> 172.21.15.64:0/4244548963 conn(0x7f99f40fba40 msgr2=0x7f99f41063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.579+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 shutdown_connections 2024-09-19T05:05:32.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:32.579+0000 7f99f9bdb640 1 -- 172.21.15.64:0/4244548963 wait complete. 2024-09-19T05:05:32.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:32 smithi103 ceph-mon[34446]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:32.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:32 smithi103 ceph-mon[34446]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:32.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:32 smithi174 ceph-mon[34488]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:32.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:32 smithi174 ceph-mon[34488]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:33.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T05:05:33.600 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.038+0000 7fb51ded4640 1 -- 172.21.15.64:0/3206706865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 msgr2=0x7fb5181073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.038+0000 7fb51ded4640 1 --2- 172.21.15.64:0/3206706865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb5181073c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb500009900 tx=0x7fb50002a410 comp rx=0 tx=0).stop 2024-09-19T05:05:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 -- 172.21.15.64:0/3206706865 shutdown_connections 2024-09-19T05:05:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 --2- 172.21.15.64:0/3206706865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb518107900 0x7fb518109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 --2- 172.21.15.64:0/3206706865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb5181073c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 --2- 172.21.15.64:0/3206706865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5181026a0 0x7fb518104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 -- 172.21.15.64:0/3206706865 >> 172.21.15.64:0/3206706865 conn(0x7fb5180fc2a0 msgr2=0x7fb5180fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 -- 172.21.15.64:0/3206706865 shutdown_connections 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 -- 172.21.15.64:0/3206706865 wait complete. 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.039+0000 7fb51ded4640 1 Processor -- start 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 -- start start 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181026a0 0x7fb518102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb518100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb518107900 0x7fb518100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810d800 con 0x7fb518104fd0 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810d680 con 0x7fb5181026a0 2024-09-19T05:05:34.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb51ded4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810d980 con 0x7fb518107900 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb518100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb518100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55778/0 (socket says 172.21.15.64:55778) 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 -- 172.21.15.64:0/1243973042 learned_addr learned my addr 172.21.15.64:0/1243973042 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb5177fe640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181026a0 0x7fb518102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 -- 172.21.15.64:0/1243973042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb518107900 msgr2=0x7fb518100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb518107900 0x7fb518100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 -- 172.21.15.64:0/1243973042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181026a0 msgr2=0x7fb518102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181026a0 0x7fb518102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 -- 172.21.15.64:0/1243973042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb518101450 con 0x7fb518104fd0 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb5177fe640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181026a0 0x7fb518102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.040+0000 7fb516ffd640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb518100650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb5000314e0 tx=0x7fb500003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.041+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50003f070 con 0x7fb518104fd0 2024-09-19T05:05:34.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.041+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5181a2070 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.041+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb500031e40 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.041+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500035030 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.041+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a25f0 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.042+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5180fdd90 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.043+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb500042070 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.044+0000 7fb514ff9640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5040a32d0 0x7fb5040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.044+0000 7fb5177fe640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5040a32d0 0x7fb5040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.044+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fb5000f0560 con 0x7fb518104fd0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.044+0000 7fb5177fe640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5040a32d0 0x7fb5040a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb508009930 tx=0x7fb508008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:34.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.048+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5000bd100 con 0x7fb518104fd0 2024-09-19T05:05:34.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.233+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb518062630 con 0x7fb5040a32d0 2024-09-19T05:05:34.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.235+0000 7fb514ff9640 1 -- 172.21.15.64:0/1243973042 <== mgr.14150 v2:172.21.15.64:6800/3907259856 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb518062630 con 0x7fb5040a32d0 2024-09-19T05:05:34.236 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T05:05:34.236 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T05:05:34.236 INFO:teuthology.orchestra.run.smithi064.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T05:05:34.237 INFO:teuthology.orchestra.run.smithi064.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T05:05:34.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5040a32d0 msgr2=0x7fb5040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5040a32d0 0x7fb5040a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb508009930 tx=0x7fb508008040 comp rx=0 tx=0).stop 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 msgr2=0x7fb518100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb518100650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb5000314e0 tx=0x7fb500003d50 comp rx=0 tx=0).stop 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 shutdown_connections 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fb5040a32d0 0x7fb5040a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb518107900 0x7fb518100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb518104fd0 0x7fb518100650 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 --2- 172.21.15.64:0/1243973042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181026a0 0x7fb518102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 >> 172.21.15.64:0/1243973042 conn(0x7fb5180fc2a0 msgr2=0x7fb518103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 shutdown_connections 2024-09-19T05:05:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:34.237+0000 7fb51ded4640 1 -- 172.21.15.64:0/1243973042 wait complete. 2024-09-19T05:05:34.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:34 smithi064 ceph-mon[30960]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:34.560 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:34 smithi064 ceph-mon[30960]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:34.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:34 smithi103 ceph-mon[34446]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:34.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:34 smithi103 ceph-mon[34446]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:34.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:34 smithi174 ceph-mon[34488]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:34.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:34 smithi174 ceph-mon[34488]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:34.856 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T05:05:34.865 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:05:34.865 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T05:05:35.122 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:35.528 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:35 smithi064 ceph-mon[30960]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.609+0000 7f63a46f4640 1 -- 172.21.15.64:0/473798630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c100ad0 msgr2=0x7f639c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.609+0000 7f63a46f4640 1 --2- 172.21.15.64:0/473798630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c100ad0 0x7f639c100ed0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f638c009ca0 tx=0x7f638c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- 172.21.15.64:0/473798630 shutdown_connections 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 --2- 172.21.15.64:0/473798630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f639c102690 0x7f639c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 --2- 172.21.15.64:0/473798630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c101cd0 0x7f639c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 --2- 172.21.15.64:0/473798630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c100ad0 0x7f639c100ed0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- 172.21.15.64:0/473798630 >> 172.21.15.64:0/473798630 conn(0x7f639c0fc280 msgr2=0x7f639c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- 172.21.15.64:0/473798630 shutdown_connections 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- 172.21.15.64:0/473798630 wait complete. 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 Processor -- start 2024-09-19T05:05:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- start start 2024-09-19T05:05:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c100ad0 0x7f639c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 0x7f639c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f639c102690 0x7f639c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10d770 con 0x7f639c101cd0 2024-09-19T05:05:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10d5f0 con 0x7f639c102690 2024-09-19T05:05:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.610+0000 7f63a46f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10d8f0 con 0x7f639c100ad0 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 0x7f639c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 0x7f639c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57280/0 (socket says 172.21.15.64:57280) 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 -- 172.21.15.64:0/2025256195 learned_addr learned my addr 172.21.15.64:0/2025256195 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a2c91640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f639c102690 0x7f639c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a2490640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c100ad0 0x7f639c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 -- 172.21.15.64:0/2025256195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c100ad0 msgr2=0x7f639c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c100ad0 0x7f639c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 -- 172.21.15.64:0/2025256195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f639c102690 msgr2=0x7f639c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f639c102690 0x7f639c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 -- 172.21.15.64:0/2025256195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c19fb70 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a1c8f640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 0x7f639c198480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f639000b4a0 tx=0x7f639000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390014070 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6390010ac0 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390017c30 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.611+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c19fe60 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.612+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a02c0 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.612+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c107fc0 con 0x7f639c101cd0 2024-09-19T05:05:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.614+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6390017070 con 0x7f639c101cd0 2024-09-19T05:05:35.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.614+0000 7f639b7fe640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f63840a3380 0x7f63840a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.614+0000 7f63a2490640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f63840a3380 0x7f63840a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.614+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f63900cec50 con 0x7f639c101cd0 2024-09-19T05:05:35.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.615+0000 7f63a2490640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f63840a3380 0x7f63840a5840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f638c000f80 tx=0x7f638c031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:35.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.619+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f639009b900 con 0x7f639c101cd0 2024-09-19T05:05:35.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:35 smithi103 ceph-mon[34446]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:35 smithi174 ceph-mon[34488]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:35.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.833+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f639c02d9e0 con 0x7f639c101cd0 2024-09-19T05:05:35.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.834+0000 7f639b7fe640 1 -- 172.21.15.64:0/2025256195 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3087 (secure 0 0 0) 0x7f639009b200 con 0x7f639c101cd0 2024-09-19T05:05:35.835 INFO:teuthology.orchestra.run.smithi064.stdout:e24 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:btime 2024-09-19T05:04:20:028478+0000 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 24 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-09-19T05:04:08.534666+0000 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-09-19T05:04:19.601259+0000 2024-09-19T05:05:35.836 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 300 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 3 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:in 0,1,2 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24479,1=24466,2=24448} 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-09-19T05:05:35.837 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:qdb_cluster leader: 24479 members: 24448,24466,24479 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b{0:24479} state up:active seq 4 addr [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.f{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/2004435266,v1:172.21.15.174:6835/2004435266] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.i{1:24466} state up:active seq 8 addr [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.h{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.103:6836/3232384876,v1:172.21.15.103:6837/3232384876] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c{2:24448} state up:active seq 10 addr [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.e{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.103:6834/642500144,v1:172.21.15.103:6835/642500144] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.64:6834/726782789,v1:172.21.15.64:6835/726782789] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.64:6836/2636818153,v1:172.21.15.64:6837/2636818153] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.g{-1:14580} state up:standby seq 1 addr [v2:172.21.15.64:6838/4284831800,v1:172.21.15.64:6839/4284831800] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a{-1:14592} state up:standby seq 1 addr [v2:172.21.15.64:6840/3250181965,v1:172.21.15.64:6841/3250181965] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.103:6832/550094745,v1:172.21.15.103:6833/550094745] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.839 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.174:6832/252917517,v1:172.21.15.174:6833/252917517] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:05:35.840 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:05:35.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f63840a3380 msgr2=0x7f63840a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f63840a3380 0x7f63840a5840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f638c000f80 tx=0x7f638c031000 comp rx=0 tx=0).stop 2024-09-19T05:05:35.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 msgr2=0x7f639c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 0x7f639c198480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f639000b4a0 tx=0x7f639000b960 comp rx=0 tx=0).stop 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 shutdown_connections 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f63840a3380 0x7f63840a5840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f639c102690 0x7f639c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f639c101cd0 0x7f639c198480 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 --2- 172.21.15.64:0/2025256195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c100ad0 0x7f639c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 >> 172.21.15.64:0/2025256195 conn(0x7f639c0fc280 msgr2=0x7f639c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 shutdown_connections 2024-09-19T05:05:35.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:35.836+0000 7f63a46f4640 1 -- 172.21.15.64:0/2025256195 wait complete. 2024-09-19T05:05:36.534 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:36 smithi064 ceph-mon[30960]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:36.534 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:36 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2025256195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:05:36.567 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T05:05:36.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:36 smithi103 ceph-mon[34446]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:36.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:36 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2025256195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:05:36.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:36 smithi174 ceph-mon[34488]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:36.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:36 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2025256195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:05:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1151484943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 msgr2=0x7f44dc104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1151484943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 0x7f44dc104a90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009900 tx=0x7f44d002a410 comp rx=0 tx=0).stop 2024-09-19T05:05:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1151484943 shutdown_connections 2024-09-19T05:05:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1151484943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc107900 0x7f44dc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1151484943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc104fd0 0x7f44dc1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1151484943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 0x7f44dc104a90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1151484943 >> 172.21.15.64:0/1151484943 conn(0x7f44dc0fc2a0 msgr2=0x7f44dc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1151484943 shutdown_connections 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.337+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1151484943 wait complete. 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 Processor -- start 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 -- start start 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1026a0 0x7f44dc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc107900 0x7f44dc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10d7b0 con 0x7f44dc104fd0 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10d630 con 0x7f44dc107900 2024-09-19T05:05:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e2f9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10d930 con 0x7f44dc1026a0 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57292/0 (socket says 172.21.15.64:57292) 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44dbfff640 1 -- 172.21.15.64:0/1674543243 learned_addr learned my addr 172.21.15.64:0/1674543243 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.338+0000 7f44e1537640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc107900 0x7f44dc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44e0d36640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1026a0 0x7f44dc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44dbfff640 1 -- 172.21.15.64:0/1674543243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1026a0 msgr2=0x7f44dc197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44dbfff640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1026a0 0x7f44dc197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44dbfff640 1 -- 172.21.15.64:0/1674543243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc107900 msgr2=0x7f44dc19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44dbfff640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc107900 0x7f44dc19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44dbfff640 1 -- 172.21.15.64:0/1674543243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc19fa40 con 0x7f44dc104fd0 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44e0d36640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1026a0 0x7f44dc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:37.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44e1537640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc107900 0x7f44dc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44dbfff640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc198350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f44c800d770 tx=0x7f44c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8017070 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc19fd30 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c8009c30 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c800f620 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.339+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc1a0210 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.340+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc0fdd90 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.341+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f44c801a070 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.342+0000 7f44d9ffb640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44cc0a3300 0x7f44cc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.342+0000 7f44e0d36640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44cc0a3300 0x7f44cc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.342+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f44c80cf660 con 0x7f44dc104fd0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.343+0000 7f44e0d36640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44cc0a3300 0x7f44cc0a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.346+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44c809c200 con 0x7f44dc104fd0 2024-09-19T05:05:37.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.524+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f44dc02d9c0 con 0x7f44dc104fd0 2024-09-19T05:05:37.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.524+0000 7f44d9ffb640 1 -- 172.21.15.64:0/1674543243 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) ==== 56+0+5338 (secure 0 0 0) 0x7f44c809bb00 con 0x7f44dc104fd0 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 79 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-09-19T04:56:23.797274+0000 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-09-19T05:04:20.028795+0000 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:crush_version 26 2024-09-19T05:05:37.526 INFO:teuthology.orchestra.run.smithi064.stdout:full_ratio 0.95 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:backfillfull_ratio 0.9 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:nearfull_ratio 0.85 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:require_min_compat_client luminous 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:min_compat_client luminous 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:require_osd_release squid 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:stretch_mode_enabled false 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:max_osd 12 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6802/198612144,v1:172.21.15.64:6803/198612144] [v2:172.21.15.64:6804/198612144,v1:172.21.15.64:6805/198612144] exists,up 52ee9cdd-278a-45e4-8b61-5fa0991e9841 2024-09-19T05:05:37.527 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6800/1592456147,v1:172.21.15.103:6801/1592456147] [v2:172.21.15.103:6802/1592456147,v1:172.21.15.103:6803/1592456147] exists,up f802a0ea-cb17-4a5e-85c1-ab9764fd2e94 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 up in weight 1 up_from 17 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/4215230653,v1:172.21.15.174:6801/4215230653] [v2:172.21.15.174:6802/4215230653,v1:172.21.15.174:6803/4215230653] exists,up 0be6c1fd-a08a-4e73-904a-fee6cb38adee 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 up in weight 1 up_from 23 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6810/973141608,v1:172.21.15.64:6811/973141608] [v2:172.21.15.64:6812/973141608,v1:172.21.15.64:6813/973141608] exists,up 62258a68-87a3-4c2e-bc80-49a435c5df82 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 up in weight 1 up_from 30 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6808/2896874846,v1:172.21.15.103:6809/2896874846] [v2:172.21.15.103:6810/2896874846,v1:172.21.15.103:6811/2896874846] exists,up 9d7e9b07-2d48-4029-8396-fdbae9a82494 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 up in weight 1 up_from 34 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/3607820315,v1:172.21.15.174:6809/3607820315] [v2:172.21.15.174:6810/3607820315,v1:172.21.15.174:6811/3607820315] exists,up 93251354-cc4c-4dcf-9536-69770ad07286 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 up in weight 1 up_from 41 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6818/3289767656,v1:172.21.15.64:6819/3289767656] [v2:172.21.15.64:6820/3289767656,v1:172.21.15.64:6821/3289767656] exists,up 346e4af1-6d30-4b90-94c9-e660333548ca 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 up in weight 1 up_from 45 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6816/944387500,v1:172.21.15.103:6817/944387500] [v2:172.21.15.103:6818/944387500,v1:172.21.15.103:6819/944387500] exists,up 29311352-ffd9-4b21-9876-4bb7c8a522b3 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 up in weight 1 up_from 52 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/1095432672,v1:172.21.15.174:6817/1095432672] [v2:172.21.15.174:6818/1095432672,v1:172.21.15.174:6819/1095432672] exists,up ae077168-4b5f-44a0-a269-03f13dba90b1 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 up in weight 1 up_from 59 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6826/1960883919,v1:172.21.15.64:6827/1960883919] [v2:172.21.15.64:6828/1960883919,v1:172.21.15.64:6829/1960883919] exists,up 1767cc43-897a-408c-9e10-ecabe9b39731 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 up in weight 1 up_from 64 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6824/1109323153,v1:172.21.15.103:6825/1109323153] [v2:172.21.15.103:6826/1109323153,v1:172.21.15.103:6827/1109323153] exists,up b507cd6f-ed66-4b6e-9560-fca660e09350 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 up in weight 1 up_from 71 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/3718770744,v1:172.21.15.174:6825/3718770744] [v2:172.21.15.174:6826/3718770744,v1:172.21.15.174:6827/3718770744] exists,up 3169c4e9-d118-46eb-9b6d-b37e181ffa88 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.0 [8,1] 2024-09-19T05:05:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.13 [2,11] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.16 [2,4] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.1d [8,11] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.6 [4,7] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.11 [5,9] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.17 [4,6] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1b [5,9] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.27 [3,9] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.3a [4,8] 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/2584640926 expires 2024-09-20T04:57:18.299954+0000 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1065736541 expires 2024-09-20T04:57:18.299954+0000 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1467175216 expires 2024-09-20T04:57:18.299954+0000 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/4164984314 expires 2024-09-20T04:56:54.761876+0000 2024-09-19T05:05:37.529 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/4164984314 expires 2024-09-20T04:56:54.761876+0000 2024-09-19T05:05:37.530 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/2584640926 expires 2024-09-20T04:57:18.299954+0000 2024-09-19T05:05:37.530 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3048471422 expires 2024-09-20T04:57:18.299954+0000 2024-09-19T05:05:37.530 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/4215744929 expires 2024-09-20T04:56:54.761876+0000 2024-09-19T05:05:37.530 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3045596814 expires 2024-09-20T04:56:54.761876+0000 2024-09-19T05:05:37.530 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/4083716942 expires 2024-09-20T04:56:54.761876+0000 2024-09-19T05:05:37.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44cc0a3300 msgr2=0x7f44cc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44cc0a3300 0x7f44cc0a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d00314a0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 msgr2=0x7f44dc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc198350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f44c800d770 tx=0x7f44c800dc30 comp rx=0 tx=0).stop 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 shutdown_connections 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f44cc0a3300 0x7f44cc0a57c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44dc107900 0x7f44dc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc198350 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 --2- 172.21.15.64:0/1674543243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1026a0 0x7f44dc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 >> 172.21.15.64:0/1674543243 conn(0x7f44dc0fc2a0 msgr2=0x7f44dc102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 shutdown_connections 2024-09-19T05:05:37.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:37.527+0000 7f44e2f9a640 1 -- 172.21.15.64:0/1674543243 wait complete. 2024-09-19T05:05:38.131 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T05:05:38.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:38 smithi064 ceph-mon[30960]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:38 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1674543243' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:05:38.401 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:38.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:38 smithi103 ceph-mon[34446]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:38.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:38 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1674543243' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:05:38.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:38 smithi174 ceph-mon[34488]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:38.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:38 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1674543243' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:05:38.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 -- 172.21.15.64:0/2231645917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f4102de0 msgr2=0x7fe1f41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/2231645917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f4102de0 0x7fe1f41031c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009ca0 tx=0x7fe1ec02a950 comp rx=0 tx=0).stop 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 -- 172.21.15.64:0/2231645917 shutdown_connections 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/2231645917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f410c180 0x7fe1f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/2231645917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f4103700 0x7fe1f410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/2231645917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f4102de0 0x7fe1f41031c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 -- 172.21.15.64:0/2231645917 >> 172.21.15.64:0/2231645917 conn(0x7fe1f40fc7a0 msgr2=0x7fe1f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:38.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 -- 172.21.15.64:0/2231645917 shutdown_connections 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.893+0000 7fe1f99f2640 1 -- 172.21.15.64:0/2231645917 wait complete. 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 Processor -- start 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 -- start start 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f4102de0 0x7fe1f41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f4103700 0x7fe1f4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 0x7fe1f419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4111e20 con 0x7fe1f410c180 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4111ca0 con 0x7fe1f4102de0 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f99f2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4111fa0 con 0x7fe1f4103700 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.894+0000 7fe1f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 0x7fe1f419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 0x7fe1f419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57312/0 (socket says 172.21.15.64:57312) 2024-09-19T05:05:38.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 -- 172.21.15.64:0/1875181462 learned_addr learned my addr 172.21.15.64:0/1875181462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f2ffd640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f4102de0 0x7fe1f41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f27fc640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f4103700 0x7fe1f4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 -- 172.21.15.64:0/1875181462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f4103700 msgr2=0x7fe1f4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f4103700 0x7fe1f4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 -- 172.21.15.64:0/1875181462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f4102de0 msgr2=0x7fe1f41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f4102de0 0x7fe1f41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 -- 172.21.15.64:0/1875181462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f419d370 con 0x7fe1f410c180 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f37fe640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 0x7fe1f419cc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c970 tx=0x7fe1e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4007bb0 con 0x7fe1f410c180 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1e4007d50 con 0x7fe1f410c180 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f419d660 con 0x7fe1f410c180 2024-09-19T05:05:38.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4004e10 con 0x7fe1f410c180 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.895+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f41a4f40 con 0x7fe1f410c180 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.896+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f41031c0 con 0x7fe1f410c180 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.897+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe1e4005020 con 0x7fe1f410c180 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.898+0000 7fe1f89f0640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe1e00a32d0 0x7fe1e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.898+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fe1e40cf680 con 0x7fe1f410c180 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.898+0000 7fe1f2ffd640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe1e00a32d0 0x7fe1e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.899+0000 7fe1f2ffd640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe1e00a32d0 0x7fe1e00a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec004ad0 tx=0x7fe1ec004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:38.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:38.902+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe1e4014030 con 0x7fe1f410c180 2024-09-19T05:05:39.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.115+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f402d7c0 con 0x7fe1f410c180 2024-09-19T05:05:39.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.116+0000 7fe1f89f0640 1 -- 172.21.15.64:0/1875181462 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7fe1e409bbd0 con 0x7fe1f410c180 2024-09-19T05:05:39.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:39.118 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-19T05:04:20:028478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:04:19.601259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24448,24466,24479]},"id":1}]} 2024-09-19T05:05:39.118 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe1e00a32d0 msgr2=0x7fe1e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe1e00a32d0 0x7fe1e00a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec004ad0 tx=0x7fe1ec004a60 comp rx=0 tx=0).stop 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 msgr2=0x7fe1f419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 0x7fe1f419cc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c970 tx=0x7fe1e400ce30 comp rx=0 tx=0).stop 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 shutdown_connections 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fe1e00a32d0 0x7fe1e00a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1f410c180 0x7fe1f419cc90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f4103700 0x7fe1f4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 --2- 172.21.15.64:0/1875181462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f4102de0 0x7fe1f41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:39.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.118+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 >> 172.21.15.64:0/1875181462 conn(0x7fe1f40fc7a0 msgr2=0x7fe1f40fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.119+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 shutdown_connections 2024-09-19T05:05:39.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:39.119+0000 7fe1f99f2640 1 -- 172.21.15.64:0/1875181462 wait complete. 2024-09-19T05:05:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:39 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1875181462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:39.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:39.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:39 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1875181462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:39 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1875181462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:40.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:40 smithi064 ceph-mon[30960]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 -- 172.21.15.64:0/1872526408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8058101280 msgr2=0x7f805810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 --2- 172.21.15.64:0/1872526408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8058101280 0x7f805810df40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8048009900 tx=0x7f804802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 -- 172.21.15.64:0/1872526408 shutdown_connections 2024-09-19T05:05:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 --2- 172.21.15.64:0/1872526408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f805810e480 0x7f8058110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 --2- 172.21.15.64:0/1872526408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8058101280 0x7f805810df40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 --2- 172.21.15.64:0/1872526408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80581008d0 0x7f8058100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.442+0000 7f80601cf640 1 -- 172.21.15.64:0/1872526408 >> 172.21.15.64:0/1872526408 conn(0x7f80580fc7a0 msgr2=0x7f80580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 -- 172.21.15.64:0/1872526408 shutdown_connections 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 -- 172.21.15.64:0/1872526408 wait complete. 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 Processor -- start 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 -- start start 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 0x7f805819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8058101280 0x7f805819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f805810e480 0x7f805819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8058114050 con 0x7f80581008d0 2024-09-19T05:05:40.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8058113ed0 con 0x7f805810e480 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f80601cf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80581141d0 con 0x7f8058101280 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.443+0000 7f805df6b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 0x7f805819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 0x7f805819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57324/0 (socket says 172.21.15.64:57324) 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 -- 172.21.15.64:0/1378946042 learned_addr learned my addr 172.21.15.64:0/1378946042 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805e76c640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f805810e480 0x7f805819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805d76a640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8058101280 0x7f805819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 -- 172.21.15.64:0/1378946042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8058101280 msgr2=0x7f805819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8058101280 0x7f805819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 -- 172.21.15.64:0/1378946042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f805810e480 msgr2=0x7f805819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f805810e480 0x7f805819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 -- 172.21.15.64:0/1378946042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805819f580 con 0x7f80581008d0 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805d76a640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8058101280 0x7f805819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f805df6b640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 0x7f805819a5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f805400c920 tx=0x7f805400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054007c70 con 0x7f80581008d0 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8054007e10 con 0x7f80581008d0 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805819f870 con 0x7f80581008d0 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054018820 con 0x7f80581008d0 2024-09-19T05:05:40.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.444+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80581a7150 con 0x7f80581008d0 2024-09-19T05:05:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.445+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8058100cb0 con 0x7f80581008d0 2024-09-19T05:05:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.447+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f80540189c0 con 0x7f80581008d0 2024-09-19T05:05:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.447+0000 7f804effd640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f80440a32d0 0x7f80440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.447+0000 7f805d76a640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f80440a32d0 0x7f80440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.448+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f80540cff70 con 0x7f80581008d0 2024-09-19T05:05:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.448+0000 7f805d76a640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f80440a32d0 0x7f80440a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8048031440 tx=0x7f8048005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.451+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8054014030 con 0x7f80581008d0 2024-09-19T05:05:40.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.662+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8058109670 con 0x7f80581008d0 2024-09-19T05:05:40.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.663+0000 7f804effd640 1 -- 172.21.15.64:0/1378946042 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f805409cba0 con 0x7f80581008d0 2024-09-19T05:05:40.664 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:40.665 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-19T05:04:20:028478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:04:19.601259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24448,24466,24479]},"id":1}]} 2024-09-19T05:05:40.665 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f80440a32d0 msgr2=0x7f80440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f80440a32d0 0x7f80440a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8048031440 tx=0x7f8048005a20 comp rx=0 tx=0).stop 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 msgr2=0x7f805819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 0x7f805819a5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f805400c920 tx=0x7f805400cde0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 shutdown_connections 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f80440a32d0 0x7f80440a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f805810e480 0x7f805819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8058101280 0x7f805819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 --2- 172.21.15.64:0/1378946042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80581008d0 0x7f805819a5d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 >> 172.21.15.64:0/1378946042 conn(0x7f80580fc7a0 msgr2=0x7f80580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 shutdown_connections 2024-09-19T05:05:40.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:40.665+0000 7f80601cf640 1 -- 172.21.15.64:0/1378946042 wait complete. 2024-09-19T05:05:40.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:40 smithi103 ceph-mon[34446]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:40.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:40 smithi174 ceph-mon[34488]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:41.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:05:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:41 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1378946042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:41.486 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:41.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:41 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1378946042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:41 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1378946042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 -- 172.21.15.64:0/1604373892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102570 msgr2=0x7f52f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 --2- 172.21.15.64:0/1604373892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102570 0x7f52f01029b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009900 tx=0x7f52d802a480 comp rx=0 tx=0).stop 2024-09-19T05:05:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 -- 172.21.15.64:0/1604373892 shutdown_connections 2024-09-19T05:05:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 --2- 172.21.15.64:0/1604373892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102ef0 0x7f52f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:41.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 --2- 172.21.15.64:0/1604373892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102570 0x7f52f01029b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:41.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 --2- 172.21.15.64:0/1604373892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f0100e20 0x7f52f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:41.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 -- 172.21.15.64:0/1604373892 >> 172.21.15.64:0/1604373892 conn(0x7f52f00fc7d0 msgr2=0x7f52f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:41.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 -- 172.21.15.64:0/1604373892 shutdown_connections 2024-09-19T05:05:41.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.907+0000 7f52f51c2640 1 -- 172.21.15.64:0/1604373892 wait complete. 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.908+0000 7f52f51c2640 1 Processor -- start 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 -- start start 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f0100e20 0x7f52f01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102570 0x7f52f01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 0x7f52f01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0110ea0 con 0x7f52f0102ef0 2024-09-19T05:05:41.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0110d20 con 0x7f52f0100e20 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52f51c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0111020 con 0x7f52f0102570 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 0x7f52f01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 0x7f52f01a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57338/0 (socket says 172.21.15.64:57338) 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 -- 172.21.15.64:0/64223131 learned_addr learned my addr 172.21.15.64:0/64223131 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52eed9d640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f0100e20 0x7f52f01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ee59c640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102570 0x7f52f01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 -- 172.21.15.64:0/64223131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102570 msgr2=0x7f52f01997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102570 0x7f52f01997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 -- 172.21.15.64:0/64223131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f0100e20 msgr2=0x7f52f01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:41.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f0100e20 0x7f52f01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ef59e640 1 -- 172.21.15.64:0/64223131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f01a4870 con 0x7f52f0102ef0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.909+0000 7f52ee59c640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102570 0x7f52f01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.910+0000 7f52ef59e640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 0x7f52f01a4330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f52e800b970 tx=0x7f52e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.910+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8019070 con 0x7f52f0102ef0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.910+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52e8014440 con 0x7f52f0102ef0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.910+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f01a4a40 con 0x7f52f0102ef0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.910+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e80075d0 con 0x7f52f0102ef0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.910+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f01a4f00 con 0x7f52f0102ef0 2024-09-19T05:05:41.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.911+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f0101200 con 0x7f52f0102ef0 2024-09-19T05:05:41.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.912+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f52e801c070 con 0x7f52f0102ef0 2024-09-19T05:05:41.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.913+0000 7f52e7fff640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f52dc0a3350 0x7f52dc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:41.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.913+0000 7f52eed9d640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f52dc0a3350 0x7f52dc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:41.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.913+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f52e80cf340 con 0x7f52f0102ef0 2024-09-19T05:05:41.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.914+0000 7f52eed9d640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f52dc0a3350 0x7f52dc0a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52e000bbe0 tx=0x7f52e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:41.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:41.916+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f52e809b9a0 con 0x7f52f0102ef0 2024-09-19T05:05:42.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.096+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52f006a560 con 0x7f52f0102ef0 2024-09-19T05:05:42.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.097+0000 7f52e7fff640 1 -- 172.21.15.64:0/64223131 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22821 (secure 0 0 0) 0x7f52f006a560 con 0x7f52f0102ef0 2024-09-19T05:05:42.099 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:42.099 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:04:20.028795+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f52dc0a3350 msgr2=0x7f52dc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f52dc0a3350 0x7f52dc0a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52e000bbe0 tx=0x7f52e000b390 comp rx=0 tx=0).stop 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 msgr2=0x7f52f01a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 0x7f52f01a4330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f52e800b970 tx=0x7f52e800be30 comp rx=0 tx=0).stop 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 shutdown_connections 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f52dc0a3350 0x7f52dc0a5810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52f0102ef0 0x7f52f01a4330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f0102570 0x7f52f01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 --2- 172.21.15.64:0/64223131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f0100e20 0x7f52f01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.099+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 >> 172.21.15.64:0/64223131 conn(0x7f52f00fc7d0 msgr2=0x7f52f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.100+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 shutdown_connections 2024-09-19T05:05:42.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:42.100+0000 7f52f51c2640 1 -- 172.21.15.64:0/64223131 wait complete. 2024-09-19T05:05:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:42 smithi064 ceph-mon[30960]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:42.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:42 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/64223131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:42.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:42 smithi103 ceph-mon[34446]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:42.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:42 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/64223131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:42.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:04:08.534666+0000', 'modified': '2024-09-19T05:04:19.601259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24466, 'mds_2': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6835/642500144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 642500144}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 642500144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/3232384876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 3232384876}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 3232384876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/2004435266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 2004435266}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 2004435266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1599126810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1599126810}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1599126810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/3328031427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3328031427}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3328031427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.103:6839/100524955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 100524955}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 100524955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24448, 24466, 24479]} 2024-09-19T05:05:42.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T05:05:42.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:05:42.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:42 smithi174 ceph-mon[34488]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:05:42.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:42 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/64223131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:43.000 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:43.393 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:43 smithi064 ceph-mon[30960]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2845203964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc102d40 msgr2=0x7f9ebc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2845203964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc102d40 0x7f9ebc103120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009900 tx=0x7f9eb802a480 comp rx=0 tx=0).stop 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2845203964 shutdown_connections 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2845203964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ebc10c170 0x7f9ebc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2845203964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ebc103660 0x7f9ebc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2845203964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc102d40 0x7f9ebc103120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2845203964 >> 172.21.15.64:0/2845203964 conn(0x7f9ebc0fc7a0 msgr2=0x7f9ebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2845203964 shutdown_connections 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.473+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2845203964 wait complete. 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 Processor -- start 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 -- start start 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ebc102d40 0x7f9ebc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 0x7f9ebc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ebc10c170 0x7f9ebc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc111eb0 con 0x7f9ebc103660 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc111d30 con 0x7f9ebc102d40 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec45ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc112030 con 0x7f9ebc10c170 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 0x7f9ebc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 0x7f9ebc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57354/0 (socket says 172.21.15.64:57354) 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec238a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ebc102d40 0x7f9ebc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 -- 172.21.15.64:0/2525722654 learned_addr learned my addr 172.21.15.64:0/2525722654 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ebc10c170 0x7f9ebc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 -- 172.21.15.64:0/2525722654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ebc10c170 msgr2=0x7f9ebc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ebc10c170 0x7f9ebc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 -- 172.21.15.64:0/2525722654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ebc102d40 msgr2=0x7f9ebc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.474+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ebc102d40 0x7f9ebc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9ec1b89640 1 -- 172.21.15.64:0/2525722654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc1a68b0 con 0x7f9ebc103660 2024-09-19T05:05:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 0x7f9ebc100ee0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00d740 tx=0x7f9eac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9eb37fe640 1 -- 172.21.15.64:0/2525722654 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac017070 con 0x7f9ebc103660 2024-09-19T05:05:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9eb37fe640 1 -- 172.21.15.64:0/2525722654 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eac00de50 con 0x7f9ebc103660 2024-09-19T05:05:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1a6ba0 con 0x7f9ebc103660 2024-09-19T05:05:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9eb37fe640 1 -- 172.21.15.64:0/2525722654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00f620 con 0x7f9ebc103660 2024-09-19T05:05:43.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.475+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a7130 con 0x7f9ebc103660 2024-09-19T05:05:43.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.477+0000 7f9eb37fe640 1 -- 172.21.15.64:0/2525722654 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9eac009c30 con 0x7f9ebc103660 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 Processor -- start 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 -- start start 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d29d0 0x55d2490d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490ddbb0 0x55d2490dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d249039090 con 0x55d2490d32f0 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d248fe0a70 con 0x55d2490ddbb0 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d249035050 con 0x55d2490d29d0 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec238a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9eb37fe640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9ea80a3350 0x7f9ea80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec238a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490dd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57358/0 (socket says 172.21.15.64:57358) 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec238a640 1 -- 172.21.15.64:0/3373767300 learned_addr learned my addr 172.21.15.64:0/3373767300 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d29d0 0x55d2490d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490ddbb0 0x55d2490dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.478+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9ea80a3350 0x7f9ea80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9eb37fe640 1 -- 172.21.15.64:0/2525722654 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f9eac0cf5f0 con 0x7f9ebc103660 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec238a640 1 -- 172.21.15.64:0/3373767300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d29d0 msgr2=0x55d2490d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec238a640 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d29d0 0x55d2490d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec238a640 1 -- 172.21.15.64:0/3373767300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490ddbb0 msgr2=0x55d2490dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec238a640 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490ddbb0 0x55d2490dffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec238a640 1 -- 172.21.15.64:0/3373767300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d248fed130 con 0x55d2490d32f0 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec238a640 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490dd670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009230 tx=0x7f9eb8004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a09358b9d19642 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:43.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9ea80a3350 0x7f9ea80a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40070b0 tx=0x7f9eb40093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:43.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9eb0ff9640 1 -- 172.21.15.64:0/3373767300 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8035040 con 0x55d2490d32f0 2024-09-19T05:05:43.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9eb0ff9640 1 -- 172.21.15.64:0/3373767300 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eb8004770 con 0x55d2490d32f0 2024-09-19T05:05:43.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.479+0000 7f9eb0ff9640 1 -- 172.21.15.64:0/3373767300 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8003810 con 0x55d2490d32f0 2024-09-19T05:05:43.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 -- 172.21.15.64:0/3373767300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 msgr2=0x55d2490dd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490dd670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009230 tx=0x7f9eb8004310 comp rx=0 tx=0).stop 2024-09-19T05:05:43.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 -- 172.21.15.64:0/3373767300 shutdown_connections 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490ddbb0 0x55d2490dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490dd670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/3373767300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d29d0 0x55d2490d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 -- 172.21.15.64:0/3373767300 >> 172.21.15.64:0/3373767300 conn(0x55d2490cef60 msgr2=0x55d2490cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 -- 172.21.15.64:0/3373767300 shutdown_connections 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 -- 172.21.15.64:0/3373767300 wait complete. 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 -- start start 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.480+0000 7f9ec69c7740 1 Processor -- start 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490dd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490dac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d2490e0780 con 0x55d2490d32f0 2024-09-19T05:05:43.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d248ff3f20 con 0x55d2490d29d0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2490e0900 con 0x55d2490ddbb0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec69c7740 10 client.0 _ll_register_callbacks cb 0x55d248fbd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490da6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490da6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57362/0 (socket says 172.21.15.64:57362) 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 -- 172.21.15.64:0/368323744 learned_addr learned my addr 172.21.15.64:0/368323744 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490dac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec238a640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 -- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 msgr2=0x55d2490dac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490dac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 -- 172.21.15.64:0/368323744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 msgr2=0x55d2490dd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490dd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 -- 172.21.15.64:0/368323744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d248daa2b0 con 0x55d2490d32f0 2024-09-19T05:05:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec238a640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490dd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490da6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cd500 tx=0x7f9eac018460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986b308768d3c58e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.481+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/368323744 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac020040 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/368323744 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f9eac0860d0 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb8004770 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/368323744 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9eac09ba90 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 10 client.15063 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2490ec030 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/368323744 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eb8004770 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/368323744 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9eac013050 con 0x55d2490d32f0 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 10 client.15063 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:05:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 20 client.15063 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d2490ee5b0 0x55d2490efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 4 client.15063 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.482+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d249046600 con 0x55d2490ee5b0 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.483+0000 7f9ec238a640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d2490ee5b0 0x55d2490efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.483+0000 7f9ea5ffb640 10 client.15063 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.483+0000 7f9ec238a640 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d2490ee5b0 0x55d2490efa20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ee97f674bb221f6 server_cookie=590afa5a5b4a0f98 in_seq=0 out_seq=0 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/368323744 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d249046600 con 0x55d2490ee5b0 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ea5ffb640 10 client.15063 handle_command_reply: tid=0 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 client.15063 shutdown 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 msgr2=0x55d2490da6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490da6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cd500 tx=0x7f9eac018460 comp rx=0 tx=0).stop 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 shutdown_connections 2024-09-19T05:05:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d2490ee5b0 0x55d2490efa20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490dac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490da6e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/368323744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490dd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ea5ffb640 0 client.15063 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 >> 172.21.15.64:0/368323744 conn(0x55d2490cef60 msgr2=0x55d2490d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 shutdown_connections 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 1 -- 172.21.15.64:0/368323744 wait complete. 2024-09-19T05:05:43.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.484+0000 7f9ec69c7740 20 client.15063 trim_cache size 0 max 16384 2024-09-19T05:05:43.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 Processor -- start 2024-09-19T05:05:43.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 -- start start 2024-09-19T05:05:43.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d32f0 0x55d2490efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 0x55d2490c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d2490edd60 con 0x55d2490ddbb0 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d2490ee430 con 0x55d2490d29d0 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2490eeb00 con 0x55d2490d32f0 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec2b8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 0x55d2490c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec2b8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 0x55d2490c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57370/0 (socket says 172.21.15.64:57370) 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.502+0000 7f9ec2b8b640 1 -- 172.21.15.64:0/1457553914 learned_addr learned my addr 172.21.15.64:0/1457553914 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec238a640 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d32f0 0x55d2490efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec2b8b640 1 -- 172.21.15.64:0/1457553914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d32f0 msgr2=0x55d2490efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d32f0 0x55d2490efa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec2b8b640 1 -- 172.21.15.64:0/1457553914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 msgr2=0x55d2490edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec2b8b640 1 -- 172.21.15.64:0/1457553914 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2490dd700 con 0x55d2490ddbb0 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec238a640 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 0x55d2490c9d50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400aa40 tx=0x7f9eb400c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7301879c06d7c54e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ea7fff640 1 -- 172.21.15.64:0/1457553914 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4013040 con 0x55d2490ddbb0 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ea7fff640 1 -- 172.21.15.64:0/1457553914 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eb402a070 con 0x55d2490ddbb0 2024-09-19T05:05:43.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.503+0000 7f9ea7fff640 1 -- 172.21.15.64:0/1457553914 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4025a60 con 0x55d2490ddbb0 2024-09-19T05:05:43.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 -- 172.21.15.64:0/1457553914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 msgr2=0x55d2490c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 0x55d2490c9d50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400aa40 tx=0x7f9eb400c070 comp rx=0 tx=0).stop 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 -- 172.21.15.64:0/1457553914 shutdown_connections 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490ddbb0 0x55d2490c9d50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490d32f0 0x55d2490efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/1457553914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 -- 172.21.15.64:0/1457553914 >> 172.21.15.64:0/1457553914 conn(0x55d2490cef60 msgr2=0x55d2490cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 -- 172.21.15.64:0/1457553914 shutdown_connections 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 -- 172.21.15.64:0/1457553914 wait complete. 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 -- start start 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.504+0000 7f9ec69c7740 1 Processor -- start 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490f8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d2490ef8a0 con 0x55d2490d32f0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d2490ef1d0 con 0x55d2490d29d0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2490eff70 con 0x55d2490ddbb0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 10 client.0 _ll_register_callbacks cb 0x7f9eac09b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57376/0 (socket says 172.21.15.64:57376) 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 -- 172.21.15.64:0/485604565 learned_addr learned my addr 172.21.15.64:0/485604565 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec238a640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 -- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 msgr2=0x55d2490f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 -- 172.21.15.64:0/485604565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 msgr2=0x55d2490f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:43.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 -- 172.21.15.64:0/485604565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2490f1c70 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec2b8b640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec1b89640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490f8570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0963d0 tx=0x7f9eac0962a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b5e6bec7a9ccad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/485604565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0b9060 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/485604565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f9eac016db0 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.505+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb402a070 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/485604565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9eac014030 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 10 client.15069 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2490f6750 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/485604565 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eb402a070 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/485604565 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9eac0ba250 con 0x55d2490d32f0 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 10 client.15069 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:43.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 20 client.15069 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d24915d230 0x55d24915bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 4 client.15069 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "status"}) -- 0x55d249040b10 con 0x55d24915d230 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.506+0000 7f9ec238a640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d24915d230 0x55d24915bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec238a640 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d24915d230 0x55d24915bf40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54abaf1b543d0de6 server_cookie=c5ebe01769de28cd in_seq=0 out_seq=0 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ea5ffb640 10 client.15069 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ea5ffb640 1 -- 172.21.15.64:0/485604565 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d249040b10 con 0x55d24915d230 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ea5ffb640 10 client.15069 handle_command_reply: tid=0 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 client.15069 shutdown 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 msgr2=0x55d2490f8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490f8570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0963d0 tx=0x7f9eac0962a0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 shutdown_connections 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d24915d230 0x55d24915bf40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2490ddbb0 0x55d2490f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2490d32f0 0x55d2490f8570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 --2- 172.21.15.64:0/485604565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2490d29d0 0x55d2490f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 >> 172.21.15.64:0/485604565 conn(0x55d2490cef60 msgr2=0x55d2490c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ea5ffb640 0 client.15069 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 shutdown_connections 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 1 -- 172.21.15.64:0/485604565 wait complete. 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.507+0000 7f9ec69c7740 20 client.15069 trim_cache size 0 max 16384 2024-09-19T05:05:43.514 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24448, 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 86.497074612000006, 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 78, 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 118.984113838 2024-09-19T05:05:43.515 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9ea80a3350 msgr2=0x7f9ea80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9ea80a3350 0x7f9ea80a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40070b0 tx=0x7f9eb40093d0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 msgr2=0x7f9ebc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 0x7f9ebc100ee0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00d740 tx=0x7f9eac00dc00 comp rx=0 tx=0).stop 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 shutdown_connections 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f9ea80a3350 0x7f9ea80a5810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ebc10c170 0x7f9ebc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc103660 0x7f9ebc100ee0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 --2- 172.21.15.64:0/2525722654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ebc102d40 0x7f9ebc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 >> 172.21.15.64:0/2525722654 conn(0x7f9ebc0fc7a0 msgr2=0x7f9ebc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 shutdown_connections 2024-09-19T05:05:43.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:43.508+0000 7f9ec45ee640 1 -- 172.21.15.64:0/2525722654 wait complete. 2024-09-19T05:05:43.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:43 smithi103 ceph-mon[34446]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:43 smithi174 ceph-mon[34488]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:05:44.496 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:05:44.754 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1575071002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94106a80 msgr2=0x7f7f94106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1575071002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94106a80 0x7f7f94106e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009930 tx=0x7f7f7c02a420 comp rx=0 tx=0).stop 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1575071002 shutdown_connections 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1575071002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f94101420 0x7f7f9410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1575071002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f94100a80 0x7f7f94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1575071002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94106a80 0x7f7f94106e60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1575071002 >> 172.21.15.64:0/1575071002 conn(0x7f7f940fc7a0 msgr2=0x7f7f940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1575071002 shutdown_connections 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.211+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1575071002 wait complete. 2024-09-19T05:05:45.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 Processor -- start 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 -- start start 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 0x7f7f94199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f94101420 0x7f7f94199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f94106a80 0x7f7f941a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94110d30 con 0x7f7f94100a80 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94110bb0 con 0x7f7f94106a80 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f98f3f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94110eb0 con 0x7f7f94101420 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 0x7f7f94199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 0x7f7f94199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46286/0 (socket says 172.21.15.64:46286) 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 -- 172.21.15.64:0/1111083167 learned_addr learned my addr 172.21.15.64:0/1111083167 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:45.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f94106a80 0x7f7f941a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 -- 172.21.15.64:0/1111083167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f94101420 msgr2=0x7f7f94199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f94101420 0x7f7f94199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 -- 172.21.15.64:0/1111083167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f94106a80 msgr2=0x7f7f941a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f94106a80 0x7f7f941a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.212+0000 7f7f9259c640 1 -- 172.21.15.64:0/1111083167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f941a4700 con 0x7f7f94100a80 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.213+0000 7f7f9259c640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 0x7f7f94199140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c004770 tx=0x7f7f7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.213+0000 7f7f8b7fe640 1 -- 172.21.15.64:0/1111083167 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c03f070 con 0x7f7f94100a80 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.213+0000 7f7f8b7fe640 1 -- 172.21.15.64:0/1111083167 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f7c03b460 con 0x7f7f94100a80 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.213+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f941a48d0 con 0x7f7f94100a80 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.213+0000 7f7f8b7fe640 1 -- 172.21.15.64:0/1111083167 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c035070 con 0x7f7f94100a80 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.213+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f941a4db0 con 0x7f7f94100a80 2024-09-19T05:05:45.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.215+0000 7f7f8b7fe640 1 -- 172.21.15.64:0/1111083167 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7f7c03b610 con 0x7f7f94100a80 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.215+0000 7f7f9b318740 1 Processor -- start 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 -- start start 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 0x557e22282c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e22291a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e22291fc0 0x557e222943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e221e9090 con 0x557e222828b0 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e22190a70 con 0x557e222831d0 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9b318740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e221e5050 con 0x557e22291fc0 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 0x557e22282c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 0x557e22282c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46290/0 (socket says 172.21.15.64:46290) 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f8b7fe640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7f800a3350 0x7f7f800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 -- 172.21.15.64:0/176047953 learned_addr learned my addr 172.21.15.64:0/176047953 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e22291fc0 0x557e222943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7f800a3350 0x7f7f800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f8b7fe640 1 -- 172.21.15.64:0/1111083167 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f7f7c0efe80 con 0x7f7f94100a80 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f9259c640 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e22291a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 -- 172.21.15.64:0/176047953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e22291fc0 msgr2=0x557e222943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e22291fc0 0x557e222943b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 -- 172.21.15.64:0/176047953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 msgr2=0x557e22291a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e22291a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.216+0000 7f7f92d9d640 1 -- 172.21.15.64:0/176047953 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e2219d130 con 0x557e222828b0 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.217+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 0x557e22282c90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c005940 tx=0x7f7f8c014b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dbb255f3c6f9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:45.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.217+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7f800a3350 0x7f7f800a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c019000 tx=0x7f7f8c01b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:45.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.217+0000 7f7f88ff9640 1 -- 172.21.15.64:0/176047953 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00f9a0 con 0x557e222828b0 2024-09-19T05:05:45.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.217+0000 7f7f88ff9640 1 -- 172.21.15.64:0/176047953 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f8c00e430 con 0x557e222828b0 2024-09-19T05:05:45.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.217+0000 7f7f88ff9640 1 -- 172.21.15.64:0/176047953 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c01c660 con 0x557e222828b0 2024-09-19T05:05:45.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 -- 172.21.15.64:0/176047953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 msgr2=0x557e22282c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 0x557e22282c90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c005940 tx=0x7f7f8c014b00 comp rx=0 tx=0).stop 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 -- 172.21.15.64:0/176047953 shutdown_connections 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e22291fc0 0x557e222943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e22291a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 --2- 172.21.15.64:0/176047953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222828b0 0x557e22282c90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 -- 172.21.15.64:0/176047953 >> 172.21.15.64:0/176047953 conn(0x557e2227ef60 msgr2=0x557e2227f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 -- 172.21.15.64:0/176047953 shutdown_connections 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 -- 172.21.15.64:0/176047953 wait complete. 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 -- start start 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.218+0000 7f7f9b318740 1 Processor -- start 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e22291a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e2228ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2228f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e22294b90 con 0x557e222831d0 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e221a3f20 con 0x557e22291fc0 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e22294d10 con 0x557e222828b0 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9b318740 10 client.0 _ll_register_callbacks cb 0x557e2216d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e2228ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e2228ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46306/0 (socket says 172.21.15.64:46306) 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 -- 172.21.15.64:0/1582975534 learned_addr learned my addr 172.21.15.64:0/1582975534 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f9259c640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e22291a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2228f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 -- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 msgr2=0x557e22291a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e22291a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 -- 172.21.15.64:0/1582975534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 msgr2=0x557e2228f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2228f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 -- 172.21.15.64:0/1582975534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e21f5a2b0 con 0x557e222831d0 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e2228ecd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7f84011660 tx=0x7f7f84011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b1abae6637496a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.219+0000 7f7f79ffb640 1 -- 172.21.15.64:0/1582975534 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84004d20 con 0x557e222831d0 2024-09-19T05:05:45.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f8c00e430 con 0x557e222831d0 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f79ffb640 1 -- 172.21.15.64:0/1582975534 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f7f84003970 con 0x557e222831d0 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f79ffb640 1 -- 172.21.15.64:0/1582975534 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f84020070 con 0x557e222831d0 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e222a0680 con 0x557e222831d0 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f79ffb640 1 -- 172.21.15.64:0/1582975534 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f8c00e430 con 0x557e222831d0 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f79ffb640 1 -- 172.21.15.64:0/1582975534 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f8400f040 con 0x557e222831d0 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:05:45.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 20 client.15081 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e222a2cd0 0x557e222a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 4 client.15081 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.220+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e221f6600 con 0x557e222a2cd0 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.221+0000 7f7f9259c640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e222a2cd0 0x557e222a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.221+0000 7f7f9259c640 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e222a2cd0 0x557e222a4140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d89056ce7631aed server_cookie=941498f5d83c7c8c in_seq=0 out_seq=0 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.221+0000 7f7f79ffb640 10 client.15081 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f79ffb640 1 -- 172.21.15.64:0/1582975534 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e221f6600 con 0x557e222a2cd0 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f79ffb640 10 client.15081 handle_command_reply: tid=0 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 client.15081 shutdown 2024-09-19T05:05:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 msgr2=0x557e2228ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e2228ecd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7f84011660 tx=0x7f7f84011b20 comp rx=0 tx=0).stop 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 shutdown_connections 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2228f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e222a2cd0 0x557e222a4140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e2228ecd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 --2- 172.21.15.64:0/1582975534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e22291a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 >> 172.21.15.64:0/1582975534 conn(0x557e2227ef60 msgr2=0x557e22284cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f79ffb640 0 client.15081 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 shutdown_connections 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 1 -- 172.21.15.64:0/1582975534 wait complete. 2024-09-19T05:05:45.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.222+0000 7f7f9b318740 20 client.15081 trim_cache size 0 max 16384 2024-09-19T05:05:45.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.240+0000 7f7f9b318740 1 Processor -- start 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.240+0000 7f7f9b318740 1 -- start start 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.240+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.240+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e222a2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.240+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 0x557e22291a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.240+0000 7f7f9b318740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e222a2480 con 0x557e22291fc0 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f9b318740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e222a2b50 con 0x557e222831d0 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f9b318740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e222a3220 con 0x557e222828b0 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 0x557e22291a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 0x557e22291a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46312/0 (socket says 172.21.15.64:46312) 2024-09-19T05:05:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 -- 172.21.15.64:0/226962222 learned_addr learned my addr 172.21.15.64:0/226962222 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f9259c640 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e222a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 -- 172.21.15.64:0/226962222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 msgr2=0x557e222a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 -- 172.21.15.64:0/226962222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 msgr2=0x557e222a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e222a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 -- 172.21.15.64:0/226962222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e2227e810 con 0x557e22291fc0 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f9259c640 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 0x557e22291a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c01f000 tx=0x7f7f8c00bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3766502d608a61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f7bfff640 1 -- 172.21.15.64:0/226962222 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c033070 con 0x557e22291fc0 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f7bfff640 1 -- 172.21.15.64:0/226962222 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f8c00e8e0 con 0x557e22291fc0 2024-09-19T05:05:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.241+0000 7f7f7bfff640 1 -- 172.21.15.64:0/226962222 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c01caf0 con 0x557e22291fc0 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 -- 172.21.15.64:0/226962222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 msgr2=0x557e22291a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 0x557e22291a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c01f000 tx=0x7f7f8c00bb90 comp rx=0 tx=0).stop 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 -- 172.21.15.64:0/226962222 shutdown_connections 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e22291fc0 0x557e22291a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e222831d0 0x557e222a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 --2- 172.21.15.64:0/226962222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 -- 172.21.15.64:0/226962222 >> 172.21.15.64:0/226962222 conn(0x557e2227ef60 msgr2=0x557e2228d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 -- 172.21.15.64:0/226962222 shutdown_connections 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 -- 172.21.15.64:0/226962222 wait complete. 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.242+0000 7f7f9b318740 1 -- start start 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 Processor -- start 2024-09-19T05:05:45.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e222a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2230b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e222a3fc0 con 0x557e222831d0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e222a38f0 con 0x557e22291fc0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e222a4690 con 0x557e222828b0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9b318740 10 client.0 _ll_register_callbacks cb 0x7f7f840099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e222a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e222a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46328/0 (socket says 172.21.15.64:46328) 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 -- 172.21.15.64:0/2940334651 learned_addr learned my addr 172.21.15.64:0/2940334651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f92d9d640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2230b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9259c640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 -- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 msgr2=0x557e222a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 -- 172.21.15.64:0/2940334651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 msgr2=0x557e2230b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2230b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f91d9b640 1 -- 172.21.15.64:0/2940334651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e2230b910 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.243+0000 7f7f9259c640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f91d9b640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e222a5dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f84000f80 tx=0x7f7f84004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb0d582926334c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f79ffb640 1 -- 172.21.15.64:0/2940334651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84005bd0 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f79ffb640 1 -- 172.21.15.64:0/2940334651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f7f84004b20 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f8c00e8e0 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f79ffb640 1 -- 172.21.15.64:0/2940334651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f84011e60 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 10 client.15087 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e2230cac0 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f79ffb640 1 -- 172.21.15.64:0/2940334651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f8c00e8e0 con 0x557e222831d0 2024-09-19T05:05:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f79ffb640 1 -- 172.21.15.64:0/2940334651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f84003c00 con 0x557e222831d0 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 10 client.15087 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 20 client.15087 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e22311710 0x557e22310420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 4 client.15087 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.244+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "status"}) -- 0x557e221f0b10 con 0x557e22311710 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f9259c640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e22311710 0x557e22310420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f79ffb640 10 client.15087 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f9259c640 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e22311710 0x557e22310420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8ec9348a3c40b0b server_cookie=b67bbd9d91cf3066 in_seq=0 out_seq=0 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f79ffb640 1 -- 172.21.15.64:0/2940334651 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557e221f0b10 con 0x557e22311710 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f79ffb640 10 client.15087 handle_command_reply: tid=0 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f9b318740 1 client.15087 shutdown 2024-09-19T05:05:45.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 msgr2=0x557e222a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.245+0000 7f7f9b318740 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e222a5dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f84000f80 tx=0x7f7f84004630 comp rx=0 tx=0).stop 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 shutdown_connections 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e22291fc0 0x557e2230b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557e22311710 0x557e22310420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e222831d0 0x557e222a5dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f79ffb640 0 client.15087 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 --2- 172.21.15.64:0/2940334651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e222828b0 0x557e222a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 >> 172.21.15.64:0/2940334651 conn(0x557e2227ef60 msgr2=0x557e2228e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 shutdown_connections 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 1 -- 172.21.15.64:0/2940334651 wait complete. 2024-09-19T05:05:45.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f9b318740 20 client.15087 trim_cache size 0 max 16384 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24466, 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 89.242277018999999, 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-19T05:05:45.257 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-09-19T05:05:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 78, 2024-09-19T05:05:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 115.71189607300001 2024-09-19T05:05:45.258 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:45.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7f800a3350 msgr2=0x7f7f800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7f800a3350 0x7f7f800a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c019000 tx=0x7f7f8c01b000 comp rx=0 tx=0).stop 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 msgr2=0x7f7f94199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 0x7f7f94199140 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c004770 tx=0x7f7f7c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 shutdown_connections 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f7f800a3350 0x7f7f800a5810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f94106a80 0x7f7f941a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f94101420 0x7f7f94199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.246+0000 7f7f98f3f640 1 --2- 172.21.15.64:0/1111083167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f94100a80 0x7f7f94199140 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.247+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 >> 172.21.15.64:0/1111083167 conn(0x7f7f940fc7a0 msgr2=0x7f7f940fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.247+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 shutdown_connections 2024-09-19T05:05:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:45.247+0000 7f7f98f3f640 1 -- 172.21.15.64:0/1111083167 wait complete. 2024-09-19T05:05:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:45 smithi103 ceph-mon[34446]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:45.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:45 smithi064 ceph-mon[30960]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:45 smithi174 ceph-mon[34488]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T05:05:45.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:05:46.183 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:46.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 -- 172.21.15.64:0/2741939859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc1008d0 msgr2=0x7f6fbc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2741939859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc1008d0 0x7f6fbc100cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009900 tx=0x7f6fb802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:46.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 -- 172.21.15.64:0/2741939859 shutdown_connections 2024-09-19T05:05:46.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2741939859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fbc10e480 0x7f6fbc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2741939859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fbc101280 0x7f6fbc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2741939859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc1008d0 0x7f6fbc100cb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 -- 172.21.15.64:0/2741939859 >> 172.21.15.64:0/2741939859 conn(0x7f6fbc0fc7a0 msgr2=0x7f6fbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 -- 172.21.15.64:0/2741939859 shutdown_connections 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.642+0000 7f6fc3774640 1 -- 172.21.15.64:0/2741939859 wait complete. 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 Processor -- start 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 -- start start 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fbc1008d0 0x7f6fbc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 0x7f6fbc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fbc10e480 0x7f6fbc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc114060 con 0x7f6fbc101280 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc113ee0 con 0x7f6fbc1008d0 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc3774640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc1141e0 con 0x7f6fbc10e480 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc0d0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 0x7f6fbc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc0d0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 0x7f6fbc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46340/0 (socket says 172.21.15.64:46340) 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/2764757288 learned_addr learned my addr 172.21.15.64:0/2764757288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:46.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc1510640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fbc1008d0 0x7f6fbc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.643+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fbc10e480 0x7f6fbc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/2764757288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fbc10e480 msgr2=0x7f6fbc19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fbc10e480 0x7f6fbc19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/2764757288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fbc1008d0 msgr2=0x7f6fbc19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fbc1008d0 0x7f6fbc19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/2764757288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc19f550 con 0x7f6fbc101280 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 0x7f6fbc19aae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c890 tx=0x7f6fb000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fae7fc640 1 -- 172.21.15.64:0/2764757288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004220 con 0x7f6fbc101280 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc19f840 con 0x7f6fbc101280 2024-09-19T05:05:46.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fae7fc640 1 -- 172.21.15.64:0/2764757288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fb00043c0 con 0x7f6fbc101280 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fae7fc640 1 -- 172.21.15.64:0/2764757288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004a00 con 0x7f6fbc101280 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.644+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc1a7120 con 0x7f6fbc101280 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.646+0000 7f6fae7fc640 1 -- 172.21.15.64:0/2764757288 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6fb0004ba0 con 0x7f6fbc101280 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 Processor -- start 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 -- start start 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580e9ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9e2f0 0x562580ea8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580eaafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562580e04090 con 0x562580e9d9d0 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562580daba70 con 0x562580ea8bb0 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562580e00050 con 0x562580e9e2f0 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580e9ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580e9ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46350/0 (socket says 172.21.15.64:46350) 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 -- 172.21.15.64:0/375187513 learned_addr learned my addr 172.21.15.64:0/375187513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1510640 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9e2f0 0x562580ea8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 -- 172.21.15.64:0/375187513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9e2f0 msgr2=0x562580ea8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9e2f0 0x562580ea8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 -- 172.21.15.64:0/375187513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 msgr2=0x562580eaafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580eaafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580eaafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1d11640 1 -- 172.21.15.64:0/375187513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562580db8130 con 0x562580e9d9d0 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fae7fc640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6fa80a32d0 0x7f6fa80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.647+0000 7f6fc1510640 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9e2f0 0x562580ea8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580e9ddb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009f10 tx=0x7f6fb4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a4554fa72b19dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fc1510640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6fa80a32d0 0x7f6fa80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fae7fc640 1 -- 172.21.15.64:0/2764757288 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f6fb00cf6d0 con 0x7f6fbc101280 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fa77fe640 1 -- 172.21.15.64:0/375187513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4016070 con 0x562580e9d9d0 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fa77fe640 1 -- 172.21.15.64:0/375187513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fb401bb10 con 0x562580e9d9d0 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fa77fe640 1 -- 172.21.15.64:0/375187513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4012c30 con 0x562580e9d9d0 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/375187513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 msgr2=0x562580e9ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fc1510640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6fa80a32d0 0x7f6fa80a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8031440 tx=0x7f6fb8005890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.648+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580e9ddb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009f10 tx=0x7f6fb4010740 comp rx=0 tx=0).stop 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/375187513 shutdown_connections 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580eaafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9e2f0 0x562580ea8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/375187513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580e9ddb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/375187513 >> 172.21.15.64:0/375187513 conn(0x562580e99f60 msgr2=0x562580e9a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/375187513 shutdown_connections 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/375187513 wait complete. 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- start start 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 Processor -- start 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9d9d0 0x562580ea84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580ea5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562580eab780 con 0x562580e9e2f0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562580dbef20 con 0x562580ea8bb0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562580eab900 con 0x562580e9d9d0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.649+0000 7f6fc5b4d740 10 client.0 _ll_register_callbacks cb 0x562580d88030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46358/0 (socket says 172.21.15.64:46358) 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/809820924 learned_addr learned my addr 172.21.15.64:0/809820924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580ea5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc1510640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9d9d0 0x562580ea84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/809820924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9d9d0 msgr2=0x562580ea84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9d9d0 0x562580ea84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 msgr2=0x562580ea5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580ea5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/809820924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562580b752b0 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580ea5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea5830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0005fd0 tx=0x7f6fb009b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34166f12898cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/809820924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb001d040 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/809820924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f6fb00cd2b0 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb401bb10 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/809820924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6fb009ff20 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc5b4d740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562580eb7130 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.650+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/809820924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fb401bb10 con 0x562580e9e2f0 2024-09-19T05:05:46.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/809820924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6fb00860c0 con 0x562580e9e2f0 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc5b4d740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc5b4d740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc5b4d740 20 client.15099 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580eb9700 0x562580ebab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc5b4d740 4 client.15099 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562580e11600 con 0x562580eb9700 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc1510640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580eb9700 0x562580ebab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fc1510640 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580eb9700 0x562580ebab70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667bb25cec08d740 server_cookie=180352932437b36c in_seq=0 out_seq=0 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.651+0000 7f6fa4ff9640 10 client.15099 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:46.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/809820924 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562580e11600 con 0x562580eb9700 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fa4ff9640 10 client.15099 handle_command_reply: tid=0 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 client.15099 shutdown 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 msgr2=0x562580ea5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea5830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0005fd0 tx=0x7f6fb009b760 comp rx=0 tx=0).stop 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 shutdown_connections 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580ea8bb0 0x562580ea5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580eb9700 0x562580ebab70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea5830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/809820924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580e9d9d0 0x562580ea84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fa4ff9640 0 client.15099 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 >> 172.21.15.64:0/809820924 conn(0x562580e99f60 msgr2=0x562580e9fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.652+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 shutdown_connections 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.653+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/809820924 wait complete. 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.653+0000 7f6fc5b4d740 20 client.15099 trim_cache size 0 max 16384 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.670+0000 7f6fc5b4d740 1 Processor -- start 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.670+0000 7f6fc5b4d740 1 -- start start 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580ebab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9e2f0 0x562580ea57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562580eb8eb0 con 0x562580e9d9d0 2024-09-19T05:05:46.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562580eb9580 con 0x562580e9e2f0 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562580eb9c50 con 0x562580ea8bb0 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580ebab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580ebab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46360/0 (socket says 172.21.15.64:46360) 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 -- 172.21.15.64:0/1954343830 learned_addr learned my addr 172.21.15.64:0/1954343830 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9e2f0 0x562580ea57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 -- 172.21.15.64:0/1954343830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 msgr2=0x562580ea5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 -- 172.21.15.64:0/1954343830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9e2f0 msgr2=0x562580ea57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9e2f0 0x562580ea57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 -- 172.21.15.64:0/1954343830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562580ea6870 con 0x562580e9d9d0 2024-09-19T05:05:46.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9e2f0 0x562580ea57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:46.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:46.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fc1510640 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580ebab70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6fb802a930 tx=0x7f6fb803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd36399131258bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:46.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.671+0000 7f6fa6ffd640 1 -- 172.21.15.64:0/1954343830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8057070 con 0x562580e9d9d0 2024-09-19T05:05:46.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fa6ffd640 1 -- 172.21.15.64:0/1954343830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fb8005040 con 0x562580e9d9d0 2024-09-19T05:05:46.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fa6ffd640 1 -- 172.21.15.64:0/1954343830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8035040 con 0x562580e9d9d0 2024-09-19T05:05:46.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/1954343830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 msgr2=0x562580ebab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580ebab70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6fb802a930 tx=0x7f6fb803e000 comp rx=0 tx=0).stop 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/1954343830 shutdown_connections 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9e2f0 0x562580ea57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/1954343830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9d9d0 0x562580ebab70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/1954343830 >> 172.21.15.64:0/1954343830 conn(0x562580e99f60 msgr2=0x562580e94ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/1954343830 shutdown_connections 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/1954343830 wait complete. 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 -- start start 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.672+0000 7f6fc5b4d740 1 Processor -- start 2024-09-19T05:05:46.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9d9d0 0x562580ec32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562580eba9f0 con 0x562580e9e2f0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562580eba320 con 0x562580e9d9d0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562580ebb0c0 con 0x562580ea8bb0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc5b4d740 10 client.0 _ll_register_callbacks cb 0x7f6fb00058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46376/0 (socket says 172.21.15.64:46376) 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/124098766 learned_addr learned my addr 172.21.15.64:0/124098766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc1d11640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc1510640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9d9d0 0x562580ec32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/124098766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 msgr2=0x562580ea2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9d9d0 msgr2=0x562580ec32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9d9d0 0x562580ec32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc0d0f640 1 -- 172.21.15.64:0/124098766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562580ea2db0 con 0x562580e9e2f0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.673+0000 7f6fc1510640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9d9d0 0x562580ec32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc0d0f640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea2330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00985a0 tx=0x7f6fb00cd4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4fd396fc5ac593 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/124098766 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00b8030 con 0x562580e9e2f0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/124098766 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f6fb0004e90 con 0x562580e9e2f0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb8005040 con 0x562580e9e2f0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/124098766 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6fb001d070 con 0x562580e9e2f0 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 10 client.15105 fetch_fsmap learned FSMap version 24 2024-09-19T05:05:46.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562580ec1a30 con 0x562580e9e2f0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/124098766 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fb8005040 con 0x562580e9e2f0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/124098766 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6fb00cfab0 con 0x562580e9e2f0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 10 client.15105 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 20 client.15105 populate_metadata read hostname 'smithi064' 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580f285d0 0x562580f272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 4 client.15105 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.674+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "status"}) -- 0x562580e0bb10 con 0x562580f285d0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.675+0000 7f6fc1510640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580f285d0 0x562580f272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.675+0000 7f6fc1510640 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580f285d0 0x562580f272e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36387fb495eccfe server_cookie=f03d6288db2e5f2a in_seq=0 out_seq=0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.675+0000 7f6fa4ff9640 10 client.15105 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.675+0000 7f6fa4ff9640 1 -- 172.21.15.64:0/124098766 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562580e0bb10 con 0x562580f285d0 2024-09-19T05:05:46.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.675+0000 7f6fa4ff9640 10 client.15105 handle_command_reply: tid=0 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.675+0000 7f6fc5b4d740 1 client.15105 shutdown 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 msgr2=0x562580ea2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea2330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00985a0 tx=0x7f6fb00cd4e0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 shutdown_connections 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562580ea8bb0 0x562580ea2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562580f285d0 0x562580f272e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562580e9e2f0 0x562580ea2330 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 --2- 172.21.15.64:0/124098766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562580e9d9d0 0x562580ec32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 >> 172.21.15.64:0/124098766 conn(0x562580e99f60 msgr2=0x562580ea57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fa4ff9640 0 client.15105 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 shutdown_connections 2024-09-19T05:05:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 1 -- 172.21.15.64:0/124098766 wait complete. 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc5b4d740 20 client.15105 trim_cache size 0 max 16384 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "5ccce474-7643-11ef-bceb-c7b262605968", 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24479, 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-19T05:05:46.699 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-19T05:05:46.700 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 97.707488079000001, 2024-09-19T05:05:46.700 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-19T05:05:46.700 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 79, 2024-09-19T05:05:46.700 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 78, 2024-09-19T05:05:46.700 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 108.211224566 2024-09-19T05:05:46.700 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6fa80a32d0 msgr2=0x7f6fa80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6fa80a32d0 0x7f6fa80a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8031440 tx=0x7f6fb8005890 comp rx=0 tx=0).stop 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 msgr2=0x7f6fbc19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.676+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 0x7f6fbc19aae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c890 tx=0x7f6fb000cd50 comp rx=0 tx=0).stop 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 shutdown_connections 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f6fa80a32d0 0x7f6fa80a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fbc10e480 0x7f6fbc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fbc101280 0x7f6fbc19aae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 --2- 172.21.15.64:0/2764757288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fbc1008d0 0x7f6fbc19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 >> 172.21.15.64:0/2764757288 conn(0x7f6fbc0fc7a0 msgr2=0x7f6fbc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 shutdown_connections 2024-09-19T05:05:46.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:46.677+0000 7f6fc3774640 1 -- 172.21.15.64:0/2764757288 wait complete. 2024-09-19T05:05:47.437 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T05:05:47.447 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T05:05:47.447 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T05:05:47.447 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T05:05:47.448 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T05:05:47.448 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:05:47.449 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:05:47.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 -- 172.21.15.64:0/1093439753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 msgr2=0x7f59dc1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 --2- 172.21.15.64:0/1093439753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc1072e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009900 tx=0x7f59c402a410 comp rx=0 tx=0).stop 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 -- 172.21.15.64:0/1093439753 shutdown_connections 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 --2- 172.21.15.64:0/1093439753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59dc107820 0x7f59dc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 --2- 172.21.15.64:0/1093439753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc1072e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 --2- 172.21.15.64:0/1093439753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59dc1025c0 0x7f59dc1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 -- 172.21.15.64:0/1093439753 >> 172.21.15.64:0/1093439753 conn(0x7f59dc0fc1e0 msgr2=0x7f59dc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 -- 172.21.15.64:0/1093439753 shutdown_connections 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 -- 172.21.15.64:0/1093439753 wait complete. 2024-09-19T05:05:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 Processor -- start 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.555+0000 7f59e301e640 1 -- start start 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59dc1025c0 0x7f59dc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59dc107820 0x7f59dc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d700 con 0x7f59dc104ef0 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d580 con 0x7f59dc107820 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d880 con 0x7f59dc1025c0 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46390/0 (socket says 172.21.15.64:46390) 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 -- 172.21.15.64:0/2241013792 learned_addr learned my addr 172.21.15.64:0/2241013792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e15bb640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59dc107820 0x7f59dc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:47.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 -- 172.21.15.64:0/2241013792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59dc1025c0 msgr2=0x7f59dc197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59dc1025c0 0x7f59dc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 -- 172.21.15.64:0/2241013792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59dc107820 msgr2=0x7f59dc19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59dc107820 0x7f59dc19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 -- 172.21.15.64:0/2241013792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc19f9e0 con 0x7f59dc104ef0 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59dbfff640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc198300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f59c40314e0 tx=0x7f59c40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c403f030 con 0x7f59dc104ef0 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59c4003cb0 con 0x7f59dc104ef0 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc19fc70 con 0x7f59dc104ef0 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.556+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4035070 con 0x7f59dc104ef0 2024-09-19T05:05:47.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.557+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a0160 con 0x7f59dc104ef0 2024-09-19T05:05:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.558+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc0fdcd0 con 0x7f59dc104ef0 2024-09-19T05:05:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.559+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f59c4042030 con 0x7f59dc104ef0 2024-09-19T05:05:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.559+0000 7f59d9ffb640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c80a3390 0x7f59c80a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.560+0000 7f59e0dba640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c80a3390 0x7f59c80a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.560+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f59c40f05a0 con 0x7f59dc104ef0 2024-09-19T05:05:47.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.560+0000 7f59e0dba640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c80a3390 0x7f59c80a5850 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005fd0 tx=0x7f59d4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.563+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59c40bd140 con 0x7f59dc104ef0 2024-09-19T05:05:47.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:47 smithi064 ceph-mon[30960]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:47.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:47 smithi103 ceph-mon[34446]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:47.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:47 smithi174 ceph-mon[34488]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:05:47.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:47.789+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f59dc02da10 con 0x7f59dc104ef0 2024-09-19T05:05:48.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.464+0000 7f59d9ffb640 1 -- 172.21.15.64:0/2241013792 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f59c40bca40 con 0x7f59dc104ef0 2024-09-19T05:05:48.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c80a3390 msgr2=0x7f59c80a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:48.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c80a3390 0x7f59c80a5850 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005fd0 tx=0x7f59d4005850 comp rx=0 tx=0).stop 2024-09-19T05:05:48.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 msgr2=0x7f59dc198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:48.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc198300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f59c40314e0 tx=0x7f59c40042e0 comp rx=0 tx=0).stop 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 shutdown_connections 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f59c80a3390 0x7f59c80a5850 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59dc107820 0x7f59dc19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59dc104ef0 0x7f59dc198300 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 --2- 172.21.15.64:0/2241013792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59dc1025c0 0x7f59dc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 >> 172.21.15.64:0/2241013792 conn(0x7f59dc0fc1e0 msgr2=0x7f59dc102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 shutdown_connections 2024-09-19T05:05:48.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:48.467+0000 7f59e301e640 1 -- 172.21.15.64:0/2241013792 wait complete. 2024-09-19T05:05:48.486 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T05:05:48.516 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T05:05:48.516 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-19T05:05:48.516 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-19T05:05:48.516 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T05:05:48.516 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:48.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:48.517 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-19T05:05:48.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:48.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-09-19T05:05:48.604 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-09-19T05:05:48.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:05:48.606 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T05:05:48.607 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T05:05:48.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T05:05:48.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2241013792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T05:05:48.719 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: ignoring --setuser ceph since I am not root 2024-09-19T05:05:48.719 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: ignoring --setgroup ceph since I am not root 2024-09-19T05:05:48.719 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-mgr[35811]: -- 172.21.15.103:0/238046317 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56061be250e0 con 0x56061be03400 2024-09-19T05:05:48.720 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:48.618+0000 7fb976ba4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:05:48.720 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:48.675+0000 7fb976ba4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:05:48.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2241013792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T05:05:48.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: ignoring --setuser ceph since I am not root 2024-09-19T05:05:48.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: ignoring --setgroup ceph since I am not root 2024-09-19T05:05:48.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-mgr[36292]: -- 172.21.15.174:0/3099311519 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1a4c870e0 con 0x55b1a4c64800 2024-09-19T05:05:48.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:48.615+0000 7fd83e396040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:05:48.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:48.673+0000 7fd83e396040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:05:48.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2241013792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T05:05:48.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ignoring --setuser ceph since I am not root 2024-09-19T05:05:48.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ignoring --setgroup ceph since I am not root 2024-09-19T05:05:48.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:48.625+0000 7fdca10fe040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:05:48.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:48.687+0000 7fdca10fe040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:05:48.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:48.847+0000 7fdca10fe040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:05:48.916 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:49.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:48 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:48.928+0000 7fdca10fe040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:05:49.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:49.086+0000 7fdca10fe040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:05:49.176 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:48.821+0000 7fb976ba4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:05:49.176 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:48 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:48.886+0000 7fb976ba4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:05:49.177 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:49.032+0000 7fb976ba4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:05:49.177 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:49.175+0000 7fb976ba4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:05:49.177 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:48.821+0000 7fd83e396040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:05:49.178 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:48.887+0000 7fd83e396040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:05:49.178 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:49.034+0000 7fd83e396040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:05:49.178 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:49.176+0000 7fd83e396040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:05:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 -- 172.21.15.64:0/2447657722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102570 msgr2=0x7f329c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 --2- 172.21.15.64:0/2447657722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102570 0x7f329c1029b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f328c009900 tx=0x7f328c02a4a0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 -- 172.21.15.64:0/2447657722 shutdown_connections 2024-09-19T05:05:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 --2- 172.21.15.64:0/2447657722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f329c102ef0 0x7f329c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 --2- 172.21.15.64:0/2447657722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102570 0x7f329c1029b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 --2- 172.21.15.64:0/2447657722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 0x7f329c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 -- 172.21.15.64:0/2447657722 >> 172.21.15.64:0/2447657722 conn(0x7f329c0fc7d0 msgr2=0x7f329c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.468+0000 7f32a3672640 1 -- 172.21.15.64:0/2447657722 shutdown_connections 2024-09-19T05:05:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.469+0000 7f32a3672640 1 -- 172.21.15.64:0/2447657722 wait complete. 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 Processor -- start 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 -- start start 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 0x7f329c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f329c102570 0x7f329c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 0x7f329c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f329c110e70 con 0x7f329c102ef0 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f329c110cf0 con 0x7f329c100e20 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a3672640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f329c110ff0 con 0x7f329c102570 2024-09-19T05:05:49.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a1c0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 0x7f329c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a1c0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 0x7f329c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46428/0 (socket says 172.21.15.64:46428) 2024-09-19T05:05:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a1c0f640 1 -- 172.21.15.64:0/2280249992 learned_addr learned my addr 172.21.15.64:0/2280249992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a140e640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 0x7f329c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.470+0000 7f32a0c0d640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f329c102570 0x7f329c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.471+0000 7f32a1c0f640 1 -- 172.21.15.64:0/2280249992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f329c102570 msgr2=0x7f329c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.471+0000 7f32a1c0f640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f329c102570 0x7f329c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.471+0000 7f32a1c0f640 1 -- 172.21.15.64:0/2280249992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 msgr2=0x7f329c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.471+0000 7f32a1c0f640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 0x7f329c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.471+0000 7f32a1c0f640 1 -- 172.21.15.64:0/2280249992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c19d4a0 con 0x7f329c102ef0 2024-09-19T05:05:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.471+0000 7f32a140e640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 0x7f329c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:49.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.474+0000 7f32a1c0f640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 0x7f329c19cdc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3294009390 tx=0x7f329400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.477+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329401c070 con 0x7f329c102ef0 2024-09-19T05:05:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.477+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c19d790 con 0x7f329c102ef0 2024-09-19T05:05:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.477+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3294007370 con 0x7f329c102ef0 2024-09-19T05:05:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.478+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c0fcbb0 con 0x7f329c102ef0 2024-09-19T05:05:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.478+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294026060 con 0x7f329c102ef0 2024-09-19T05:05:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.479+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c1021b0 con 0x7f329c102ef0 2024-09-19T05:05:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.480+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f329401f070 con 0x7f329c102ef0 2024-09-19T05:05:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.481+0000 7f32927fc640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 0x7f32880a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.481+0000 7f32a140e640 1 -- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 msgr2=0x7f32880a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.481+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7f32940d7a30 con 0x7f329c102ef0 2024-09-19T05:05:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.481+0000 7f32a140e640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 0x7f32880a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:05:49.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.487+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32940a46d0 con 0x7f329c102ef0 2024-09-19T05:05:49.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2241013792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T05:05:49.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-mon[30960]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T05:05:49.489 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:49.236+0000 7fdca10fe040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:05:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.682+0000 7f32a140e640 1 -- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 msgr2=0x7f32880a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:49.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.682+0000 7f32a140e640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 0x7f32880a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:05:49.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.705+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f329c10a0a0 con 0x7f329c102ef0 2024-09-19T05:05:49.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.705+0000 7f32927fc640 1 -- 172.21.15.64:0/2280249992 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f32940a3fd0 con 0x7f329c102ef0 2024-09-19T05:05:49.707 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 msgr2=0x7f32880a57d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 0x7f32880a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 msgr2=0x7f329c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 0x7f329c19cdc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3294009390 tx=0x7f329400bee0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 shutdown_connections 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7f32880a3310 0x7f32880a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f329c102ef0 0x7f329c19cdc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f329c102570 0x7f329c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 --2- 172.21.15.64:0/2280249992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c100e20 0x7f329c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 >> 172.21.15.64:0/2280249992 conn(0x7f329c0fc7d0 msgr2=0x7f329c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 shutdown_connections 2024-09-19T05:05:49.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:49.707+0000 7f32a3672640 1 -- 172.21.15.64:0/2280249992 wait complete. 2024-09-19T05:05:49.734 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2241013792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T05:05:49.734 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-mon[34488]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T05:05:49.734 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:49.487+0000 7fd83e396040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:05:49.734 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:49.548+0000 7fd83e396040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:05:49.734 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:49.732+0000 7fd83e396040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:05:49.737 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2241013792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T05:05:49.737 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-mon[34446]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T05:05:49.737 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:49.490+0000 7fb976ba4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:05:49.737 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:49.552+0000 7fb976ba4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:05:49.842 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:49.578+0000 7fdca10fe040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:05:49.842 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:49.642+0000 7fdca10fe040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:05:49.842 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:49.841+0000 7fdca10fe040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:05:50.218 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:49 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:49.736+0000 7fb976ba4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:05:50.437 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T05:05:50.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T05:05:50.438 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-09-19T05:05:50.438 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T05:05:50.438 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:50.438 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T05:05:50.438 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T05:05:50.438 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T05:05:50.439 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:05:50.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:50.439 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:05:50.458 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T05:05:50.459 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-09-19T05:05:50.460 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T05:05:50.460 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42210sec preferred_lft 42210sec 2024-09-19T05:05:50.460 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-19T05:05:50.460 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:05:50.461 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T05:05:50.461 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-19T05:05:50.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:05:50.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:50 smithi064 ceph-mon[30960]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T05:05:50.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:50 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2280249992' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:05:50.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:05:50.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:50.673 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T05:05:50.700 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-09-19T05:05:50.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:50.702 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-09-19T05:05:50.775 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-09-19T05:05:50.775 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T05:05:50.775 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T05:05:50.775 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T05:05:50.776 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T05:05:50.776 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:05:50.776 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-19T05:05:50.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T05:05:50.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T05:05:50.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T05:05:50.777 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-19T05:05:50.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:50 smithi174 ceph-mon[34488]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T05:05:50.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:50 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2280249992' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:05:50.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:50.612+0000 7fd83e396040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:05:50.778 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:50.678+0000 7fd83e396040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:05:50.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:05:50.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:50 smithi103 ceph-mon[34446]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T05:05:50.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:50 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2280249992' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:05:50.969 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:50 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:50.618+0000 7fb976ba4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:05:50.969 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:50 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:50.684+0000 7fb976ba4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:05:50.982 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:50.793+0000 7fdca10fe040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:05:50.982 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:50.866+0000 7fdca10fe040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:05:51.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:05:51.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:51.070 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-19T05:05:51.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:51.086 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-09-19T05:05:51.142 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:05:51.143 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-19T05:05:51.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:05:51.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T05:05:51.143 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-19T05:05:51.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:05:51.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:05:51.264 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T05:05:51.265 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-19T05:05:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:05:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:05:51.393 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:05:51.393 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-19T05:05:51.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-09-19T05:05:51.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T05:05:51.394 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-19T05:05:51.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:05:51.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:05:51.474 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-19T05:05:51.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:05:51.474 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:51.491 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:05:51.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:05:51.492 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:51.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-09-19T05:05:51.616 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T05:05:51.637 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.379+0000 7fd83e396040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:05:51.638 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.494+0000 7fd83e396040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:05:51.646 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.389+0000 7fb976ba4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:05:51.647 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.504+0000 7fb976ba4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:05:51.674 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-09-19T05:05:51.674 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-09-19T05:05:51.674 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-09-19T05:05:51.674 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-09-19T05:05:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-09-19T05:05:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-09-19T05:05:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-09-19T05:05:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-09-19T05:05:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-09-19T05:05:51.675 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-09-19T05:05:51.676 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-09-19T05:05:51.676 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-09-19T05:05:51.676 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:05:51.676 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-09-19T05:05:51.676 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-09-19T05:05:51.676 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-09-19T05:05:51.677 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-09-19T05:05:51.677 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T05:05:51.677 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-09-19T05:05:51.677 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-09-19T05:05:51.677 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T05:05:51.677 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T05:05:51.678 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T05:05:51.678 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-09-19T05:05:51.678 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-09-19T05:05:51.678 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-09-19T05:05:51.678 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/59c9a8c8a70d5d09932882399fe5ce5c89f01503d79d0501ce2d2de807934b5e/merged 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/3a99e0189f8193694eb7dbe51db90f039fd44c1bbd434ae8d12c0acf24727205/merged 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/bb22627ba28d8eaf238774f7ebc97f21a759a1b01e0b57cec5b8ee3098cbb95b/merged 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/b87d90dbe9d06d1fc6cd3aef988d741dfcf1a78942d1f22cd3bac42ba0f28089/merged 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/f8aa431e68d7fec950cfe22f936c9e78ce1e3d0b98589c572fde06153227f4ee/merged 2024-09-19T05:05:51.679 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5a37665174a9fbeb4919badbe29d4e1993d8524d6adcc4c839c773de5db3077a/merged 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/c2a87d33de11eeef5efb6fb60625ef08d16a321dee58b4671e24354d73c870aa/merged 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5ef3a603f7cf424317d402a91debb713f1b8f178f7e2798723f1c9a02eb7b97d/merged 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/09db34bba56e0e7dd5ff44607f9f16e2cce8392a55647a62b3d42962cf55e318/merged 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/079a35e61be0d5d43d5ee89c1212b3b52760454f7a02f29ac133f75bae538199/merged 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:05:51.680 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:05:51.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:51.681 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-09-19T05:05:51.730 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T05:05:51.731 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/32bf837e-c916-4a18-9e8b-e6b294dbeebe --client_fs=cephfs) 2024-09-19T05:05:51.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-09-19T05:05:51.804 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-09-19T05:05:51.861 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-09-19T05:05:51.862 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/59c9a8c8a70d5d09932882399fe5ce5c89f01503d79d0501ce2d2de807934b5e/merged 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/3a99e0189f8193694eb7dbe51db90f039fd44c1bbd434ae8d12c0acf24727205/merged 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/bb22627ba28d8eaf238774f7ebc97f21a759a1b01e0b57cec5b8ee3098cbb95b/merged 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/b87d90dbe9d06d1fc6cd3aef988d741dfcf1a78942d1f22cd3bac42ba0f28089/merged 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/f8aa431e68d7fec950cfe22f936c9e78ce1e3d0b98589c572fde06153227f4ee/merged 2024-09-19T05:05:51.863 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5a37665174a9fbeb4919badbe29d4e1993d8524d6adcc4c839c773de5db3077a/merged 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/c2a87d33de11eeef5efb6fb60625ef08d16a321dee58b4671e24354d73c870aa/merged 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/5ef3a603f7cf424317d402a91debb713f1b8f178f7e2798723f1c9a02eb7b97d/merged 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/09db34bba56e0e7dd5ff44607f9f16e2cce8392a55647a62b3d42962cf55e318/merged 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/079a35e61be0d5d43d5ee89c1212b3b52760454f7a02f29ac133f75bae538199/merged 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:05:51.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:51.864 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-09-19T05:05:51.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-09-19T05:05:51.883+0000 7fbf0ad88480 -1 init, newargv = 0x56059befcb40 newargc=15 2024-09-19T05:05:51.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[80601]: starting ceph client 2024-09-19T05:05:51.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[80601]: starting fuse 2024-09-19T05:05:51.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:51.646+0000 7fdca10fe040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:05:51.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:51.770+0000 7fdca10fe040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:05:51.901 INFO:teuthology.orchestra.run.smithi064.stdout:120 2024-09-19T05:05:51.902 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T05:05:51.902 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-09-19T05:05:51.903 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-09-19T05:05:51.904 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T05:05:51.904 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-19T05:05:51.904 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-09-19T05:05:51.904 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-09-19T05:05:51.916 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.635+0000 7fd83e396040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:05:51.916 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.697+0000 7fd83e396040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:05:51.916 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.845+0000 7fd83e396040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:05:51.924 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.645+0000 7fb976ba4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:05:51.924 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.706+0000 7fb976ba4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:05:51.924 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.854+0000 7fb976ba4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:05:51.924 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.923+0000 7fb976ba4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:05:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.80601.asok 2024-09-19T05:05:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:05:52.040 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80601.asok 2024-09-19T05:05:52.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:52.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80601.asok status 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-09-19T05:05:52.179 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "80601", 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/volumes/qa/sv_0/32bf837e-c916-4a18-9e8b-e6b294dbeebe" 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 15138, 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 15138 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-09-19T05:05:52.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3079718954 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3079718954 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.15138 192.168.0.1:0/3079718954", 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3079718954", 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 5, 2024-09-19T05:05:52.181 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 24, 2024-09-19T05:05:52.182 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 79, 2024-09-19T05:05:52.182 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-09-19T05:05:52.182 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-09-19T05:05:52.182 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-09-19T05:05:52.182 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:05:52.187 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:51.930+0000 7fdca10fe040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:05:52.187 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.002+0000 7fdca10fe040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:05:52.187 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.166+0000 7fdca10fe040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:05:52.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:52.188 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-09-19T05:05:52.188 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-09-19T05:05:52.207 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:52.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:05:52.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:05:52.207 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.914+0000 7fd83e396040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:05:52.207 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:51.975+0000 7fd83e396040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:05:52.208 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:52.138+0000 7fd83e396040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:05:52.208 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:52.204+0000 7fd83e396040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:05:52.213 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:51.985+0000 7fb976ba4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:05:52.213 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:52.146+0000 7fb976ba4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:05:52.277 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T05:05:52.287 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:05:52.287 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:05:52.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 -- 172.21.15.64:0/730228498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4101c30 msgr2=0x7fd9e41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/730228498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4101c30 0x7fd9e41020b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009900 tx=0x7fd9d802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 -- 172.21.15.64:0/730228498 shutdown_connections 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/730228498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e41025f0 0x7fd9e4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/730228498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4101c30 0x7fd9e41020b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/730228498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e4100a40 0x7fd9e4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 -- 172.21.15.64:0/730228498 >> 172.21.15.64:0/730228498 conn(0x7fd9e40fc1e0 msgr2=0x7fd9e40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 -- 172.21.15.64:0/730228498 shutdown_connections 2024-09-19T05:05:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.421+0000 7fd9ea41e640 1 -- 172.21.15.64:0/730228498 wait complete. 2024-09-19T05:05:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 Processor -- start 2024-09-19T05:05:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 -- start start 2024-09-19T05:05:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 0x7fd9e410b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e4101c30 0x7fd9e4109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e41025f0 0x7fd9e4109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e410d720 con 0x7fd9e4100a40 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e410d5a0 con 0x7fd9e41025f0 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9ea41e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e410d8a0 con 0x7fd9e4101c30 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 0x7fd9e410b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 0x7fd9e410b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46444/0 (socket says 172.21.15.64:46444) 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9e3fff640 1 -- 172.21.15.64:0/1994605726 learned_addr learned my addr 172.21.15.64:0/1994605726 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9e89bb640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e41025f0 0x7fd9e4109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.422+0000 7fd9e37fe640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e4101c30 0x7fd9e4109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e3fff640 1 -- 172.21.15.64:0/1994605726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e4101c30 msgr2=0x7fd9e4109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e3fff640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e4101c30 0x7fd9e4109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e3fff640 1 -- 172.21.15.64:0/1994605726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e41025f0 msgr2=0x7fd9e4109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e3fff640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e41025f0 0x7fd9e4109f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e3fff640 1 -- 172.21.15.64:0/1994605726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e410a7d0 con 0x7fd9e4100a40 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e37fe640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e4101c30 0x7fd9e4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e89bb640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e41025f0 0x7fd9e4109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e3fff640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 0x7fd9e410b3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e9c0 tx=0x7fd9d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400cde0 con 0x7fd9e4100a40 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9d40108e0 con 0x7fd9e4100a40 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4018950 con 0x7fd9e4100a40 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e41a1e00 con 0x7fd9e4100a40 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.423+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e41a2200 con 0x7fd9e4100a40 2024-09-19T05:05:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.424+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e4107db0 con 0x7fd9e4100a40 2024-09-19T05:05:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.425+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fd9d4010a90 con 0x7fd9e4100a40 2024-09-19T05:05:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.426+0000 7fd9e17fa640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 0x7fd9cc0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.426+0000 7fd9e37fe640 1 -- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 msgr2=0x7fd9cc0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.426+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fd9d40d0600 con 0x7fd9e4100a40 2024-09-19T05:05:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.426+0000 7fd9e37fe640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 0x7fd9cc0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:05:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.429+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9d4026070 con 0x7fd9e4100a40 2024-09-19T05:05:52.468 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:52.212+0000 7fb976ba4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:05:52.469 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:52.279+0000 7fb976ba4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:05:52.469 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:52.362+0000 7fb976ba4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:05:52.469 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:52.421+0000 7fb976ba4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:05:52.473 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:52.270+0000 7fd83e396040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:05:52.473 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:52.353+0000 7fd83e396040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:05:52.473 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:52.412+0000 7fd83e396040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:05:52.473 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:52.471+0000 7fd83e396040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:05:52.480 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.241+0000 7fdca10fe040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:05:52.481 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.307+0000 7fdca10fe040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:05:52.481 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.479+0000 7fdca10fe040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:05:52.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.627+0000 7fd9e37fe640 1 -- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 msgr2=0x7fd9cc0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:52.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.627+0000 7fd9e37fe640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 0x7fd9cc0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:05:52.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.646+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fd9e402d7c0 con 0x7fd9e4100a40 2024-09-19T05:05:52.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.698+0000 7fd9e17fa640 1 -- 172.21.15.64:0/1994605726 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fd9d409d2e0 con 0x7fd9e4100a40 2024-09-19T05:05:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 msgr2=0x7fd9cc0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:05:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 0x7fd9cc0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 msgr2=0x7fd9e410b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 0x7fd9e410b3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e9c0 tx=0x7fd9d400ee80 comp rx=0 tx=0).stop 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 shutdown_connections 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fd9cc0a3340 0x7fd9cc0a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9e41025f0 0x7fd9e4109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9e4101c30 0x7fd9e4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 --2- 172.21.15.64:0/1994605726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9e4100a40 0x7fd9e410b3c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 >> 172.21.15.64:0/1994605726 conn(0x7fd9e40fc1e0 msgr2=0x7fd9e4104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 shutdown_connections 2024-09-19T05:05:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.700+0000 7fd9ea41e640 1 -- 172.21.15.64:0/1994605726 wait complete. 2024-09-19T05:05:52.713 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T05:05:52.735 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1994605726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T05:05:52.735 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.548+0000 7fdca10fe040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:05:52.736 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.619+0000 7fdca10fe040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:05:52.736 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.708+0000 7fdca10fe040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:05:52.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:52 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1994605726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T05:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 -- 172.21.15.64:0/1798397155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf38104f90 msgr2=0x7fcf38107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 --2- 172.21.15.64:0/1798397155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf38104f90 0x7fcf38107380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009900 tx=0x7fcf2802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:52.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 -- 172.21.15.64:0/1798397155 shutdown_connections 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 --2- 172.21.15.64:0/1798397155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf381078c0 0x7fcf38109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 --2- 172.21.15.64:0/1798397155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf38104f90 0x7fcf38107380 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 --2- 172.21.15.64:0/1798397155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf38102660 0x7fcf38104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 -- 172.21.15.64:0/1798397155 >> 172.21.15.64:0/1798397155 conn(0x7fcf380fc1e0 msgr2=0x7fcf380fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 -- 172.21.15.64:0/1798397155 shutdown_connections 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 -- 172.21.15.64:0/1798397155 wait complete. 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 Processor -- start 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.819+0000 7fcf4038d640 1 -- start start 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf4038d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf38102660 0x7fcf38197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf4038d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf38104f90 0x7fcf381982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf4038d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf381078c0 0x7fcf3819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf4038d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3810d6d0 con 0x7fcf381078c0 2024-09-19T05:05:52.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf4038d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3810d550 con 0x7fcf38104f90 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf4038d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3810d850 con 0x7fcf38102660 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf381078c0 0x7fcf3819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf381078c0 0x7fcf3819f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46462/0 (socket says 172.21.15.64:46462) 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 -- 172.21.15.64:0/665470887 learned_addr learned my addr 172.21.15.64:0/665470887 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3d928640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf38104f90 0x7fcf381982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e129640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf38102660 0x7fcf38197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf38102660 msgr2=0x7fcf38197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf38102660 0x7fcf38197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf38104f90 msgr2=0x7fcf381982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf38104f90 0x7fcf381982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 -- 172.21.15.64:0/665470887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3819f9c0 con 0x7fcf381078c0 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf3e92a640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf381078c0 0x7fcf3819f320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009c90 tx=0x7fcf3000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:52.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30010630 con 0x7fcf381078c0 2024-09-19T05:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.820+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf30010c80 con 0x7fcf381078c0 2024-09-19T05:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.821+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3819fcb0 con 0x7fcf381078c0 2024-09-19T05:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.821+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30017c60 con 0x7fcf381078c0 2024-09-19T05:05:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.821+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf381a01b0 con 0x7fcf381078c0 2024-09-19T05:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.821+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf380fdcd0 con 0x7fcf381078c0 2024-09-19T05:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.823+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fcf300107d0 con 0x7fcf381078c0 2024-09-19T05:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.824+0000 7fcf2f7fe640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 0x7fcf240a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.824+0000 7fcf3e129640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 msgr2=0x7fcf240a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.824+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8269+0+0 (secure 0 0 0) 0x7fcf300d0500 con 0x7fcf381078c0 2024-09-19T05:05:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.824+0000 7fcf3e129640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 0x7fcf240a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:05:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:52.828+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf3009d1a0 con 0x7fcf381078c0 2024-09-19T05:05:52.968 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:52.481+0000 7fb976ba4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:05:52.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:52 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1994605726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T05:05:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.025+0000 7fcf3e129640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 msgr2=0x7fcf240a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:53.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.025+0000 7fcf3e129640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 0x7fcf240a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:05:53.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.029+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 --> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fcf380626d0 con 0x7fcf240a33c0 2024-09-19T05:05:53.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.773+0000 7fdca10fe040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:05:53.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:52.839+0000 7fdca10fe040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:05:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.426+0000 7fcf3e129640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 msgr2=0x7fcf240a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3907259856 2024-09-19T05:05:53.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.426+0000 7fcf3e129640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 0x7fcf240a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T05:05:53.527 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z[36288]: 2024-09-19T05:05:53.259+0000 7fd83e396040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:05:53.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.696+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 20) ==== 138040+0+0 (secure 0 0 0) 0x7fcf30099110 con 0x7fcf381078c0 2024-09-19T05:05:53.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.697+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 msgr2=0x7fcf240a5880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:05:53.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:53.697+0000 7fcf2f7fe640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/3907259856,v1:172.21.15.64:6801/3907259856] conn(0x7fcf240a33c0 0x7fcf240a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:53.706 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y[35807]: 2024-09-19T05:05:53.273+0000 7fb976ba4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1994605726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: Standby manager daemon z restarted 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: Standby manager daemon z started 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: Standby manager daemon y restarted 2024-09-19T05:05:53.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: Standby manager daemon y started 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: Active manager daemon x restarted 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: Activating manager daemon x 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: mgrmap e20: x(active, starting, since 0.0169613s), standbys: z, y 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:05:53.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:05:53.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:05:53.943 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.678+0000 7fdca10fe040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1994605726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: Standby manager daemon z restarted 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: Standby manager daemon z started 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:05:53.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: Standby manager daemon y restarted 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: Standby manager daemon y started 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: Active manager daemon x restarted 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: Activating manager daemon x 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: mgrmap e20: x(active, starting, since 0.0169613s), standbys: z, y 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:05:53.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:05:53.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:05:53.972 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:53 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:05:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1994605726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T05:05:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T05:05:54.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: Standby manager daemon z restarted 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: Standby manager daemon z started 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.174:0/3452156402' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: Standby manager daemon y restarted 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: Standby manager daemon y started 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.? 172.21.15.103:0/592918966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: Active manager daemon x restarted 2024-09-19T05:05:54.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: Activating manager daemon x 2024-09-19T05:05:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:05:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: mgrmap e20: x(active, starting, since 0.0169613s), standbys: z, y 2024-09-19T05:05:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:05:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:05:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:05:54.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:05:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:05:54.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:05:54.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:53 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:05:54.242 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.937+0000 7fdc0b215640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.242 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.937+0000 7fdc0b215640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.937+0000 7fdc0b215640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.937+0000 7fdc0b215640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.937+0000 7fdc0b215640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.939+0000 7fdc089d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.939+0000 7fdc089d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.939+0000 7fdc089d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.939+0000 7fdc089d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.243 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:53.939+0000 7fdc089d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:54.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.722+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7fcf300ce000 con 0x7fcf381078c0 2024-09-19T05:05:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.723+0000 7fcf2f7fe640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcf24051dd0 0x7fcf240521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:54.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.723+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fcf24034b90 con 0x7fcf24051dd0 2024-09-19T05:05:54.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.726+0000 7fcf3e129640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcf24051dd0 0x7fcf240521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.729+0000 7fcf3e129640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcf24051dd0 0x7fcf240521b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf38199280 tx=0x7fcf34005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:54.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.746+0000 7fcf2f7fe640 1 -- 172.21.15.64:0/665470887 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fcf24034b90 con 0x7fcf24051dd0 2024-09-19T05:05:54.747 INFO:teuthology.orchestra.run.smithi064.stdout:{} 2024-09-19T05:05:54.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcf24051dd0 msgr2=0x7fcf240521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:54.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcf24051dd0 0x7fcf240521b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf38199280 tx=0x7fcf34005f90 comp rx=0 tx=0).stop 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf381078c0 msgr2=0x7fcf3819f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 --2- 172.21.15.64:0/665470887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf381078c0 0x7fcf3819f320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009c90 tx=0x7fcf3000c6a0 comp rx=0 tx=0).stop 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf3e92a640 1 -- 172.21.15.64:0/665470887 reap_dead start 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 shutdown_connections 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 >> 172.21.15.64:0/665470887 conn(0x7fcf380fc1e0 msgr2=0x7fcf38103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 shutdown_connections 2024-09-19T05:05:54.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.748+0000 7fcf4038d640 1 -- 172.21.15.64:0/665470887 wait complete. 2024-09-19T05:05:54.761 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T05:05:54.772 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T05:05:54.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:05:54.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 -- 172.21.15.64:0/3792127790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4104ec0 msgr2=0x7fb0e41072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:54.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 --2- 172.21.15.64:0/3792127790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4104ec0 0x7fb0e41072b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009900 tx=0x7fb0d802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:54.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 -- 172.21.15.64:0/3792127790 shutdown_connections 2024-09-19T05:05:54.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 --2- 172.21.15.64:0/3792127790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e41077f0 0x7fb0e4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:54.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 --2- 172.21.15.64:0/3792127790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4104ec0 0x7fb0e41072b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:54.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 --2- 172.21.15.64:0/3792127790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e4102590 0x7fb0e4104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:54.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 -- 172.21.15.64:0/3792127790 >> 172.21.15.64:0/3792127790 conn(0x7fb0e40fc170 msgr2=0x7fb0e40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:54.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 -- 172.21.15.64:0/3792127790 shutdown_connections 2024-09-19T05:05:54.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.912+0000 7fb0ea703640 1 -- 172.21.15.64:0/3792127790 wait complete. 2024-09-19T05:05:54.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 Processor -- start 2024-09-19T05:05:54.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 -- start start 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 0x7fb0e4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e4104ec0 0x7fb0e4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e41077f0 0x7fb0e419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e410d680 con 0x7fb0e4102590 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e410d500 con 0x7fb0e41077f0 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.913+0000 7fb0ea703640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e410d800 con 0x7fb0e4104ec0 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 0x7fb0e4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:54.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 0x7fb0e4197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42398/0 (socket says 172.21.15.64:42398) 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 -- 172.21.15.64:0/646120580 learned_addr learned my addr 172.21.15.64:0/646120580 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e37fe640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e4104ec0 0x7fb0e4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e8ca0640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e41077f0 0x7fb0e419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 -- 172.21.15.64:0/646120580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e4104ec0 msgr2=0x7fb0e4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e4104ec0 0x7fb0e4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 -- 172.21.15.64:0/646120580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e41077f0 msgr2=0x7fb0e419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e41077f0 0x7fb0e419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 -- 172.21.15.64:0/646120580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e419fa10 con 0x7fb0e4102590 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e3fff640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 0x7fb0e4197df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400ea60 tx=0x7fb0d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400cde0 con 0x7fb0e4102590 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0d40035f0 con 0x7fb0e4102590 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e419fd00 con 0x7fb0e4102590 2024-09-19T05:05:54.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4010690 con 0x7fb0e4102590 2024-09-19T05:05:54.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.914+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e41a01c0 con 0x7fb0e4102590 2024-09-19T05:05:54.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.915+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e40fdc60 con 0x7fb0e4102590 2024-09-19T05:05:54.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.917+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7fb0d40037a0 con 0x7fb0e4102590 2024-09-19T05:05:54.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.917+0000 7fb0e17fa640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb0cc0a30d0 0x7fb0cc0a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:54.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.917+0000 7fb0e37fe640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb0cc0a30d0 0x7fb0cc0a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:54.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.917+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb0d40cf820 con 0x7fb0e4102590 2024-09-19T05:05:54.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.918+0000 7fb0e37fe640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb0cc0a30d0 0x7fb0cc0a5590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004770 tx=0x7fb0d8000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:54.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:54.922+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0d409c710 con 0x7fb0e4102590 2024-09-19T05:05:54.962 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:05:54.962 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:05:54.962 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:05:54.962 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:05:54.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:05:54.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: Manager daemon x is now available 2024-09-19T05:05:54.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:54.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:05:54.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:54 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:05:55.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: Manager daemon x is now available 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:05:55.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:54 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:05:55.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:54.736+0000 7fdc0598a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:55.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:54.736+0000 7fdc0598a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:55.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:54.736+0000 7fdc0598a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:55.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:54.736+0000 7fdc0598a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:55.061 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:54.736+0000 7fdc0598a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:05:55.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: Manager daemon x is now available 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:05:55.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:54 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:05:55.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.104+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0e4062e90 con 0x7fb0e4102590 2024-09-19T05:05:55.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.106+0000 7fb0e17fa640 1 -- 172.21.15.64:0/646120580 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23198 (secure 0 0 0) 0x7fb0d40bc0d0 con 0x7fb0e4102590 2024-09-19T05:05:55.108 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:55.108 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":80,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:05:53.680165+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:6800/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/868126904":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/2000236557":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/2343862077":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/1076683694":"2024-09-20T05:05:53.680120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb0cc0a30d0 msgr2=0x7fb0cc0a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb0cc0a30d0 0x7fb0cc0a5590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004770 tx=0x7fb0d8000bc0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 msgr2=0x7fb0e4197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 0x7fb0e4197df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400ea60 tx=0x7fb0d400ef20 comp rx=0 tx=0).stop 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 shutdown_connections 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb0cc0a30d0 0x7fb0cc0a5590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e41077f0 0x7fb0e419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e4104ec0 0x7fb0e4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 --2- 172.21.15.64:0/646120580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e4102590 0x7fb0e4197df0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 >> 172.21.15.64:0/646120580 conn(0x7fb0e40fc170 msgr2=0x7fb0e4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:55.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 shutdown_connections 2024-09-19T05:05:55.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.108+0000 7fb0ea703640 1 -- 172.21.15.64:0/646120580 wait complete. 2024-09-19T05:05:55.120 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:59:34.802774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:04:04.880203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:04:06.925227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:05:55.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T05:05:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 -- 172.21.15.64:0/701958014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101bf0 msgr2=0x7fd684102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 --2- 172.21.15.64:0/701958014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101bf0 0x7fd684102070 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd670009900 tx=0x7fd67002a410 comp rx=0 tx=0).stop 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 -- 172.21.15.64:0/701958014 shutdown_connections 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 --2- 172.21.15.64:0/701958014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6841025b0 0x7fd684108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 --2- 172.21.15.64:0/701958014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101bf0 0x7fd684102070 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 --2- 172.21.15.64:0/701958014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd684100a00 0x7fd684100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 -- 172.21.15.64:0/701958014 >> 172.21.15.64:0/701958014 conn(0x7fd6840fc140 msgr2=0x7fd6840fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 -- 172.21.15.64:0/701958014 shutdown_connections 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.229+0000 7fd68ab66640 1 -- 172.21.15.64:0/701958014 wait complete. 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 Processor -- start 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 -- start start 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 0x7fd684197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd684101bf0 0x7fd684198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6841025b0 0x7fd68419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410c530 con 0x7fd684100a00 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410c3b0 con 0x7fd684101bf0 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd68ab66640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410c6b0 con 0x7fd6841025b0 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 0x7fd684197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 0x7fd684197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42424/0 (socket says 172.21.15.64:42424) 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 -- 172.21.15.64:0/3773947251 learned_addr learned my addr 172.21.15.64:0/3773947251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd689103640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6841025b0 0x7fd68419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd683fff640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd684101bf0 0x7fd684198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 -- 172.21.15.64:0/3773947251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6841025b0 msgr2=0x7fd68419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6841025b0 0x7fd68419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 -- 172.21.15.64:0/3773947251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd684101bf0 msgr2=0x7fd684198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd684101bf0 0x7fd684198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd688902640 1 -- 172.21.15.64:0/3773947251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68419fa10 con 0x7fd684100a00 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.230+0000 7fd683fff640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd684101bf0 0x7fd684198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.231+0000 7fd688902640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 0x7fd684197df0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9c0 tx=0x7fd67800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.231+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800cd80 con 0x7fd684100a00 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.231+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68419fd00 con 0x7fd684100a00 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.231+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6780035f0 con 0x7fd684100a00 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.231+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678010690 con 0x7fd684100a00 2024-09-19T05:05:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.231+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841a01a0 con 0x7fd684100a00 2024-09-19T05:05:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.232+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6841a0430 con 0x7fd684100a00 2024-09-19T05:05:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.233+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7fd6780037a0 con 0x7fd684100a00 2024-09-19T05:05:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.234+0000 7fd681ffb640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd6740a3050 0x7fd6740a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.234+0000 7fd683fff640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd6740a3050 0x7fd6740a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.234+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fd6780cfc10 con 0x7fd684100a00 2024-09-19T05:05:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.234+0000 7fd683fff640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd6740a3050 0x7fd6740a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd670006010 tx=0x7fd670000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:55.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.242+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd67809cb00 con 0x7fd684100a00 2024-09-19T05:05:55.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.429+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6841a0600 con 0x7fd684100a00 2024-09-19T05:05:55.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.430+0000 7fd681ffb640 1 -- 172.21.15.64:0/3773947251 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fd67809c400 con 0x7fd684100a00 2024-09-19T05:05:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-19T05:05:55.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd6740a3050 msgr2=0x7fd6740a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd6740a3050 0x7fd6740a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd670006010 tx=0x7fd670000bc0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 msgr2=0x7fd684197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 0x7fd684197df0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9c0 tx=0x7fd67800ee80 comp rx=0 tx=0).stop 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 shutdown_connections 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd6740a3050 0x7fd6740a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6841025b0 0x7fd68419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd684101bf0 0x7fd684198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 --2- 172.21.15.64:0/3773947251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684100a00 0x7fd684197df0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 >> 172.21.15.64:0/3773947251 conn(0x7fd6840fc140 msgr2=0x7fd6840693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 shutdown_connections 2024-09-19T05:05:55.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.432+0000 7fd68ab66640 1 -- 172.21.15.64:0/3773947251 wait complete. 2024-09-19T05:05:55.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T05:05:55.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 -- 172.21.15.64:0/3496998234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 msgr2=0x7f78f0109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 --2- 172.21.15.64:0/3496998234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f0109c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009ca0 tx=0x7f78e802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 -- 172.21.15.64:0/3496998234 shutdown_connections 2024-09-19T05:05:55.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 --2- 172.21.15.64:0/3496998234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f0109c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 --2- 172.21.15.64:0/3496998234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f0104ee0 0x7f78f01072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 --2- 172.21.15.64:0/3496998234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f01025b0 0x7f78f01049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 -- 172.21.15.64:0/3496998234 >> 172.21.15.64:0/3496998234 conn(0x7f78f00fc1b0 msgr2=0x7f78f00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:55.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 -- 172.21.15.64:0/3496998234 shutdown_connections 2024-09-19T05:05:55.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 -- 172.21.15.64:0/3496998234 wait complete. 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.568+0000 7f78f56d0640 1 Processor -- start 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 -- start start 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 0x7f78f0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f0104ee0 0x7f78f0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78f010d680 con 0x7f78f01025b0 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78f010d500 con 0x7f78f0107810 2024-09-19T05:05:55.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78f56d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78f010d800 con 0x7f78f0104ee0 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 0x7f78f0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 0x7f78f0197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42446/0 (socket says 172.21.15.64:42446) 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 -- 172.21.15.64:0/632672127 learned_addr learned my addr 172.21.15.64:0/632672127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78ee7fc640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f0104ee0 0x7f78f0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 -- 172.21.15.64:0/632672127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f0104ee0 msgr2=0x7f78f0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f0104ee0 0x7f78f0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 -- 172.21.15.64:0/632672127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 msgr2=0x7f78f019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 -- 172.21.15.64:0/632672127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f019f9d0 con 0x7f78f01025b0 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78ef7fe640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:55.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.569+0000 7f78eeffd640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 0x7f78f0197dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c890 tx=0x7f78e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.570+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0004220 con 0x7f78f01025b0 2024-09-19T05:05:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.570+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e00043c0 con 0x7f78f01025b0 2024-09-19T05:05:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.570+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f019fcc0 con 0x7f78f01025b0 2024-09-19T05:05:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.570+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0004a00 con 0x7f78f01025b0 2024-09-19T05:05:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.570+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f01a01c0 con 0x7f78f01025b0 2024-09-19T05:05:55.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.571+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f00fdca0 con 0x7f78f01025b0 2024-09-19T05:05:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.572+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 138167+0+0 (secure 0 0 0) 0x7f78e0004ba0 con 0x7f78f01025b0 2024-09-19T05:05:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.573+0000 7f78d7fff640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78d80a3050 0x7f78d80a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.573+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f78e00cf2d0 con 0x7f78f01025b0 2024-09-19T05:05:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.573+0000 7f78ee7fc640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78d80a3050 0x7f78d80a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.574+0000 7f78ee7fc640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78d80a3050 0x7f78d80a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78f006aae0 tx=0x7f78e4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:55.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.578+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f78e0014030 con 0x7f78f01025b0 2024-09-19T05:05:55.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:55 smithi064 ceph-mon[30960]: mgrmap e21: x(active, since 1.04697s), standbys: z, y 2024-09-19T05:05:55.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:55 smithi064 ceph-mon[30960]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:55.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.738+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 22) ==== 138526+0+0 (secure 0 0 0) 0x7f78e0098150 con 0x7f78f01025b0 2024-09-19T05:05:55.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.809+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f78f0062df0 con 0x7f78f01025b0 2024-09-19T05:05:55.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.810+0000 7f78d7fff640 1 -- 172.21.15.64:0/632672127 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f78e00578f0 con 0x7f78f01025b0 2024-09-19T05:05:55.811 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78d80a3050 msgr2=0x7f78d80a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78d80a3050 0x7f78d80a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78f006aae0 tx=0x7f78e4005f90 comp rx=0 tx=0).stop 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 msgr2=0x7f78f0197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 0x7f78f0197dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c890 tx=0x7f78e000cd50 comp rx=0 tx=0).stop 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 shutdown_connections 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78d80a3050 0x7f78d80a5510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78f0107810 0x7f78f019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f0104ee0 0x7f78f0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 --2- 172.21.15.64:0/632672127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78f01025b0 0x7f78f0197dc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 >> 172.21.15.64:0/632672127 conn(0x7f78f00fc1b0 msgr2=0x7f78f0102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:55.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 shutdown_connections 2024-09-19T05:05:55.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.812+0000 7f78f56d0640 1 -- 172.21.15.64:0/632672127 wait complete. 2024-09-19T05:05:55.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T05:05:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 -- 172.21.15.64:0/1943293623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520102d10 msgr2=0x7fb520103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 --2- 172.21.15.64:0/1943293623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520102d10 0x7fb520103190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb514009900 tx=0x7fb51402a410 comp rx=0 tx=0).stop 2024-09-19T05:05:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 -- 172.21.15.64:0/1943293623 shutdown_connections 2024-09-19T05:05:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 --2- 172.21.15.64:0/1943293623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5201036d0 0x7fb520109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 --2- 172.21.15.64:0/1943293623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520102d10 0x7fb520103190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 --2- 172.21.15.64:0/1943293623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520101b20 0x7fb520101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.935+0000 7fb525986640 1 -- 172.21.15.64:0/1943293623 >> 172.21.15.64:0/1943293623 conn(0x7fb5200fd2a0 msgr2=0x7fb5200ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 -- 172.21.15.64:0/1943293623 shutdown_connections 2024-09-19T05:05:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 -- 172.21.15.64:0/1943293623 wait complete. 2024-09-19T05:05:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 Processor -- start 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 -- start start 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 0x7fb520075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520102d10 0x7fb5200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5201036d0 0x7fb520074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010d680 con 0x7fb520101b20 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010d500 con 0x7fb520102d10 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.936+0000 7fb525986640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010d800 con 0x7fb5201036d0 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 0x7fb520075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 0x7fb520075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42472/0 (socket says 172.21.15.64:42472) 2024-09-19T05:05:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 -- 172.21.15.64:0/3945423921 learned_addr learned my addr 172.21.15.64:0/3945423921 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51e7fc640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520102d10 0x7fb5200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51f7fe640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5201036d0 0x7fb520074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 -- 172.21.15.64:0/3945423921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5201036d0 msgr2=0x7fb520074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5201036d0 0x7fb520074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 -- 172.21.15.64:0/3945423921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520102d10 msgr2=0x7fb5200745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520102d10 0x7fb5200745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 -- 172.21.15.64:0/3945423921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5200753b0 con 0x7fb520101b20 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51e7fc640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520102d10 0x7fb5200745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb51effd640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 0x7fb520075fa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e9c0 tx=0x7fb51000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51000cd80 con 0x7fb520101b20 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5100035f0 con 0x7fb520101b20 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5201a1e90 con 0x7fb520101b20 2024-09-19T05:05:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.937+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510010690 con 0x7fb520101b20 2024-09-19T05:05:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.938+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5201a2290 con 0x7fb520101b20 2024-09-19T05:05:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.939+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb520109000 con 0x7fb520101b20 2024-09-19T05:05:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.940+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 138526+0+0 (secure 0 0 0) 0x7fb5100037a0 con 0x7fb520101b20 2024-09-19T05:05:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.940+0000 7fb524984640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb5080a3450 0x7fb5080a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.941+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb5100d04e0 con 0x7fb520101b20 2024-09-19T05:05:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.941+0000 7fb51e7fc640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb5080a3450 0x7fb5080a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.941+0000 7fb51e7fc640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb5080a3450 0x7fb5080a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5140314e0 tx=0x7fb514005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:55.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:55.944+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb51009cd50 con 0x7fb520101b20 2024-09-19T05:05:56.004 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:55 smithi064 ceph-mon[30960]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:05:56.004 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:55 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/646120580' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:56.004 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:55 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3773947251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:05:56.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:55 smithi174 ceph-mon[34488]: mgrmap e21: x(active, since 1.04697s), standbys: z, y 2024-09-19T05:05:56.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:55 smithi174 ceph-mon[34488]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:56.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:55 smithi174 ceph-mon[34488]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:05:56.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:55 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/646120580' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:56.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:55 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3773947251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:05:56.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.124+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb520062e90 con 0x7fb520101b20 2024-09-19T05:05:56.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.124+0000 7fb524984640 1 -- 172.21.15.64:0/3945423921 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fb51009c650 con 0x7fb520101b20 2024-09-19T05:05:56.127 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb5080a3450 msgr2=0x7fb5080a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb5080a3450 0x7fb5080a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5140314e0 tx=0x7fb514005a20 comp rx=0 tx=0).stop 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 msgr2=0x7fb520075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 0x7fb520075fa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e9c0 tx=0x7fb51000ee80 comp rx=0 tx=0).stop 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 shutdown_connections 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb5080a3450 0x7fb5080a5910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5201036d0 0x7fb520074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb520102d10 0x7fb5200745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 --2- 172.21.15.64:0/3945423921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520101b20 0x7fb520075fa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.126+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 >> 172.21.15.64:0/3945423921 conn(0x7fb5200fd2a0 msgr2=0x7fb5201061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.127+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 shutdown_connections 2024-09-19T05:05:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.127+0000 7fb525986640 1 -- 172.21.15.64:0/3945423921 wait complete. 2024-09-19T05:05:56.140 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T05:05:56.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:56.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:55 smithi103 ceph-mon[34446]: mgrmap e21: x(active, since 1.04697s), standbys: z, y 2024-09-19T05:05:56.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:55 smithi103 ceph-mon[34446]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:05:56.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:55 smithi103 ceph-mon[34446]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:05:56.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:55 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/646120580' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:05:56.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:55 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3773947251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:56.001+0000 7fdbe688d640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:05:05:56] ENGINE Error in HTTPServer.serve 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: Traceback (most recent call last): 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._connections.run(self.expiration_interval) 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._run(expiration_interval) 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:05:56.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: s = self.context.wrap_socket( 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: return self.sslsocket_class._create( 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self.do_handshake() 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: self._sslobj.do_handshake() 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:05:56.402 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:05:56.426 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:56.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-19T05:05:56.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: [19/Sep/2024:05:05:55] ENGINE Bus STARTING 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/632672127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: [19/Sep/2024:05:05:55] ENGINE Serving on http://172.21.15.64:8765 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: [19/Sep/2024:05:05:56] ENGINE Serving on https://172.21.15.64:7150 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: [19/Sep/2024:05:05:56] ENGINE Bus STARTED 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: [19/Sep/2024:05:05:56] ENGINE Error in HTTPServer.serve 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:05:56.830 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:05:56.831 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3945423921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: Unable to set osd_memory_target on smithi174 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:56.833 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:56.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:56.834 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:56 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/820090800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d010c160 msgr2=0x7fd9d010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:56.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/820090800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d010c160 0x7fd9d010e5f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009ca0 tx=0x7fd9c802a970 comp rx=0 tx=0).stop 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/820090800 shutdown_connections 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/820090800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d010c160 0x7fd9d010e5f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/820090800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d01036e0 0x7fd9d010bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/820090800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d0102dc0 0x7fd9d01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/820090800 >> 172.21.15.64:0/820090800 conn(0x7fd9d00fc780 msgr2=0x7fd9d00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/820090800 shutdown_connections 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.914+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/820090800 wait complete. 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.915+0000 7fd9d7d7f640 1 Processor -- start 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.915+0000 7fd9d7d7f640 1 -- start start 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.915+0000 7fd9d7d7f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d0102dc0 0x7fd9d01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:56.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.915+0000 7fd9d7d7f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d01036e0 0x7fd9d0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.915+0000 7fd9d7d7f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 0x7fd9d019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d7d7f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111e60 con 0x7fd9d01036e0 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d7d7f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111ce0 con 0x7fd9d010c160 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d7d7f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111fe0 con 0x7fd9d0102dc0 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 0x7fd9d019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d531a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d01036e0 0x7fd9d0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 0x7fd9d019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:46528/0 (socket says 172.21.15.64:46528) 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 -- 172.21.15.64:0/1040229193 learned_addr learned my addr 172.21.15.64:0/1040229193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d5b1b640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d0102dc0 0x7fd9d01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 -- 172.21.15.64:0/1040229193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d0102dc0 msgr2=0x7fd9d01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d0102dc0 0x7fd9d01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 -- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d01036e0 msgr2=0x7fd9d0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d01036e0 0x7fd9d0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 -- 172.21.15.64:0/1040229193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d019d370 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.916+0000 7fd9d631c640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 0x7fd9d019cc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8004af0 tx=0x7fd9c802e390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.917+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c804f070 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.917+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c8030b20 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.917+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d019d600 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.917+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8034050 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.917+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d0100950 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.918+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d006afd0 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.919+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 22) ==== 138526+0+0 (secure 0 0 0) 0x7fd9c8052070 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.919+0000 7fd9c6ffd640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9bc0a3440 0x7fd9bc0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.920+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fd9c80f9250 con 0x7fd9d010c160 2024-09-19T05:05:56.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.920+0000 7fd9d5b1b640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9bc0a3440 0x7fd9bc0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:56.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.920+0000 7fd9d5b1b640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9bc0a3440 0x7fd9bc0a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc0090e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:56.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:56.923+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9c80c5470 con 0x7fd9d010c160 2024-09-19T05:05:57.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: [19/Sep/2024:05:05:55] ENGINE Bus STARTING 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/632672127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: [19/Sep/2024:05:05:55] ENGINE Serving on http://172.21.15.64:8765 2024-09-19T05:05:57.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: [19/Sep/2024:05:05:56] ENGINE Serving on https://172.21.15.64:7150 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: [19/Sep/2024:05:05:56] ENGINE Bus STARTED 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: [19/Sep/2024:05:05:56] ENGINE Error in HTTPServer.serve 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: Traceback (most recent call last): 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._run(expiration_interval) 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:05:57.029 INFO:journalctl@ceph.mon.c.smithi174.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: s = self.context.wrap_socket( 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: return self.sslsocket_class._create( 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: self.do_handshake() 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._sslobj.do_handshake() 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3945423921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: Unable to set osd_memory_target on smithi174 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T05:05:57.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:57.032 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:56 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:57.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.164+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d0107220 con 0x7fd9d010c160 2024-09-19T05:05:57.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.165+0000 7fd9c6ffd640 1 -- 172.21.15.64:0/1040229193 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fd9c80c9770 con 0x7fd9d010c160 2024-09-19T05:05:57.166 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:57.167 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:05:57.167 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9bc0a3440 msgr2=0x7fd9bc0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9bc0a3440 0x7fd9bc0a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc0090e0 comp rx=0 tx=0).stop 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 msgr2=0x7fd9d019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 0x7fd9d019cc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8004af0 tx=0x7fd9c802e390 comp rx=0 tx=0).stop 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 shutdown_connections 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9bc0a3440 0x7fd9bc0a5900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9d010c160 0x7fd9d019cc90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9d01036e0 0x7fd9d0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 --2- 172.21.15.64:0/1040229193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d0102dc0 0x7fd9d01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 >> 172.21.15.64:0/1040229193 conn(0x7fd9d00fc780 msgr2=0x7fd9d00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 shutdown_connections 2024-09-19T05:05:57.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:57.167+0000 7fd9d7d7f640 1 -- 172.21.15.64:0/1040229193 wait complete. 2024-09-19T05:05:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-19T05:05:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: [19/Sep/2024:05:05:55] ENGINE Bus STARTING 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/632672127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: [19/Sep/2024:05:05:55] ENGINE Serving on http://172.21.15.64:8765 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: [19/Sep/2024:05:05:56] ENGINE Serving on https://172.21.15.64:7150 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: [19/Sep/2024:05:05:56] ENGINE Bus STARTED 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: [19/Sep/2024:05:05:56] ENGINE Error in HTTPServer.serve 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout: Traceback (most recent call last): 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._run(expiration_interval) 2024-09-19T05:05:57.220 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: self.do_handshake() 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T05:05:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3945423921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T05:05:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: Unable to set osd_memory_target on smithi174 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:57.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:56 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T05:05:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1040229193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:57.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:57.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:57 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T05:05:58.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-19T05:05:58.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T05:05:58.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1040229193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:57 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.093 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T05:05:58.093 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T05:05:58.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:05:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.197+0000 7f5920c30640 1 -- 172.21.15.64:0/2370774064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c101320 msgr2=0x7f591c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.197+0000 7f5920c30640 1 --2- 172.21.15.64:0/2370774064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c101320 0x7f591c10d850 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5914009cd0 tx=0x7f591402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 -- 172.21.15.64:0/2370774064 shutdown_connections 2024-09-19T05:05:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 --2- 172.21.15.64:0/2370774064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c101320 0x7f591c10d850 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 --2- 172.21.15.64:0/2370774064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c100980 0x7f591c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 --2- 172.21.15.64:0/2370774064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c106990 0x7f591c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 -- 172.21.15.64:0/2370774064 >> 172.21.15.64:0/2370774064 conn(0x7f591c077a10 msgr2=0x7f591c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 -- 172.21.15.64:0/2370774064 shutdown_connections 2024-09-19T05:05:58.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 -- 172.21.15.64:0/2370774064 wait complete. 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 Processor -- start 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.198+0000 7f5920c30640 1 -- start start 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c100980 0x7f591c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c101320 0x7f591c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 0x7f591c1a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f591c110c80 con 0x7f591c106990 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f591c110b00 con 0x7f591c101320 2024-09-19T05:05:58.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f591c110e00 con 0x7f591c100980 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 0x7f591c1a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 0x7f591c1a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42500/0 (socket says 172.21.15.64:42500) 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 -- 172.21.15.64:0/2163374070 learned_addr learned my addr 172.21.15.64:0/2163374070 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591a59c640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c100980 0x7f591c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5919d9b640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c101320 0x7f591c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 -- 172.21.15.64:0/2163374070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c100980 msgr2=0x7f591c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c100980 0x7f591c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 -- 172.21.15.64:0/2163374070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c101320 msgr2=0x7f591c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c101320 0x7f591c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 -- 172.21.15.64:0/2163374070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c1a46f0 con 0x7f591c106990 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591a59c640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c100980 0x7f591c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5919d9b640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c101320 0x7f591c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:58.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f591ad9d640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 0x7f591c1a41b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5914033040 tx=0x7f5914003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914004040 con 0x7f591c106990 2024-09-19T05:05:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1a48c0 con 0x7f591c106990 2024-09-19T05:05:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5914004690 con 0x7f591c106990 2024-09-19T05:05:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914030e10 con 0x7f591c106990 2024-09-19T05:05:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.199+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a4dd0 con 0x7f591c106990 2024-09-19T05:05:58.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.200+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c100de0 con 0x7f591c106990 2024-09-19T05:05:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.202+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 138526+0+0 (secure 0 0 0) 0x7f59140305a0 con 0x7f591c106990 2024-09-19T05:05:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.202+0000 7f590b7fe640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59040a3450 0x7f59040a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.202+0000 7f591a59c640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59040a3450 0x7f59040a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.203+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f591402e030 con 0x7f591c106990 2024-09-19T05:05:58.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.203+0000 7f591a59c640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59040a3450 0x7f59040a5910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f590c005fd0 tx=0x7f590c00b5d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:58.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.205+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f591404b060 con 0x7f591c106990 2024-09-19T05:05:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T05:05:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-19T05:05:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-19T05:05:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T05:05:58.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1040229193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: Updating smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:57 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:58.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.379+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f591c108f80 con 0x7f59040a3450 2024-09-19T05:05:58.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.386+0000 7f590b7fe640 1 -- 172.21.15.64:0/2163374070 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257853 (secure 0 0 0) 0x7f590c00fb60 con 0x7f59040a3450 2024-09-19T05:05:58.390 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:58.391 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:05:58.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59040a3450 msgr2=0x7f59040a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59040a3450 0x7f59040a5910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f590c005fd0 tx=0x7f590c00b5d0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 msgr2=0x7f591c1a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 0x7f591c1a41b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5914033040 tx=0x7f5914003970 comp rx=0 tx=0).stop 2024-09-19T05:05:58.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 shutdown_connections 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59040a3450 0x7f59040a5910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c106990 0x7f591c1a41b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f591c101320 0x7f591c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 --2- 172.21.15.64:0/2163374070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f591c100980 0x7f591c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 >> 172.21.15.64:0/2163374070 conn(0x7f591c077a10 msgr2=0x7f591c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 shutdown_connections 2024-09-19T05:05:58.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.389+0000 7f5920c30640 1 -- 172.21.15.64:0/2163374070 wait complete. 2024-09-19T05:05:58.402 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T05:05:57.703956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189099,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":845,"num_read_kb":584,"num_write":478,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367548,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574276,"statfs":{"total":1151940427776,"available":1151564058624,"internally_reserved":0,"allocated":14385152,"data_stored":8731554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.967014"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689413+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:05:53.689413+0000","last_peered":"2024-09-19T05:05:53.689413+0000","last_clean":"2024-09-19T05:05:53.689413+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:05:53.689413+0000","last_undegraded":"2024-09-19T05:05:53.689413+0000","last_fullsized":"2024-09-19T05:05:53.689413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973308+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:05:53.973308+0000","last_peered":"2024-09-19T05:05:53.973308+0000","last_clean":"2024-09-19T05:05:53.973308+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:05:53.973308+0000","last_undegraded":"2024-09-19T05:05:53.973308+0000","last_fullsized":"2024-09-19T05:05:53.973308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972992+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:05:53.972992+0000","last_peered":"2024-09-19T05:05:53.972992+0000","last_clean":"2024-09-19T05:05:53.972992+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:05:53.972992+0000","last_undegraded":"2024-09-19T05:05:53.972992+0000","last_fullsized":"2024-09-19T05:05:53.972992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946819+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:05:53.946819+0000","last_peered":"2024-09-19T05:05:53.946819+0000","last_clean":"2024-09-19T05:05:53.946819+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:05:53.946819+0000","last_undegraded":"2024-09-19T05:05:53.946819+0000","last_fullsized":"2024-09-19T05:05:53.946819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783606+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:05:53.783606+0000","last_peered":"2024-09-19T05:05:53.783606+0000","last_clean":"2024-09-19T05:05:53.783606+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:05:53.783606+0000","last_undegraded":"2024-09-19T05:05:53.783606+0000","last_fullsized":"2024-09-19T05:05:53.783606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691370+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:05:53.691370+0000","last_peered":"2024-09-19T05:05:53.691370+0000","last_clean":"2024-09-19T05:05:53.691370+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:05:53.691370+0000","last_undegraded":"2024-09-19T05:05:53.691370+0000","last_fullsized":"2024-09-19T05:05:53.691370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945667+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:05:53.945667+0000","last_peered":"2024-09-19T05:05:53.945667+0000","last_clean":"2024-09-19T05:05:53.945667+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:05:53.945667+0000","last_undegraded":"2024-09-19T05:05:53.945667+0000","last_fullsized":"2024-09-19T05:05:53.945667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689751+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:05:53.689751+0000","last_peered":"2024-09-19T05:05:53.689751+0000","last_clean":"2024-09-19T05:05:53.689751+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:05:53.689751+0000","last_undegraded":"2024-09-19T05:05:53.689751+0000","last_fullsized":"2024-09-19T05:05:53.689751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948971+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:05:53.948971+0000","last_peered":"2024-09-19T05:05:53.948971+0000","last_clean":"2024-09-19T05:05:53.948971+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:05:53.948971+0000","last_undegraded":"2024-09-19T05:05:53.948971+0000","last_fullsized":"2024-09-19T05:05:53.948971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691740+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:05:53.691740+0000","last_peered":"2024-09-19T05:05:53.691740+0000","last_clean":"2024-09-19T05:05:53.691740+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:05:53.691740+0000","last_undegraded":"2024-09-19T05:05:53.691740+0000","last_fullsized":"2024-09-19T05:05:53.691740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946061+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:05:53.946061+0000","last_peered":"2024-09-19T05:05:53.946061+0000","last_clean":"2024-09-19T05:05:53.946061+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:05:53.946061+0000","last_undegraded":"2024-09-19T05:05:53.946061+0000","last_fullsized":"2024-09-19T05:05:53.946061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946960+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:05:53.946960+0000","last_peered":"2024-09-19T05:05:53.946960+0000","last_clean":"2024-09-19T05:05:53.946960+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:05:53.946960+0000","last_undegraded":"2024-09-19T05:05:53.946960+0000","last_fullsized":"2024-09-19T05:05:53.946960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971919+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:05:53.971919+0000","last_peered":"2024-09-19T05:05:53.971919+0000","last_clean":"2024-09-19T05:05:53.971919+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:05:53.971919+0000","last_undegraded":"2024-09-19T05:05:53.971919+0000","last_fullsized":"2024-09-19T05:05:53.971919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972311+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:05:53.972311+0000","last_peered":"2024-09-19T05:05:53.972311+0000","last_clean":"2024-09-19T05:05:53.972311+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:05:53.972311+0000","last_undegraded":"2024-09-19T05:05:53.972311+0000","last_fullsized":"2024-09-19T05:05:53.972311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972019+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:05:53.972019+0000","last_peered":"2024-09-19T05:05:53.972019+0000","last_clean":"2024-09-19T05:05:53.972019+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:05:53.972019+0000","last_undegraded":"2024-09-19T05:05:53.972019+0000","last_fullsized":"2024-09-19T05:05:53.972019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948575+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:05:53.948575+0000","last_peered":"2024-09-19T05:05:53.948575+0000","last_clean":"2024-09-19T05:05:53.948575+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:05:53.948575+0000","last_undegraded":"2024-09-19T05:05:53.948575+0000","last_fullsized":"2024-09-19T05:05:53.948575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689453+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:05:53.689453+0000","last_peered":"2024-09-19T05:05:53.689453+0000","last_clean":"2024-09-19T05:05:53.689453+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:05:53.689453+0000","last_undegraded":"2024-09-19T05:05:53.689453+0000","last_fullsized":"2024-09-19T05:05:53.689453+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781607+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:05:53.781607+0000","last_peered":"2024-09-19T05:05:53.781607+0000","last_clean":"2024-09-19T05:05:53.781607+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:05:53.781607+0000","last_undegraded":"2024-09-19T05:05:53.781607+0000","last_fullsized":"2024-09-19T05:05:53.781607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945659+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:05:53.945659+0000","last_peered":"2024-09-19T05:05:53.945659+0000","last_clean":"2024-09-19T05:05:53.945659+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:05:53.945659+0000","last_undegraded":"2024-09-19T05:05:53.945659+0000","last_fullsized":"2024-09-19T05:05:53.945659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689033+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:05:53.689033+0000","last_peered":"2024-09-19T05:05:53.689033+0000","last_clean":"2024-09-19T05:05:53.689033+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:05:53.689033+0000","last_undegraded":"2024-09-19T05:05:53.689033+0000","last_fullsized":"2024-09-19T05:05:53.689033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972075+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:05:53.972075+0000","last_peered":"2024-09-19T05:05:53.972075+0000","last_clean":"2024-09-19T05:05:53.972075+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:05:53.972075+0000","last_undegraded":"2024-09-19T05:05:53.972075+0000","last_fullsized":"2024-09-19T05:05:53.972075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948706+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:05:53.948706+0000","last_peered":"2024-09-19T05:05:53.948706+0000","last_clean":"2024-09-19T05:05:53.948706+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:05:53.948706+0000","last_undegraded":"2024-09-19T05:05:53.948706+0000","last_fullsized":"2024-09-19T05:05:53.948706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689785+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:05:53.689785+0000","last_peered":"2024-09-19T05:05:53.689785+0000","last_clean":"2024-09-19T05:05:53.689785+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:05:53.689785+0000","last_undegraded":"2024-09-19T05:05:53.689785+0000","last_fullsized":"2024-09-19T05:05:53.689785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789295+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:05:53.789295+0000","last_peered":"2024-09-19T05:05:53.789295+0000","last_clean":"2024-09-19T05:05:53.789295+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:05:53.789295+0000","last_undegraded":"2024-09-19T05:05:53.789295+0000","last_fullsized":"2024-09-19T05:05:53.789295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781636+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:05:53.781636+0000","last_peered":"2024-09-19T05:05:53.781636+0000","last_clean":"2024-09-19T05:05:53.781636+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:05:53.781636+0000","last_undegraded":"2024-09-19T05:05:53.781636+0000","last_fullsized":"2024-09-19T05:05:53.781636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945753+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:05:53.945753+0000","last_peered":"2024-09-19T05:05:53.945753+0000","last_clean":"2024-09-19T05:05:53.945753+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:05:53.945753+0000","last_undegraded":"2024-09-19T05:05:53.945753+0000","last_fullsized":"2024-09-19T05:05:53.945753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689249+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:05:53.689249+0000","last_peered":"2024-09-19T05:05:53.689249+0000","last_clean":"2024-09-19T05:05:53.689249+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:05:53.689249+0000","last_undegraded":"2024-09-19T05:05:53.689249+0000","last_fullsized":"2024-09-19T05:05:53.689249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971123+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:05:53.971123+0000","last_peered":"2024-09-19T05:05:53.971123+0000","last_clean":"2024-09-19T05:05:53.971123+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:05:53.971123+0000","last_undegraded":"2024-09-19T05:05:53.971123+0000","last_fullsized":"2024-09-19T05:05:53.971123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972635+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:05:53.972635+0000","last_peered":"2024-09-19T05:05:53.972635+0000","last_clean":"2024-09-19T05:05:53.972635+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:05:53.972635+0000","last_undegraded":"2024-09-19T05:05:53.972635+0000","last_fullsized":"2024-09-19T05:05:53.972635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945375+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:05:53.945375+0000","last_peered":"2024-09-19T05:05:53.945375+0000","last_clean":"2024-09-19T05:05:53.945375+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:05:53.945375+0000","last_undegraded":"2024-09-19T05:05:53.945375+0000","last_fullsized":"2024-09-19T05:05:53.945375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945424+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:05:53.945424+0000","last_peered":"2024-09-19T05:05:53.945424+0000","last_clean":"2024-09-19T05:05:53.945424+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:05:53.945424+0000","last_undegraded":"2024-09-19T05:05:53.945424+0000","last_fullsized":"2024-09-19T05:05:53.945424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689079+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:05:53.689079+0000","last_peered":"2024-09-19T05:05:53.689079+0000","last_clean":"2024-09-19T05:05:53.689079+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:05:53.689079+0000","last_undegraded":"2024-09-19T05:05:53.689079+0000","last_fullsized":"2024-09-19T05:05:53.689079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783659+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:05:53.783659+0000","last_peered":"2024-09-19T05:05:53.783659+0000","last_clean":"2024-09-19T05:05:53.783659+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:05:53.783659+0000","last_undegraded":"2024-09-19T05:05:53.783659+0000","last_fullsized":"2024-09-19T05:05:53.783659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'9","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.163491+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:05:54.163491+0000","last_peered":"2024-09-19T05:05:54.163491+0000","last_clean":"2024-09-19T05:05:54.163491+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:05:54.163491+0000","last_undegraded":"2024-09-19T05:05:54.163491+0000","last_fullsized":"2024-09-19T05:05:54.163491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781306+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:05:53.781306+0000","last_peered":"2024-09-19T05:05:53.781306+0000","last_clean":"2024-09-19T05:05:53.781306+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:05:53.781306+0000","last_undegraded":"2024-09-19T05:05:53.781306+0000","last_fullsized":"2024-09-19T05:05:53.781306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691365+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:05:53.691365+0000","last_peered":"2024-09-19T05:05:53.691365+0000","last_clean":"2024-09-19T05:05:53.691365+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:05:53.691365+0000","last_undegraded":"2024-09-19T05:05:53.691365+0000","last_fullsized":"2024-09-19T05:05:53.691365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971411+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:05:53.971411+0000","last_peered":"2024-09-19T05:05:53.971411+0000","last_clean":"2024-09-19T05:05:53.971411+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:05:53.971411+0000","last_undegraded":"2024-09-19T05:05:53.971411+0000","last_fullsized":"2024-09-19T05:05:53.971411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689842+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:05:53.689842+0000","last_peered":"2024-09-19T05:05:53.689842+0000","last_clean":"2024-09-19T05:05:53.689842+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:05:53.689842+0000","last_undegraded":"2024-09-19T05:05:53.689842+0000","last_fullsized":"2024-09-19T05:05:53.689842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789504+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:05:53.789504+0000","last_peered":"2024-09-19T05:05:53.789504+0000","last_clean":"2024-09-19T05:05:53.789504+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:05:53.789504+0000","last_undegraded":"2024-09-19T05:05:53.789504+0000","last_fullsized":"2024-09-19T05:05:53.789504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781090+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:05:53.781090+0000","last_peered":"2024-09-19T05:05:53.781090+0000","last_clean":"2024-09-19T05:05:53.781090+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:05:53.781090+0000","last_undegraded":"2024-09-19T05:05:53.781090+0000","last_fullsized":"2024-09-19T05:05:53.781090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691578+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:05:53.691578+0000","last_peered":"2024-09-19T05:05:53.691578+0000","last_clean":"2024-09-19T05:05:53.691578+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:05:53.691578+0000","last_undegraded":"2024-09-19T05:05:53.691578+0000","last_fullsized":"2024-09-19T05:05:53.691578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948853+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:05:53.948853+0000","last_peered":"2024-09-19T05:05:53.948853+0000","last_clean":"2024-09-19T05:05:53.948853+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:05:53.948853+0000","last_undegraded":"2024-09-19T05:05:53.948853+0000","last_fullsized":"2024-09-19T05:05:53.948853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946143+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:05:53.946143+0000","last_peered":"2024-09-19T05:05:53.946143+0000","last_clean":"2024-09-19T05:05:53.946143+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:05:53.946143+0000","last_undegraded":"2024-09-19T05:05:53.946143+0000","last_fullsized":"2024-09-19T05:05:53.946143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973668+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:05:53.973668+0000","last_peered":"2024-09-19T05:05:53.973668+0000","last_clean":"2024-09-19T05:05:53.973668+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:05:53.973668+0000","last_undegraded":"2024-09-19T05:05:53.973668+0000","last_fullsized":"2024-09-19T05:05:53.973668+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789562+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:05:53.789562+0000","last_peered":"2024-09-19T05:05:53.789562+0000","last_clean":"2024-09-19T05:05:53.789562+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:05:53.789562+0000","last_undegraded":"2024-09-19T05:05:53.789562+0000","last_fullsized":"2024-09-19T05:05:53.789562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948706+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:05:53.948706+0000","last_peered":"2024-09-19T05:05:53.948706+0000","last_clean":"2024-09-19T05:05:53.948706+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:05:53.948706+0000","last_undegraded":"2024-09-19T05:05:53.948706+0000","last_fullsized":"2024-09-19T05:05:53.948706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'37","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.166232+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:05:54.166232+0000","last_peered":"2024-09-19T05:05:54.166232+0000","last_clean":"2024-09-19T05:05:54.166232+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:05:54.166232+0000","last_undegraded":"2024-09-19T05:05:54.166232+0000","last_fullsized":"2024-09-19T05:05:54.166232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67761,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":169,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783871+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:05:53.783871+0000","last_peered":"2024-09-19T05:05:53.783871+0000","last_clean":"2024-09-19T05:05:53.783871+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:05:53.783871+0000","last_undegraded":"2024-09-19T05:05:53.783871+0000","last_fullsized":"2024-09-19T05:05:53.783871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689009+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:05:53.689009+0000","last_peered":"2024-09-19T05:05:53.689009+0000","last_clean":"2024-09-19T05:05:53.689009+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:05:53.689009+0000","last_undegraded":"2024-09-19T05:05:53.689009+0000","last_fullsized":"2024-09-19T05:05:53.689009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971016+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:05:53.971016+0000","last_peered":"2024-09-19T05:05:53.971016+0000","last_clean":"2024-09-19T05:05:53.971016+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:05:53.971016+0000","last_undegraded":"2024-09-19T05:05:53.971016+0000","last_fullsized":"2024-09-19T05:05:53.971016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789394+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:05:53.789394+0000","last_peered":"2024-09-19T05:05:53.789394+0000","last_clean":"2024-09-19T05:05:53.789394+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:05:53.789394+0000","last_undegraded":"2024-09-19T05:05:53.789394+0000","last_fullsized":"2024-09-19T05:05:53.789394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945962+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:05:53.945962+0000","last_peered":"2024-09-19T05:05:53.945962+0000","last_clean":"2024-09-19T05:05:53.945962+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:05:53.945962+0000","last_undegraded":"2024-09-19T05:05:53.945962+0000","last_fullsized":"2024-09-19T05:05:53.945962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.183533+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:05:54.183533+0000","last_peered":"2024-09-19T05:05:54.183533+0000","last_clean":"2024-09-19T05:05:54.183533+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:05:54.183533+0000","last_undegraded":"2024-09-19T05:05:54.183533+0000","last_fullsized":"2024-09-19T05:05:54.183533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945636+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:05:53.945636+0000","last_peered":"2024-09-19T05:05:53.945636+0000","last_clean":"2024-09-19T05:05:53.945636+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:05:53.945636+0000","last_undegraded":"2024-09-19T05:05:53.945636+0000","last_fullsized":"2024-09-19T05:05:53.945636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783541+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:05:53.783541+0000","last_peered":"2024-09-19T05:05:53.783541+0000","last_clean":"2024-09-19T05:05:53.783541+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:05:53.783541+0000","last_undegraded":"2024-09-19T05:05:53.783541+0000","last_fullsized":"2024-09-19T05:05:53.783541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.780938+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:05:53.780938+0000","last_peered":"2024-09-19T05:05:53.780938+0000","last_clean":"2024-09-19T05:05:53.780938+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:05:53.780938+0000","last_undegraded":"2024-09-19T05:05:53.780938+0000","last_fullsized":"2024-09-19T05:05:53.780938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.784003+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:05:53.784003+0000","last_peered":"2024-09-19T05:05:53.784003+0000","last_clean":"2024-09-19T05:05:53.784003+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:05:53.784003+0000","last_undegraded":"2024-09-19T05:05:53.784003+0000","last_fullsized":"2024-09-19T05:05:53.784003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689229+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:05:53.689229+0000","last_peered":"2024-09-19T05:05:53.689229+0000","last_clean":"2024-09-19T05:05:53.689229+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:05:53.689229+0000","last_undegraded":"2024-09-19T05:05:53.689229+0000","last_fullsized":"2024-09-19T05:05:53.689229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971302+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:05:53.971302+0000","last_peered":"2024-09-19T05:05:53.971302+0000","last_clean":"2024-09-19T05:05:53.971302+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:05:53.971302+0000","last_undegraded":"2024-09-19T05:05:53.971302+0000","last_fullsized":"2024-09-19T05:05:53.971302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945960+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:05:53.945960+0000","last_peered":"2024-09-19T05:05:53.945960+0000","last_clean":"2024-09-19T05:05:53.945960+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:05:53.945960+0000","last_undegraded":"2024-09-19T05:05:53.945960+0000","last_fullsized":"2024-09-19T05:05:53.945960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":71,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.894173+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:05:53.894173+0000","last_peered":"2024-09-19T05:05:53.894173+0000","last_clean":"2024-09-19T05:05:53.894173+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:05:53.894173+0000","last_undegraded":"2024-09-19T05:05:53.894173+0000","last_fullsized":"2024-09-19T05:05:53.894173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945898+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:05:53.945898+0000","last_peered":"2024-09-19T05:05:53.945898+0000","last_clean":"2024-09-19T05:05:53.945898+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:05:53.945898+0000","last_undegraded":"2024-09-19T05:05:53.945898+0000","last_fullsized":"2024-09-19T05:05:53.945898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945656+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:05:53.945656+0000","last_peered":"2024-09-19T05:05:53.945656+0000","last_clean":"2024-09-19T05:05:53.945656+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:05:53.945656+0000","last_undegraded":"2024-09-19T05:05:53.945656+0000","last_fullsized":"2024-09-19T05:05:53.945656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946106+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:05:53.946106+0000","last_peered":"2024-09-19T05:05:53.946106+0000","last_clean":"2024-09-19T05:05:53.946106+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:05:53.946106+0000","last_undegraded":"2024-09-19T05:05:53.946106+0000","last_fullsized":"2024-09-19T05:05:53.946106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":346,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945800+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:05:53.945800+0000","last_peered":"2024-09-19T05:05:53.945800+0000","last_clean":"2024-09-19T05:05:53.945800+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:05:53.945800+0000","last_undegraded":"2024-09-19T05:05:53.945800+0000","last_fullsized":"2024-09-19T05:05:53.945800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946523+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:05:53.946523+0000","last_peered":"2024-09-19T05:05:53.946523+0000","last_clean":"2024-09-19T05:05:53.946523+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:05:53.946523+0000","last_undegraded":"2024-09-19T05:05:53.946523+0000","last_fullsized":"2024-09-19T05:05:53.946523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690501+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:05:53.690501+0000","last_peered":"2024-09-19T05:05:53.690501+0000","last_clean":"2024-09-19T05:05:53.690501+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:05:53.690501+0000","last_undegraded":"2024-09-19T05:05:53.690501+0000","last_fullsized":"2024-09-19T05:05:53.690501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945921+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:05:53.945921+0000","last_peered":"2024-09-19T05:05:53.945921+0000","last_clean":"2024-09-19T05:05:53.945921+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:05:53.945921+0000","last_undegraded":"2024-09-19T05:05:53.945921+0000","last_fullsized":"2024-09-19T05:05:53.945921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.970734+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:05:53.970734+0000","last_peered":"2024-09-19T05:05:53.970734+0000","last_clean":"2024-09-19T05:05:53.970734+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:05:53.970734+0000","last_undegraded":"2024-09-19T05:05:53.970734+0000","last_fullsized":"2024-09-19T05:05:53.970734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":126,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971558+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:05:53.971558+0000","last_peered":"2024-09-19T05:05:53.971558+0000","last_clean":"2024-09-19T05:05:53.971558+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:05:53.971558+0000","last_undegraded":"2024-09-19T05:05:53.971558+0000","last_fullsized":"2024-09-19T05:05:53.971558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789355+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:05:53.789355+0000","last_peered":"2024-09-19T05:05:53.789355+0000","last_clean":"2024-09-19T05:05:53.789355+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:05:53.789355+0000","last_undegraded":"2024-09-19T05:05:53.789355+0000","last_fullsized":"2024-09-19T05:05:53.789355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945849+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:05:53.945849+0000","last_peered":"2024-09-19T05:05:53.945849+0000","last_clean":"2024-09-19T05:05:53.945849+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:05:53.945849+0000","last_undegraded":"2024-09-19T05:05:53.945849+0000","last_fullsized":"2024-09-19T05:05:53.945849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691485+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:05:53.691485+0000","last_peered":"2024-09-19T05:05:53.691485+0000","last_clean":"2024-09-19T05:05:53.691485+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:05:53.691485+0000","last_undegraded":"2024-09-19T05:05:53.691485+0000","last_fullsized":"2024-09-19T05:05:53.691485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781445+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:05:53.781445+0000","last_peered":"2024-09-19T05:05:53.781445+0000","last_clean":"2024-09-19T05:05:53.781445+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:05:53.781445+0000","last_undegraded":"2024-09-19T05:05:53.781445+0000","last_fullsized":"2024-09-19T05:05:53.781445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691488+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:05:53.691488+0000","last_peered":"2024-09-19T05:05:53.691488+0000","last_clean":"2024-09-19T05:05:53.691488+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:05:53.691488+0000","last_undegraded":"2024-09-19T05:05:53.691488+0000","last_fullsized":"2024-09-19T05:05:53.691488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783918+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:05:53.783918+0000","last_peered":"2024-09-19T05:05:53.783918+0000","last_clean":"2024-09-19T05:05:53.783918+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:05:53.783918+0000","last_undegraded":"2024-09-19T05:05:53.783918+0000","last_fullsized":"2024-09-19T05:05:53.783918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971243+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:05:53.971243+0000","last_peered":"2024-09-19T05:05:53.971243+0000","last_clean":"2024-09-19T05:05:53.971243+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:05:53.971243+0000","last_undegraded":"2024-09-19T05:05:53.971243+0000","last_fullsized":"2024-09-19T05:05:53.971243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'47","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.901455+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:05:53.901455+0000","last_peered":"2024-09-19T05:05:53.901455+0000","last_clean":"2024-09-19T05:05:53.901455+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:05:53.901455+0000","last_undegraded":"2024-09-19T05:05:53.901455+0000","last_fullsized":"2024-09-19T05:05:53.901455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690050+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:05:53.690050+0000","last_peered":"2024-09-19T05:05:53.690050+0000","last_clean":"2024-09-19T05:05:53.690050+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:05:53.690050+0000","last_undegraded":"2024-09-19T05:05:53.690050+0000","last_fullsized":"2024-09-19T05:05:53.690050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946983+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:05:53.946983+0000","last_peered":"2024-09-19T05:05:53.946983+0000","last_clean":"2024-09-19T05:05:53.946983+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:05:53.946983+0000","last_undegraded":"2024-09-19T05:05:53.946983+0000","last_fullsized":"2024-09-19T05:05:53.946983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973426+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:05:53.973426+0000","last_peered":"2024-09-19T05:05:53.973426+0000","last_clean":"2024-09-19T05:05:53.973426+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:05:53.973426+0000","last_undegraded":"2024-09-19T05:05:53.973426+0000","last_fullsized":"2024-09-19T05:05:53.973426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948732+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:05:53.948732+0000","last_peered":"2024-09-19T05:05:53.948732+0000","last_clean":"2024-09-19T05:05:53.948732+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:05:53.948732+0000","last_undegraded":"2024-09-19T05:05:53.948732+0000","last_fullsized":"2024-09-19T05:05:53.948732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783541+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:05:53.783541+0000","last_peered":"2024-09-19T05:05:53.783541+0000","last_clean":"2024-09-19T05:05:53.783541+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:05:53.783541+0000","last_undegraded":"2024-09-19T05:05:53.783541+0000","last_fullsized":"2024-09-19T05:05:53.783541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689947+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:05:53.689947+0000","last_peered":"2024-09-19T05:05:53.689947+0000","last_clean":"2024-09-19T05:05:53.689947+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:05:53.689947+0000","last_undegraded":"2024-09-19T05:05:53.689947+0000","last_fullsized":"2024-09-19T05:05:53.689947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689258+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:05:53.689258+0000","last_peered":"2024-09-19T05:05:53.689258+0000","last_clean":"2024-09-19T05:05:53.689258+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:05:53.689258+0000","last_undegraded":"2024-09-19T05:05:53.689258+0000","last_fullsized":"2024-09-19T05:05:53.689258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945400+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:05:53.945400+0000","last_peered":"2024-09-19T05:05:53.945400+0000","last_clean":"2024-09-19T05:05:53.945400+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:05:53.945400+0000","last_undegraded":"2024-09-19T05:05:53.945400+0000","last_fullsized":"2024-09-19T05:05:53.945400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689907+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:05:53.689907+0000","last_peered":"2024-09-19T05:05:53.689907+0000","last_clean":"2024-09-19T05:05:53.689907+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:05:53.689907+0000","last_undegraded":"2024-09-19T05:05:53.689907+0000","last_fullsized":"2024-09-19T05:05:53.689907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783621+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:05:53.783621+0000","last_peered":"2024-09-19T05:05:53.783621+0000","last_clean":"2024-09-19T05:05:53.783621+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:05:53.783621+0000","last_undegraded":"2024-09-19T05:05:53.783621+0000","last_fullsized":"2024-09-19T05:05:53.783621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945782+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:05:53.945782+0000","last_peered":"2024-09-19T05:05:53.945782+0000","last_clean":"2024-09-19T05:05:53.945782+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:05:53.945782+0000","last_undegraded":"2024-09-19T05:05:53.945782+0000","last_fullsized":"2024-09-19T05:05:53.945782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789418+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:05:53.789418+0000","last_peered":"2024-09-19T05:05:53.789418+0000","last_clean":"2024-09-19T05:05:53.789418+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:05:53.789418+0000","last_undegraded":"2024-09-19T05:05:53.789418+0000","last_fullsized":"2024-09-19T05:05:53.789418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783794+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:05:53.783794+0000","last_peered":"2024-09-19T05:05:53.783794+0000","last_clean":"2024-09-19T05:05:53.783794+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:05:53.783794+0000","last_undegraded":"2024-09-19T05:05:53.783794+0000","last_fullsized":"2024-09-19T05:05:53.783794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948663+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:05:53.948663+0000","last_peered":"2024-09-19T05:05:53.948663+0000","last_clean":"2024-09-19T05:05:53.948663+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:05:53.948663+0000","last_undegraded":"2024-09-19T05:05:53.948663+0000","last_fullsized":"2024-09-19T05:05:53.948663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689087+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:05:53.689087+0000","last_peered":"2024-09-19T05:05:53.689087+0000","last_clean":"2024-09-19T05:05:53.689087+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:05:53.689087+0000","last_undegraded":"2024-09-19T05:05:53.689087+0000","last_fullsized":"2024-09-19T05:05:53.689087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973779+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:05:53.973779+0000","last_peered":"2024-09-19T05:05:53.973779+0000","last_clean":"2024-09-19T05:05:53.973779+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:05:53.973779+0000","last_undegraded":"2024-09-19T05:05:53.973779+0000","last_fullsized":"2024-09-19T05:05:53.973779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690350+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:05:53.690350+0000","last_peered":"2024-09-19T05:05:53.690350+0000","last_clean":"2024-09-19T05:05:53.690350+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:05:53.690350+0000","last_undegraded":"2024-09-19T05:05:53.690350+0000","last_fullsized":"2024-09-19T05:05:53.690350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690198+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:05:53.690198+0000","last_peered":"2024-09-19T05:05:53.690198+0000","last_clean":"2024-09-19T05:05:53.690198+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:05:53.690198+0000","last_undegraded":"2024-09-19T05:05:53.690198+0000","last_fullsized":"2024-09-19T05:05:53.690198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781831+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:05:53.781831+0000","last_peered":"2024-09-19T05:05:53.781831+0000","last_clean":"2024-09-19T05:05:53.781831+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:05:53.781831+0000","last_undegraded":"2024-09-19T05:05:53.781831+0000","last_fullsized":"2024-09-19T05:05:53.781831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598497,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":709,"num_read_kb":465,"num_write":295,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1795491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678050,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":431785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1223481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1294336,"data_stored":825203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1294336,"data_stored":825123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528373,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1015985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":431376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888127,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1642496,"data_stored":1156152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":432701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712568,"statfs":{"total":95995035648,"available":95961669632,"internally_reserved":0,"allocated":1433600,"data_stored":956503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":501668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":498930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":909312,"data_stored":432647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:05:58.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:05:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/2787655214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f881035b0 msgr2=0x7f7f8810ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/2787655214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f881035b0 0x7f7f8810ba90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009900 tx=0x7f7f7c02a480 comp rx=0 tx=0).stop 2024-09-19T05:05:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/2787655214 shutdown_connections 2024-09-19T05:05:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/2787655214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8810c0e0 0x7f7f8810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/2787655214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f881035b0 0x7f7f8810ba90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/2787655214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f88102c90 0x7f7f88103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/2787655214 >> 172.21.15.64:0/2787655214 conn(0x7f7f880fc6a0 msgr2=0x7f7f880feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/2787655214 shutdown_connections 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.544+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/2787655214 wait complete. 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 Processor -- start 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 -- start start 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88102c90 0x7f7f88198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f881035b0 0x7f7f881988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 0x7f7f8819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88111d70 con 0x7f7f8810c0e0 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88111bf0 con 0x7f7f881035b0 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.545+0000 7f7f8ddcd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88111ef0 con 0x7f7f88102c90 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 0x7f7f8819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 0x7f7f8819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42514/0 (socket says 172.21.15.64:42514) 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 -- 172.21.15.64:0/3783153390 learned_addr learned my addr 172.21.15.64:0/3783153390 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f86ffd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f881035b0 0x7f7f881988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 -- 172.21.15.64:0/3783153390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88102c90 msgr2=0x7f7f88198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f877fe640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88102c90 0x7f7f88198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88102c90 0x7f7f88198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 -- 172.21.15.64:0/3783153390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f881035b0 msgr2=0x7f7f881988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f881035b0 0x7f7f881988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 -- 172.21.15.64:0/3783153390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8819d370 con 0x7f7f8810c0e0 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f86ffd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f881035b0 0x7f7f881988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f87fff640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 0x7f7f8819cc40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000eaa0 tx=0x7f7f8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000cdb0 con 0x7f7f8810c0e0 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f80005020 con 0x7f7f8810c0e0 2024-09-19T05:05:58.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8819d660 con 0x7f7f8810c0e0 2024-09-19T05:05:58.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800116d0 con 0x7f7f8810c0e0 2024-09-19T05:05:58.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.546+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f881a4ef0 con 0x7f7f8810c0e0 2024-09-19T05:05:58.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.547+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f88103070 con 0x7f7f8810c0e0 2024-09-19T05:05:58.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.548+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 138526+0+0 (secure 0 0 0) 0x7f7f800051d0 con 0x7f7f8810c0e0 2024-09-19T05:05:58.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.550+0000 7f7f84ff9640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7f700a34d0 0x7f7f700a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.550+0000 7f7f877fe640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7f700a34d0 0x7f7f700a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.550+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7f800d0050 con 0x7f7f8810c0e0 2024-09-19T05:05:58.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.550+0000 7f7f877fe640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7f700a34d0 0x7f7f700a5990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009aa0 tx=0x7f7f78005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:58.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.554+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7f80016030 con 0x7f7f8810c0e0 2024-09-19T05:05:58.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.643+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f7f80098830 con 0x7f7f8810c0e0 2024-09-19T05:05:58.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.737+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f88062630 con 0x7f7f700a34d0 2024-09-19T05:05:58.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.744+0000 7f7f84ff9640 1 -- 172.21.15.64:0/3783153390 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257853 (secure 0 0 0) 0x7f7f88062630 con 0x7f7f700a34d0 2024-09-19T05:05:58.748 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7f700a34d0 msgr2=0x7f7f700a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7f700a34d0 0x7f7f700a5990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009aa0 tx=0x7f7f78005f90 comp rx=0 tx=0).stop 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 msgr2=0x7f7f8819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 0x7f7f8819cc40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000eaa0 tx=0x7f7f8000ef60 comp rx=0 tx=0).stop 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 shutdown_connections 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7f700a34d0 0x7f7f700a5990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f8810c0e0 0x7f7f8819cc40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f881035b0 0x7f7f881988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 --2- 172.21.15.64:0/3783153390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88102c90 0x7f7f88198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 >> 172.21.15.64:0/3783153390 conn(0x7f7f880fc6a0 msgr2=0x7f7f880fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:58.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 shutdown_connections 2024-09-19T05:05:58.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.747+0000 7f7f8ddcd640 1 -- 172.21.15.64:0/3783153390 wait complete. 2024-09-19T05:05:58.759 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T05:05:57.703956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189099,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":845,"num_read_kb":584,"num_write":478,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367548,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574276,"statfs":{"total":1151940427776,"available":1151564058624,"internally_reserved":0,"allocated":14385152,"data_stored":8731554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.967014"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689413+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:05:53.689413+0000","last_peered":"2024-09-19T05:05:53.689413+0000","last_clean":"2024-09-19T05:05:53.689413+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:05:53.689413+0000","last_undegraded":"2024-09-19T05:05:53.689413+0000","last_fullsized":"2024-09-19T05:05:53.689413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973308+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:05:53.973308+0000","last_peered":"2024-09-19T05:05:53.973308+0000","last_clean":"2024-09-19T05:05:53.973308+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:05:53.973308+0000","last_undegraded":"2024-09-19T05:05:53.973308+0000","last_fullsized":"2024-09-19T05:05:53.973308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972992+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:05:53.972992+0000","last_peered":"2024-09-19T05:05:53.972992+0000","last_clean":"2024-09-19T05:05:53.972992+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:05:53.972992+0000","last_undegraded":"2024-09-19T05:05:53.972992+0000","last_fullsized":"2024-09-19T05:05:53.972992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946819+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:05:53.946819+0000","last_peered":"2024-09-19T05:05:53.946819+0000","last_clean":"2024-09-19T05:05:53.946819+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:05:53.946819+0000","last_undegraded":"2024-09-19T05:05:53.946819+0000","last_fullsized":"2024-09-19T05:05:53.946819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783606+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:05:53.783606+0000","last_peered":"2024-09-19T05:05:53.783606+0000","last_clean":"2024-09-19T05:05:53.783606+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:05:53.783606+0000","last_undegraded":"2024-09-19T05:05:53.783606+0000","last_fullsized":"2024-09-19T05:05:53.783606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691370+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:05:53.691370+0000","last_peered":"2024-09-19T05:05:53.691370+0000","last_clean":"2024-09-19T05:05:53.691370+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:05:53.691370+0000","last_undegraded":"2024-09-19T05:05:53.691370+0000","last_fullsized":"2024-09-19T05:05:53.691370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945667+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:05:53.945667+0000","last_peered":"2024-09-19T05:05:53.945667+0000","last_clean":"2024-09-19T05:05:53.945667+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:05:53.945667+0000","last_undegraded":"2024-09-19T05:05:53.945667+0000","last_fullsized":"2024-09-19T05:05:53.945667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689751+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:05:53.689751+0000","last_peered":"2024-09-19T05:05:53.689751+0000","last_clean":"2024-09-19T05:05:53.689751+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:05:53.689751+0000","last_undegraded":"2024-09-19T05:05:53.689751+0000","last_fullsized":"2024-09-19T05:05:53.689751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948971+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:05:53.948971+0000","last_peered":"2024-09-19T05:05:53.948971+0000","last_clean":"2024-09-19T05:05:53.948971+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:05:53.948971+0000","last_undegraded":"2024-09-19T05:05:53.948971+0000","last_fullsized":"2024-09-19T05:05:53.948971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691740+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:05:53.691740+0000","last_peered":"2024-09-19T05:05:53.691740+0000","last_clean":"2024-09-19T05:05:53.691740+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:05:53.691740+0000","last_undegraded":"2024-09-19T05:05:53.691740+0000","last_fullsized":"2024-09-19T05:05:53.691740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946061+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:05:53.946061+0000","last_peered":"2024-09-19T05:05:53.946061+0000","last_clean":"2024-09-19T05:05:53.946061+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:05:53.946061+0000","last_undegraded":"2024-09-19T05:05:53.946061+0000","last_fullsized":"2024-09-19T05:05:53.946061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946960+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:05:53.946960+0000","last_peered":"2024-09-19T05:05:53.946960+0000","last_clean":"2024-09-19T05:05:53.946960+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:05:53.946960+0000","last_undegraded":"2024-09-19T05:05:53.946960+0000","last_fullsized":"2024-09-19T05:05:53.946960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971919+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:05:53.971919+0000","last_peered":"2024-09-19T05:05:53.971919+0000","last_clean":"2024-09-19T05:05:53.971919+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:05:53.971919+0000","last_undegraded":"2024-09-19T05:05:53.971919+0000","last_fullsized":"2024-09-19T05:05:53.971919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972311+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:05:53.972311+0000","last_peered":"2024-09-19T05:05:53.972311+0000","last_clean":"2024-09-19T05:05:53.972311+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:05:53.972311+0000","last_undegraded":"2024-09-19T05:05:53.972311+0000","last_fullsized":"2024-09-19T05:05:53.972311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972019+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:05:53.972019+0000","last_peered":"2024-09-19T05:05:53.972019+0000","last_clean":"2024-09-19T05:05:53.972019+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:05:53.972019+0000","last_undegraded":"2024-09-19T05:05:53.972019+0000","last_fullsized":"2024-09-19T05:05:53.972019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948575+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:05:53.948575+0000","last_peered":"2024-09-19T05:05:53.948575+0000","last_clean":"2024-09-19T05:05:53.948575+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:05:53.948575+0000","last_undegraded":"2024-09-19T05:05:53.948575+0000","last_fullsized":"2024-09-19T05:05:53.948575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689453+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:05:53.689453+0000","last_peered":"2024-09-19T05:05:53.689453+0000","last_clean":"2024-09-19T05:05:53.689453+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:05:53.689453+0000","last_undegraded":"2024-09-19T05:05:53.689453+0000","last_fullsized":"2024-09-19T05:05:53.689453+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781607+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:05:53.781607+0000","last_peered":"2024-09-19T05:05:53.781607+0000","last_clean":"2024-09-19T05:05:53.781607+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:05:53.781607+0000","last_undegraded":"2024-09-19T05:05:53.781607+0000","last_fullsized":"2024-09-19T05:05:53.781607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945659+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:05:53.945659+0000","last_peered":"2024-09-19T05:05:53.945659+0000","last_clean":"2024-09-19T05:05:53.945659+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:05:53.945659+0000","last_undegraded":"2024-09-19T05:05:53.945659+0000","last_fullsized":"2024-09-19T05:05:53.945659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689033+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:05:53.689033+0000","last_peered":"2024-09-19T05:05:53.689033+0000","last_clean":"2024-09-19T05:05:53.689033+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:05:53.689033+0000","last_undegraded":"2024-09-19T05:05:53.689033+0000","last_fullsized":"2024-09-19T05:05:53.689033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972075+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:05:53.972075+0000","last_peered":"2024-09-19T05:05:53.972075+0000","last_clean":"2024-09-19T05:05:53.972075+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:05:53.972075+0000","last_undegraded":"2024-09-19T05:05:53.972075+0000","last_fullsized":"2024-09-19T05:05:53.972075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948706+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:05:53.948706+0000","last_peered":"2024-09-19T05:05:53.948706+0000","last_clean":"2024-09-19T05:05:53.948706+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:05:53.948706+0000","last_undegraded":"2024-09-19T05:05:53.948706+0000","last_fullsized":"2024-09-19T05:05:53.948706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689785+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:05:53.689785+0000","last_peered":"2024-09-19T05:05:53.689785+0000","last_clean":"2024-09-19T05:05:53.689785+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:05:53.689785+0000","last_undegraded":"2024-09-19T05:05:53.689785+0000","last_fullsized":"2024-09-19T05:05:53.689785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789295+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:05:53.789295+0000","last_peered":"2024-09-19T05:05:53.789295+0000","last_clean":"2024-09-19T05:05:53.789295+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:05:53.789295+0000","last_undegraded":"2024-09-19T05:05:53.789295+0000","last_fullsized":"2024-09-19T05:05:53.789295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781636+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:05:53.781636+0000","last_peered":"2024-09-19T05:05:53.781636+0000","last_clean":"2024-09-19T05:05:53.781636+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:05:53.781636+0000","last_undegraded":"2024-09-19T05:05:53.781636+0000","last_fullsized":"2024-09-19T05:05:53.781636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945753+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:05:53.945753+0000","last_peered":"2024-09-19T05:05:53.945753+0000","last_clean":"2024-09-19T05:05:53.945753+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:05:53.945753+0000","last_undegraded":"2024-09-19T05:05:53.945753+0000","last_fullsized":"2024-09-19T05:05:53.945753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689249+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:05:53.689249+0000","last_peered":"2024-09-19T05:05:53.689249+0000","last_clean":"2024-09-19T05:05:53.689249+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:05:53.689249+0000","last_undegraded":"2024-09-19T05:05:53.689249+0000","last_fullsized":"2024-09-19T05:05:53.689249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971123+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:05:53.971123+0000","last_peered":"2024-09-19T05:05:53.971123+0000","last_clean":"2024-09-19T05:05:53.971123+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:05:53.971123+0000","last_undegraded":"2024-09-19T05:05:53.971123+0000","last_fullsized":"2024-09-19T05:05:53.971123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972635+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:05:53.972635+0000","last_peered":"2024-09-19T05:05:53.972635+0000","last_clean":"2024-09-19T05:05:53.972635+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:05:53.972635+0000","last_undegraded":"2024-09-19T05:05:53.972635+0000","last_fullsized":"2024-09-19T05:05:53.972635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945375+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:05:53.945375+0000","last_peered":"2024-09-19T05:05:53.945375+0000","last_clean":"2024-09-19T05:05:53.945375+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:05:53.945375+0000","last_undegraded":"2024-09-19T05:05:53.945375+0000","last_fullsized":"2024-09-19T05:05:53.945375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945424+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:05:53.945424+0000","last_peered":"2024-09-19T05:05:53.945424+0000","last_clean":"2024-09-19T05:05:53.945424+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:05:53.945424+0000","last_undegraded":"2024-09-19T05:05:53.945424+0000","last_fullsized":"2024-09-19T05:05:53.945424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689079+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:05:53.689079+0000","last_peered":"2024-09-19T05:05:53.689079+0000","last_clean":"2024-09-19T05:05:53.689079+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:05:53.689079+0000","last_undegraded":"2024-09-19T05:05:53.689079+0000","last_fullsized":"2024-09-19T05:05:53.689079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783659+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:05:53.783659+0000","last_peered":"2024-09-19T05:05:53.783659+0000","last_clean":"2024-09-19T05:05:53.783659+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:05:53.783659+0000","last_undegraded":"2024-09-19T05:05:53.783659+0000","last_fullsized":"2024-09-19T05:05:53.783659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'9","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.163491+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:05:54.163491+0000","last_peered":"2024-09-19T05:05:54.163491+0000","last_clean":"2024-09-19T05:05:54.163491+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:05:54.163491+0000","last_undegraded":"2024-09-19T05:05:54.163491+0000","last_fullsized":"2024-09-19T05:05:54.163491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781306+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:05:53.781306+0000","last_peered":"2024-09-19T05:05:53.781306+0000","last_clean":"2024-09-19T05:05:53.781306+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:05:53.781306+0000","last_undegraded":"2024-09-19T05:05:53.781306+0000","last_fullsized":"2024-09-19T05:05:53.781306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691365+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:05:53.691365+0000","last_peered":"2024-09-19T05:05:53.691365+0000","last_clean":"2024-09-19T05:05:53.691365+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:05:53.691365+0000","last_undegraded":"2024-09-19T05:05:53.691365+0000","last_fullsized":"2024-09-19T05:05:53.691365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971411+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:05:53.971411+0000","last_peered":"2024-09-19T05:05:53.971411+0000","last_clean":"2024-09-19T05:05:53.971411+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:05:53.971411+0000","last_undegraded":"2024-09-19T05:05:53.971411+0000","last_fullsized":"2024-09-19T05:05:53.971411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689842+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:05:53.689842+0000","last_peered":"2024-09-19T05:05:53.689842+0000","last_clean":"2024-09-19T05:05:53.689842+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:05:53.689842+0000","last_undegraded":"2024-09-19T05:05:53.689842+0000","last_fullsized":"2024-09-19T05:05:53.689842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789504+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:05:53.789504+0000","last_peered":"2024-09-19T05:05:53.789504+0000","last_clean":"2024-09-19T05:05:53.789504+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:05:53.789504+0000","last_undegraded":"2024-09-19T05:05:53.789504+0000","last_fullsized":"2024-09-19T05:05:53.789504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781090+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:05:53.781090+0000","last_peered":"2024-09-19T05:05:53.781090+0000","last_clean":"2024-09-19T05:05:53.781090+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:05:53.781090+0000","last_undegraded":"2024-09-19T05:05:53.781090+0000","last_fullsized":"2024-09-19T05:05:53.781090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691578+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:05:53.691578+0000","last_peered":"2024-09-19T05:05:53.691578+0000","last_clean":"2024-09-19T05:05:53.691578+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:05:53.691578+0000","last_undegraded":"2024-09-19T05:05:53.691578+0000","last_fullsized":"2024-09-19T05:05:53.691578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948853+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:05:53.948853+0000","last_peered":"2024-09-19T05:05:53.948853+0000","last_clean":"2024-09-19T05:05:53.948853+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:05:53.948853+0000","last_undegraded":"2024-09-19T05:05:53.948853+0000","last_fullsized":"2024-09-19T05:05:53.948853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946143+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:05:53.946143+0000","last_peered":"2024-09-19T05:05:53.946143+0000","last_clean":"2024-09-19T05:05:53.946143+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:05:53.946143+0000","last_undegraded":"2024-09-19T05:05:53.946143+0000","last_fullsized":"2024-09-19T05:05:53.946143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973668+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:05:53.973668+0000","last_peered":"2024-09-19T05:05:53.973668+0000","last_clean":"2024-09-19T05:05:53.973668+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:05:53.973668+0000","last_undegraded":"2024-09-19T05:05:53.973668+0000","last_fullsized":"2024-09-19T05:05:53.973668+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789562+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:05:53.789562+0000","last_peered":"2024-09-19T05:05:53.789562+0000","last_clean":"2024-09-19T05:05:53.789562+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:05:53.789562+0000","last_undegraded":"2024-09-19T05:05:53.789562+0000","last_fullsized":"2024-09-19T05:05:53.789562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948706+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:05:53.948706+0000","last_peered":"2024-09-19T05:05:53.948706+0000","last_clean":"2024-09-19T05:05:53.948706+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:05:53.948706+0000","last_undegraded":"2024-09-19T05:05:53.948706+0000","last_fullsized":"2024-09-19T05:05:53.948706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'37","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.166232+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:05:54.166232+0000","last_peered":"2024-09-19T05:05:54.166232+0000","last_clean":"2024-09-19T05:05:54.166232+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:05:54.166232+0000","last_undegraded":"2024-09-19T05:05:54.166232+0000","last_fullsized":"2024-09-19T05:05:54.166232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67761,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":169,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783871+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:05:53.783871+0000","last_peered":"2024-09-19T05:05:53.783871+0000","last_clean":"2024-09-19T05:05:53.783871+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:05:53.783871+0000","last_undegraded":"2024-09-19T05:05:53.783871+0000","last_fullsized":"2024-09-19T05:05:53.783871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689009+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:05:53.689009+0000","last_peered":"2024-09-19T05:05:53.689009+0000","last_clean":"2024-09-19T05:05:53.689009+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:05:53.689009+0000","last_undegraded":"2024-09-19T05:05:53.689009+0000","last_fullsized":"2024-09-19T05:05:53.689009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971016+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:05:53.971016+0000","last_peered":"2024-09-19T05:05:53.971016+0000","last_clean":"2024-09-19T05:05:53.971016+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:05:53.971016+0000","last_undegraded":"2024-09-19T05:05:53.971016+0000","last_fullsized":"2024-09-19T05:05:53.971016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789394+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:05:53.789394+0000","last_peered":"2024-09-19T05:05:53.789394+0000","last_clean":"2024-09-19T05:05:53.789394+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:05:53.789394+0000","last_undegraded":"2024-09-19T05:05:53.789394+0000","last_fullsized":"2024-09-19T05:05:53.789394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945962+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:05:53.945962+0000","last_peered":"2024-09-19T05:05:53.945962+0000","last_clean":"2024-09-19T05:05:53.945962+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:05:53.945962+0000","last_undegraded":"2024-09-19T05:05:53.945962+0000","last_fullsized":"2024-09-19T05:05:53.945962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.183533+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:05:54.183533+0000","last_peered":"2024-09-19T05:05:54.183533+0000","last_clean":"2024-09-19T05:05:54.183533+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:05:54.183533+0000","last_undegraded":"2024-09-19T05:05:54.183533+0000","last_fullsized":"2024-09-19T05:05:54.183533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945636+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:05:53.945636+0000","last_peered":"2024-09-19T05:05:53.945636+0000","last_clean":"2024-09-19T05:05:53.945636+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:05:53.945636+0000","last_undegraded":"2024-09-19T05:05:53.945636+0000","last_fullsized":"2024-09-19T05:05:53.945636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783541+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:05:53.783541+0000","last_peered":"2024-09-19T05:05:53.783541+0000","last_clean":"2024-09-19T05:05:53.783541+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:05:53.783541+0000","last_undegraded":"2024-09-19T05:05:53.783541+0000","last_fullsized":"2024-09-19T05:05:53.783541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.780938+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:05:53.780938+0000","last_peered":"2024-09-19T05:05:53.780938+0000","last_clean":"2024-09-19T05:05:53.780938+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:05:53.780938+0000","last_undegraded":"2024-09-19T05:05:53.780938+0000","last_fullsized":"2024-09-19T05:05:53.780938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.784003+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:05:53.784003+0000","last_peered":"2024-09-19T05:05:53.784003+0000","last_clean":"2024-09-19T05:05:53.784003+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:05:53.784003+0000","last_undegraded":"2024-09-19T05:05:53.784003+0000","last_fullsized":"2024-09-19T05:05:53.784003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689229+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:05:53.689229+0000","last_peered":"2024-09-19T05:05:53.689229+0000","last_clean":"2024-09-19T05:05:53.689229+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:05:53.689229+0000","last_undegraded":"2024-09-19T05:05:53.689229+0000","last_fullsized":"2024-09-19T05:05:53.689229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971302+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:05:53.971302+0000","last_peered":"2024-09-19T05:05:53.971302+0000","last_clean":"2024-09-19T05:05:53.971302+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:05:53.971302+0000","last_undegraded":"2024-09-19T05:05:53.971302+0000","last_fullsized":"2024-09-19T05:05:53.971302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945960+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:05:53.945960+0000","last_peered":"2024-09-19T05:05:53.945960+0000","last_clean":"2024-09-19T05:05:53.945960+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:05:53.945960+0000","last_undegraded":"2024-09-19T05:05:53.945960+0000","last_fullsized":"2024-09-19T05:05:53.945960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":71,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.894173+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:05:53.894173+0000","last_peered":"2024-09-19T05:05:53.894173+0000","last_clean":"2024-09-19T05:05:53.894173+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:05:53.894173+0000","last_undegraded":"2024-09-19T05:05:53.894173+0000","last_fullsized":"2024-09-19T05:05:53.894173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945898+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:05:53.945898+0000","last_peered":"2024-09-19T05:05:53.945898+0000","last_clean":"2024-09-19T05:05:53.945898+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:05:53.945898+0000","last_undegraded":"2024-09-19T05:05:53.945898+0000","last_fullsized":"2024-09-19T05:05:53.945898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945656+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:05:53.945656+0000","last_peered":"2024-09-19T05:05:53.945656+0000","last_clean":"2024-09-19T05:05:53.945656+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:05:53.945656+0000","last_undegraded":"2024-09-19T05:05:53.945656+0000","last_fullsized":"2024-09-19T05:05:53.945656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946106+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:05:53.946106+0000","last_peered":"2024-09-19T05:05:53.946106+0000","last_clean":"2024-09-19T05:05:53.946106+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:05:53.946106+0000","last_undegraded":"2024-09-19T05:05:53.946106+0000","last_fullsized":"2024-09-19T05:05:53.946106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":346,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945800+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:05:53.945800+0000","last_peered":"2024-09-19T05:05:53.945800+0000","last_clean":"2024-09-19T05:05:53.945800+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:05:53.945800+0000","last_undegraded":"2024-09-19T05:05:53.945800+0000","last_fullsized":"2024-09-19T05:05:53.945800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946523+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:05:53.946523+0000","last_peered":"2024-09-19T05:05:53.946523+0000","last_clean":"2024-09-19T05:05:53.946523+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:05:53.946523+0000","last_undegraded":"2024-09-19T05:05:53.946523+0000","last_fullsized":"2024-09-19T05:05:53.946523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690501+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:05:53.690501+0000","last_peered":"2024-09-19T05:05:53.690501+0000","last_clean":"2024-09-19T05:05:53.690501+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:05:53.690501+0000","last_undegraded":"2024-09-19T05:05:53.690501+0000","last_fullsized":"2024-09-19T05:05:53.690501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945921+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:05:53.945921+0000","last_peered":"2024-09-19T05:05:53.945921+0000","last_clean":"2024-09-19T05:05:53.945921+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:05:53.945921+0000","last_undegraded":"2024-09-19T05:05:53.945921+0000","last_fullsized":"2024-09-19T05:05:53.945921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.970734+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:05:53.970734+0000","last_peered":"2024-09-19T05:05:53.970734+0000","last_clean":"2024-09-19T05:05:53.970734+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:05:53.970734+0000","last_undegraded":"2024-09-19T05:05:53.970734+0000","last_fullsized":"2024-09-19T05:05:53.970734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":126,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971558+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:05:53.971558+0000","last_peered":"2024-09-19T05:05:53.971558+0000","last_clean":"2024-09-19T05:05:53.971558+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:05:53.971558+0000","last_undegraded":"2024-09-19T05:05:53.971558+0000","last_fullsized":"2024-09-19T05:05:53.971558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789355+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:05:53.789355+0000","last_peered":"2024-09-19T05:05:53.789355+0000","last_clean":"2024-09-19T05:05:53.789355+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:05:53.789355+0000","last_undegraded":"2024-09-19T05:05:53.789355+0000","last_fullsized":"2024-09-19T05:05:53.789355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945849+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:05:53.945849+0000","last_peered":"2024-09-19T05:05:53.945849+0000","last_clean":"2024-09-19T05:05:53.945849+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:05:53.945849+0000","last_undegraded":"2024-09-19T05:05:53.945849+0000","last_fullsized":"2024-09-19T05:05:53.945849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691485+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:05:53.691485+0000","last_peered":"2024-09-19T05:05:53.691485+0000","last_clean":"2024-09-19T05:05:53.691485+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:05:53.691485+0000","last_undegraded":"2024-09-19T05:05:53.691485+0000","last_fullsized":"2024-09-19T05:05:53.691485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781445+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:05:53.781445+0000","last_peered":"2024-09-19T05:05:53.781445+0000","last_clean":"2024-09-19T05:05:53.781445+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:05:53.781445+0000","last_undegraded":"2024-09-19T05:05:53.781445+0000","last_fullsized":"2024-09-19T05:05:53.781445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691488+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:05:53.691488+0000","last_peered":"2024-09-19T05:05:53.691488+0000","last_clean":"2024-09-19T05:05:53.691488+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:05:53.691488+0000","last_undegraded":"2024-09-19T05:05:53.691488+0000","last_fullsized":"2024-09-19T05:05:53.691488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783918+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:05:53.783918+0000","last_peered":"2024-09-19T05:05:53.783918+0000","last_clean":"2024-09-19T05:05:53.783918+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:05:53.783918+0000","last_undegraded":"2024-09-19T05:05:53.783918+0000","last_fullsized":"2024-09-19T05:05:53.783918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971243+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:05:53.971243+0000","last_peered":"2024-09-19T05:05:53.971243+0000","last_clean":"2024-09-19T05:05:53.971243+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:05:53.971243+0000","last_undegraded":"2024-09-19T05:05:53.971243+0000","last_fullsized":"2024-09-19T05:05:53.971243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'47","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.901455+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:05:53.901455+0000","last_peered":"2024-09-19T05:05:53.901455+0000","last_clean":"2024-09-19T05:05:53.901455+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:05:53.901455+0000","last_undegraded":"2024-09-19T05:05:53.901455+0000","last_fullsized":"2024-09-19T05:05:53.901455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690050+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:05:53.690050+0000","last_peered":"2024-09-19T05:05:53.690050+0000","last_clean":"2024-09-19T05:05:53.690050+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:05:53.690050+0000","last_undegraded":"2024-09-19T05:05:53.690050+0000","last_fullsized":"2024-09-19T05:05:53.690050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946983+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:05:53.946983+0000","last_peered":"2024-09-19T05:05:53.946983+0000","last_clean":"2024-09-19T05:05:53.946983+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:05:53.946983+0000","last_undegraded":"2024-09-19T05:05:53.946983+0000","last_fullsized":"2024-09-19T05:05:53.946983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973426+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:05:53.973426+0000","last_peered":"2024-09-19T05:05:53.973426+0000","last_clean":"2024-09-19T05:05:53.973426+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:05:53.973426+0000","last_undegraded":"2024-09-19T05:05:53.973426+0000","last_fullsized":"2024-09-19T05:05:53.973426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948732+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:05:53.948732+0000","last_peered":"2024-09-19T05:05:53.948732+0000","last_clean":"2024-09-19T05:05:53.948732+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:05:53.948732+0000","last_undegraded":"2024-09-19T05:05:53.948732+0000","last_fullsized":"2024-09-19T05:05:53.948732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783541+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:05:53.783541+0000","last_peered":"2024-09-19T05:05:53.783541+0000","last_clean":"2024-09-19T05:05:53.783541+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:05:53.783541+0000","last_undegraded":"2024-09-19T05:05:53.783541+0000","last_fullsized":"2024-09-19T05:05:53.783541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689947+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:05:53.689947+0000","last_peered":"2024-09-19T05:05:53.689947+0000","last_clean":"2024-09-19T05:05:53.689947+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:05:53.689947+0000","last_undegraded":"2024-09-19T05:05:53.689947+0000","last_fullsized":"2024-09-19T05:05:53.689947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689258+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:05:53.689258+0000","last_peered":"2024-09-19T05:05:53.689258+0000","last_clean":"2024-09-19T05:05:53.689258+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:05:53.689258+0000","last_undegraded":"2024-09-19T05:05:53.689258+0000","last_fullsized":"2024-09-19T05:05:53.689258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945400+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:05:53.945400+0000","last_peered":"2024-09-19T05:05:53.945400+0000","last_clean":"2024-09-19T05:05:53.945400+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:05:53.945400+0000","last_undegraded":"2024-09-19T05:05:53.945400+0000","last_fullsized":"2024-09-19T05:05:53.945400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689907+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:05:53.689907+0000","last_peered":"2024-09-19T05:05:53.689907+0000","last_clean":"2024-09-19T05:05:53.689907+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:05:53.689907+0000","last_undegraded":"2024-09-19T05:05:53.689907+0000","last_fullsized":"2024-09-19T05:05:53.689907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783621+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:05:53.783621+0000","last_peered":"2024-09-19T05:05:53.783621+0000","last_clean":"2024-09-19T05:05:53.783621+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:05:53.783621+0000","last_undegraded":"2024-09-19T05:05:53.783621+0000","last_fullsized":"2024-09-19T05:05:53.783621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945782+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:05:53.945782+0000","last_peered":"2024-09-19T05:05:53.945782+0000","last_clean":"2024-09-19T05:05:53.945782+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:05:53.945782+0000","last_undegraded":"2024-09-19T05:05:53.945782+0000","last_fullsized":"2024-09-19T05:05:53.945782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789418+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:05:53.789418+0000","last_peered":"2024-09-19T05:05:53.789418+0000","last_clean":"2024-09-19T05:05:53.789418+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:05:53.789418+0000","last_undegraded":"2024-09-19T05:05:53.789418+0000","last_fullsized":"2024-09-19T05:05:53.789418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783794+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:05:53.783794+0000","last_peered":"2024-09-19T05:05:53.783794+0000","last_clean":"2024-09-19T05:05:53.783794+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:05:53.783794+0000","last_undegraded":"2024-09-19T05:05:53.783794+0000","last_fullsized":"2024-09-19T05:05:53.783794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948663+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:05:53.948663+0000","last_peered":"2024-09-19T05:05:53.948663+0000","last_clean":"2024-09-19T05:05:53.948663+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:05:53.948663+0000","last_undegraded":"2024-09-19T05:05:53.948663+0000","last_fullsized":"2024-09-19T05:05:53.948663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689087+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:05:53.689087+0000","last_peered":"2024-09-19T05:05:53.689087+0000","last_clean":"2024-09-19T05:05:53.689087+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:05:53.689087+0000","last_undegraded":"2024-09-19T05:05:53.689087+0000","last_fullsized":"2024-09-19T05:05:53.689087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973779+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:05:53.973779+0000","last_peered":"2024-09-19T05:05:53.973779+0000","last_clean":"2024-09-19T05:05:53.973779+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:05:53.973779+0000","last_undegraded":"2024-09-19T05:05:53.973779+0000","last_fullsized":"2024-09-19T05:05:53.973779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690350+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:05:53.690350+0000","last_peered":"2024-09-19T05:05:53.690350+0000","last_clean":"2024-09-19T05:05:53.690350+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:05:53.690350+0000","last_undegraded":"2024-09-19T05:05:53.690350+0000","last_fullsized":"2024-09-19T05:05:53.690350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690198+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:05:53.690198+0000","last_peered":"2024-09-19T05:05:53.690198+0000","last_clean":"2024-09-19T05:05:53.690198+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:05:53.690198+0000","last_undegraded":"2024-09-19T05:05:53.690198+0000","last_fullsized":"2024-09-19T05:05:53.690198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781831+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:05:53.781831+0000","last_peered":"2024-09-19T05:05:53.781831+0000","last_clean":"2024-09-19T05:05:53.781831+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:05:53.781831+0000","last_undegraded":"2024-09-19T05:05:53.781831+0000","last_fullsized":"2024-09-19T05:05:53.781831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598497,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":709,"num_read_kb":465,"num_write":295,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1795491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678050,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":431785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1223481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1294336,"data_stored":825203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1294336,"data_stored":825123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528373,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1015985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":431376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888127,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1642496,"data_stored":1156152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":432701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712568,"statfs":{"total":95995035648,"available":95961669632,"internally_reserved":0,"allocated":1433600,"data_stored":956503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":501668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":498930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":909312,"data_stored":432647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:05:58.763 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T05:05:58.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:05:58.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 -- 172.21.15.64:0/869496343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914102450 msgr2=0x7ff91410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 --2- 172.21.15.64:0/869496343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914102450 0x7ff91410e980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009cd0 tx=0x7ff90c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 -- 172.21.15.64:0/869496343 shutdown_connections 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 --2- 172.21.15.64:0/869496343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914102450 0x7ff91410e980 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 --2- 172.21.15.64:0/869496343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914101ab0 0x7ff914101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 --2- 172.21.15.64:0/869496343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914107ac0 0x7ff914107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 -- 172.21.15.64:0/869496343 >> 172.21.15.64:0/869496343 conn(0x7ff9140fd7d0 msgr2=0x7ff9140ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 -- 172.21.15.64:0/869496343 shutdown_connections 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 -- 172.21.15.64:0/869496343 wait complete. 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 Processor -- start 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.902+0000 7ff91ab96640 1 -- start start 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff91ab96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 0x7ff914198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff91ab96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914102450 0x7ff914198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff91ab96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914107ac0 0x7ff91419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff91ab96640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff914111db0 con 0x7ff914101ab0 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff91ab96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff914111c30 con 0x7ff914107ac0 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff91ab96640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff914111f30 con 0x7ff914102450 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 0x7ff914198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 0x7ff914198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42540/0 (socket says 172.21.15.64:42540) 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 -- 172.21.15.64:0/2433318356 learned_addr learned my addr 172.21.15.64:0/2433318356 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff919133640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914107ac0 0x7ff91419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 -- 172.21.15.64:0/2433318356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914102450 msgr2=0x7ff914198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff913fff640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914102450 0x7ff914198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914102450 0x7ff914198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 -- 172.21.15.64:0/2433318356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914107ac0 msgr2=0x7ff91419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914107ac0 0x7ff91419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 -- 172.21.15.64:0/2433318356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91419d310 con 0x7ff914101ab0 2024-09-19T05:05:58.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff913fff640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914102450 0x7ff914198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff919133640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914107ac0 0x7ff91419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff918932640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 0x7ff914198320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00c970 tx=0x7ff8fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc007bb0 con 0x7ff914101ab0 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.903+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8fc007d50 con 0x7ff914101ab0 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.904+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91419d600 con 0x7ff914101ab0 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.904+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00f450 con 0x7ff914101ab0 2024-09-19T05:05:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.904+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9141a4e90 con 0x7ff914101ab0 2024-09-19T05:05:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.905+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff914101f10 con 0x7ff914101ab0 2024-09-19T05:05:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.906+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7ff8fc004f20 con 0x7ff914101ab0 2024-09-19T05:05:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.906+0000 7ff911ffb640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff9000a34e0 0x7ff9000a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.907+0000 7ff913fff640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff9000a34e0 0x7ff9000a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.907+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff8fc0cfcc0 con 0x7ff914101ab0 2024-09-19T05:05:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.907+0000 7ff913fff640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff9000a34e0 0x7ff9000a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff914100f70 tx=0x7ff904005d30 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:58.910+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8fc014030 con 0x7ff914101ab0 2024-09-19T05:05:59.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:05:59.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:59.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:58 smithi174 ceph-mon[34488]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-19T05:05:59.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.087+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff914062630 con 0x7ff9000a34e0 2024-09-19T05:05:59.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.094+0000 7ff911ffb640 1 -- 172.21.15.64:0/2433318356 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257853 (secure 0 0 0) 0x7ff914062630 con 0x7ff9000a34e0 2024-09-19T05:05:59.098 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:05:59.099 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-19T05:05:59.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff9000a34e0 msgr2=0x7ff9000a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:59.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff9000a34e0 0x7ff9000a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff914100f70 tx=0x7ff904005d30 comp rx=0 tx=0).stop 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 msgr2=0x7ff914198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 0x7ff914198320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00c970 tx=0x7ff8fc00ce30 comp rx=0 tx=0).stop 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 shutdown_connections 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff9000a34e0 0x7ff9000a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff914107ac0 0x7ff91419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff914102450 0x7ff914198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 --2- 172.21.15.64:0/2433318356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff914101ab0 0x7ff914198320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 >> 172.21.15.64:0/2433318356 conn(0x7ff9140fd7d0 msgr2=0x7ff9140ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.097+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 shutdown_connections 2024-09-19T05:05:59.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.098+0000 7ff91ab96640 1 -- 172.21.15.64:0/2433318356 wait complete. 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:59.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:58 smithi064 ceph-mon[30960]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-19T05:05:59.110 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T05:05:57.703956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189099,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":845,"num_read_kb":584,"num_write":478,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180770,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367548,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574276,"statfs":{"total":1151940427776,"available":1151564058624,"internally_reserved":0,"allocated":14385152,"data_stored":8731554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.967014"},"pg_stats":[{"pgid":"3.3f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689413+0000","last_change":"2024-09-19T05:04:07.984610+0000","last_active":"2024-09-19T05:05:53.689413+0000","last_peered":"2024-09-19T05:05:53.689413+0000","last_clean":"2024-09-19T05:05:53.689413+0000","last_became_active":"2024-09-19T05:04:07.983693+0000","last_became_peered":"2024-09-19T05:04:07.983693+0000","last_unstale":"2024-09-19T05:05:53.689413+0000","last_undegraded":"2024-09-19T05:05:53.689413+0000","last_fullsized":"2024-09-19T05:05:53.689413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:01.675523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973308+0000","last_change":"2024-09-19T05:04:07.979046+0000","last_active":"2024-09-19T05:05:53.973308+0000","last_peered":"2024-09-19T05:05:53.973308+0000","last_clean":"2024-09-19T05:05:53.973308+0000","last_became_active":"2024-09-19T05:04:07.978707+0000","last_became_peered":"2024-09-19T05:04:07.978707+0000","last_unstale":"2024-09-19T05:05:53.973308+0000","last_undegraded":"2024-09-19T05:05:53.973308+0000","last_fullsized":"2024-09-19T05:05:53.973308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:55.636736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972992+0000","last_change":"2024-09-19T05:04:07.984684+0000","last_active":"2024-09-19T05:05:53.972992+0000","last_peered":"2024-09-19T05:05:53.972992+0000","last_clean":"2024-09-19T05:05:53.972992+0000","last_became_active":"2024-09-19T05:04:07.984379+0000","last_became_peered":"2024-09-19T05:04:07.984379+0000","last_unstale":"2024-09-19T05:05:53.972992+0000","last_undegraded":"2024-09-19T05:05:53.972992+0000","last_fullsized":"2024-09-19T05:05:53.972992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:16.486311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946819+0000","last_change":"2024-09-19T05:04:07.977896+0000","last_active":"2024-09-19T05:05:53.946819+0000","last_peered":"2024-09-19T05:05:53.946819+0000","last_clean":"2024-09-19T05:05:53.946819+0000","last_became_active":"2024-09-19T05:04:07.977584+0000","last_became_peered":"2024-09-19T05:04:07.977584+0000","last_unstale":"2024-09-19T05:05:53.946819+0000","last_undegraded":"2024-09-19T05:05:53.946819+0000","last_fullsized":"2024-09-19T05:05:53.946819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:29:51.782594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783606+0000","last_change":"2024-09-19T05:04:07.974656+0000","last_active":"2024-09-19T05:05:53.783606+0000","last_peered":"2024-09-19T05:05:53.783606+0000","last_clean":"2024-09-19T05:05:53.783606+0000","last_became_active":"2024-09-19T05:04:07.974261+0000","last_became_peered":"2024-09-19T05:04:07.974261+0000","last_unstale":"2024-09-19T05:05:53.783606+0000","last_undegraded":"2024-09-19T05:05:53.783606+0000","last_fullsized":"2024-09-19T05:05:53.783606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:31.910712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"79'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691370+0000","last_change":"2024-09-19T05:04:20.046644+0000","last_active":"2024-09-19T05:05:53.691370+0000","last_peered":"2024-09-19T05:05:53.691370+0000","last_clean":"2024-09-19T05:05:53.691370+0000","last_became_active":"2024-09-19T05:04:20.046281+0000","last_became_peered":"2024-09-19T05:04:20.046281+0000","last_unstale":"2024-09-19T05:05:53.691370+0000","last_undegraded":"2024-09-19T05:05:53.691370+0000","last_fullsized":"2024-09-19T05:05:53.691370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945667+0000","last_change":"2024-09-19T05:04:07.978959+0000","last_active":"2024-09-19T05:05:53.945667+0000","last_peered":"2024-09-19T05:05:53.945667+0000","last_clean":"2024-09-19T05:05:53.945667+0000","last_became_active":"2024-09-19T05:04:07.978641+0000","last_became_peered":"2024-09-19T05:04:07.978641+0000","last_unstale":"2024-09-19T05:05:53.945667+0000","last_undegraded":"2024-09-19T05:05:53.945667+0000","last_fullsized":"2024-09-19T05:05:53.945667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:18.895793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689751+0000","last_change":"2024-09-19T05:04:07.977699+0000","last_active":"2024-09-19T05:05:53.689751+0000","last_peered":"2024-09-19T05:05:53.689751+0000","last_clean":"2024-09-19T05:05:53.689751+0000","last_became_active":"2024-09-19T05:04:07.977384+0000","last_became_peered":"2024-09-19T05:04:07.977384+0000","last_unstale":"2024-09-19T05:05:53.689751+0000","last_undegraded":"2024-09-19T05:05:53.689751+0000","last_fullsized":"2024-09-19T05:05:53.689751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:31.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948971+0000","last_change":"2024-09-19T05:04:07.982510+0000","last_active":"2024-09-19T05:05:53.948971+0000","last_peered":"2024-09-19T05:05:53.948971+0000","last_clean":"2024-09-19T05:05:53.948971+0000","last_became_active":"2024-09-19T05:04:07.982119+0000","last_became_peered":"2024-09-19T05:04:07.982119+0000","last_unstale":"2024-09-19T05:05:53.948971+0000","last_undegraded":"2024-09-19T05:05:53.948971+0000","last_fullsized":"2024-09-19T05:05:53.948971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:38.551692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"79'16","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691740+0000","last_change":"2024-09-19T05:04:07.984538+0000","last_active":"2024-09-19T05:05:53.691740+0000","last_peered":"2024-09-19T05:05:53.691740+0000","last_clean":"2024-09-19T05:05:53.691740+0000","last_became_active":"2024-09-19T05:04:07.984190+0000","last_became_peered":"2024-09-19T05:04:07.984190+0000","last_unstale":"2024-09-19T05:05:53.691740+0000","last_undegraded":"2024-09-19T05:05:53.691740+0000","last_fullsized":"2024-09-19T05:05:53.691740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:59:08.399669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946061+0000","last_change":"2024-09-19T05:04:07.989276+0000","last_active":"2024-09-19T05:05:53.946061+0000","last_peered":"2024-09-19T05:05:53.946061+0000","last_clean":"2024-09-19T05:05:53.946061+0000","last_became_active":"2024-09-19T05:04:07.988922+0000","last_became_peered":"2024-09-19T05:04:07.988922+0000","last_unstale":"2024-09-19T05:05:53.946061+0000","last_undegraded":"2024-09-19T05:05:53.946061+0000","last_fullsized":"2024-09-19T05:05:53.946061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:29.564560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946960+0000","last_change":"2024-09-19T05:04:07.981365+0000","last_active":"2024-09-19T05:05:53.946960+0000","last_peered":"2024-09-19T05:05:53.946960+0000","last_clean":"2024-09-19T05:05:53.946960+0000","last_became_active":"2024-09-19T05:04:07.981062+0000","last_became_peered":"2024-09-19T05:04:07.981062+0000","last_unstale":"2024-09-19T05:05:53.946960+0000","last_undegraded":"2024-09-19T05:05:53.946960+0000","last_fullsized":"2024-09-19T05:05:53.946960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:28.571982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971919+0000","last_change":"2024-09-19T05:04:07.982664+0000","last_active":"2024-09-19T05:05:53.971919+0000","last_peered":"2024-09-19T05:05:53.971919+0000","last_clean":"2024-09-19T05:05:53.971919+0000","last_became_active":"2024-09-19T05:04:07.982265+0000","last_became_peered":"2024-09-19T05:04:07.982265+0000","last_unstale":"2024-09-19T05:05:53.971919+0000","last_undegraded":"2024-09-19T05:05:53.971919+0000","last_fullsized":"2024-09-19T05:05:53.971919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.648577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972311+0000","last_change":"2024-09-19T05:04:07.980428+0000","last_active":"2024-09-19T05:05:53.972311+0000","last_peered":"2024-09-19T05:05:53.972311+0000","last_clean":"2024-09-19T05:05:53.972311+0000","last_became_active":"2024-09-19T05:04:07.980079+0000","last_became_peered":"2024-09-19T05:04:07.980079+0000","last_unstale":"2024-09-19T05:05:53.972311+0000","last_undegraded":"2024-09-19T05:05:53.972311+0000","last_fullsized":"2024-09-19T05:05:53.972311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:14.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972019+0000","last_change":"2024-09-19T05:04:07.984514+0000","last_active":"2024-09-19T05:05:53.972019+0000","last_peered":"2024-09-19T05:05:53.972019+0000","last_clean":"2024-09-19T05:05:53.972019+0000","last_became_active":"2024-09-19T05:04:07.983937+0000","last_became_peered":"2024-09-19T05:04:07.983937+0000","last_unstale":"2024-09-19T05:05:53.972019+0000","last_undegraded":"2024-09-19T05:05:53.972019+0000","last_fullsized":"2024-09-19T05:05:53.972019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:40.717307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948575+0000","last_change":"2024-09-19T05:04:07.978954+0000","last_active":"2024-09-19T05:05:53.948575+0000","last_peered":"2024-09-19T05:05:53.948575+0000","last_clean":"2024-09-19T05:05:53.948575+0000","last_became_active":"2024-09-19T05:04:07.978636+0000","last_became_peered":"2024-09-19T05:04:07.978636+0000","last_unstale":"2024-09-19T05:05:53.948575+0000","last_undegraded":"2024-09-19T05:05:53.948575+0000","last_fullsized":"2024-09-19T05:05:53.948575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:54.123614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689453+0000","last_change":"2024-09-19T05:04:07.984920+0000","last_active":"2024-09-19T05:05:53.689453+0000","last_peered":"2024-09-19T05:05:53.689453+0000","last_clean":"2024-09-19T05:05:53.689453+0000","last_became_active":"2024-09-19T05:04:07.984205+0000","last_became_peered":"2024-09-19T05:04:07.984205+0000","last_unstale":"2024-09-19T05:05:53.689453+0000","last_undegraded":"2024-09-19T05:05:53.689453+0000","last_fullsized":"2024-09-19T05:05:53.689453+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781607+0000","last_change":"2024-09-19T05:04:07.979382+0000","last_active":"2024-09-19T05:05:53.781607+0000","last_peered":"2024-09-19T05:05:53.781607+0000","last_clean":"2024-09-19T05:05:53.781607+0000","last_became_active":"2024-09-19T05:04:07.979068+0000","last_became_peered":"2024-09-19T05:04:07.979068+0000","last_unstale":"2024-09-19T05:05:53.781607+0000","last_undegraded":"2024-09-19T05:05:53.781607+0000","last_fullsized":"2024-09-19T05:05:53.781607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:00.163341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945659+0000","last_change":"2024-09-19T05:04:07.983335+0000","last_active":"2024-09-19T05:05:53.945659+0000","last_peered":"2024-09-19T05:05:53.945659+0000","last_clean":"2024-09-19T05:05:53.945659+0000","last_became_active":"2024-09-19T05:04:07.982997+0000","last_became_peered":"2024-09-19T05:04:07.982997+0000","last_unstale":"2024-09-19T05:05:53.945659+0000","last_undegraded":"2024-09-19T05:05:53.945659+0000","last_fullsized":"2024-09-19T05:05:53.945659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:38.834526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689033+0000","last_change":"2024-09-19T05:04:07.974137+0000","last_active":"2024-09-19T05:05:53.689033+0000","last_peered":"2024-09-19T05:05:53.689033+0000","last_clean":"2024-09-19T05:05:53.689033+0000","last_became_active":"2024-09-19T05:04:07.973756+0000","last_became_peered":"2024-09-19T05:04:07.973756+0000","last_unstale":"2024-09-19T05:05:53.689033+0000","last_undegraded":"2024-09-19T05:05:53.689033+0000","last_fullsized":"2024-09-19T05:05:53.689033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:30.269762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972075+0000","last_change":"2024-09-19T05:04:07.971995+0000","last_active":"2024-09-19T05:05:53.972075+0000","last_peered":"2024-09-19T05:05:53.972075+0000","last_clean":"2024-09-19T05:05:53.972075+0000","last_became_active":"2024-09-19T05:04:07.971565+0000","last_became_peered":"2024-09-19T05:04:07.971565+0000","last_unstale":"2024-09-19T05:05:53.972075+0000","last_undegraded":"2024-09-19T05:05:53.972075+0000","last_fullsized":"2024-09-19T05:05:53.972075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:36.874820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948706+0000","last_change":"2024-09-19T05:04:07.977994+0000","last_active":"2024-09-19T05:05:53.948706+0000","last_peered":"2024-09-19T05:05:53.948706+0000","last_clean":"2024-09-19T05:05:53.948706+0000","last_became_active":"2024-09-19T05:04:07.977641+0000","last_became_peered":"2024-09-19T05:04:07.977641+0000","last_unstale":"2024-09-19T05:05:53.948706+0000","last_undegraded":"2024-09-19T05:05:53.948706+0000","last_fullsized":"2024-09-19T05:05:53.948706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:42.896886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689785+0000","last_change":"2024-09-19T05:04:07.976148+0000","last_active":"2024-09-19T05:05:53.689785+0000","last_peered":"2024-09-19T05:05:53.689785+0000","last_clean":"2024-09-19T05:05:53.689785+0000","last_became_active":"2024-09-19T05:04:07.975848+0000","last_became_peered":"2024-09-19T05:04:07.975848+0000","last_unstale":"2024-09-19T05:05:53.689785+0000","last_undegraded":"2024-09-19T05:05:53.689785+0000","last_fullsized":"2024-09-19T05:05:53.689785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:55.894049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789295+0000","last_change":"2024-09-19T05:04:07.977288+0000","last_active":"2024-09-19T05:05:53.789295+0000","last_peered":"2024-09-19T05:05:53.789295+0000","last_clean":"2024-09-19T05:05:53.789295+0000","last_became_active":"2024-09-19T05:04:07.976940+0000","last_became_peered":"2024-09-19T05:04:07.976940+0000","last_unstale":"2024-09-19T05:05:53.789295+0000","last_undegraded":"2024-09-19T05:05:53.789295+0000","last_fullsized":"2024-09-19T05:05:53.789295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:09.532707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"79'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781636+0000","last_change":"2024-09-19T05:04:20.220466+0000","last_active":"2024-09-19T05:05:53.781636+0000","last_peered":"2024-09-19T05:05:53.781636+0000","last_clean":"2024-09-19T05:05:53.781636+0000","last_became_active":"2024-09-19T05:04:20.220137+0000","last_became_peered":"2024-09-19T05:04:20.220137+0000","last_unstale":"2024-09-19T05:05:53.781636+0000","last_undegraded":"2024-09-19T05:05:53.781636+0000","last_fullsized":"2024-09-19T05:05:53.781636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:01.540297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945753+0000","last_change":"2024-09-19T05:04:07.981108+0000","last_active":"2024-09-19T05:05:53.945753+0000","last_peered":"2024-09-19T05:05:53.945753+0000","last_clean":"2024-09-19T05:05:53.945753+0000","last_became_active":"2024-09-19T05:04:07.980510+0000","last_became_peered":"2024-09-19T05:04:07.980510+0000","last_unstale":"2024-09-19T05:05:53.945753+0000","last_undegraded":"2024-09-19T05:05:53.945753+0000","last_fullsized":"2024-09-19T05:05:53.945753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:40.347836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689249+0000","last_change":"2024-09-19T05:04:07.977305+0000","last_active":"2024-09-19T05:05:53.689249+0000","last_peered":"2024-09-19T05:05:53.689249+0000","last_clean":"2024-09-19T05:05:53.689249+0000","last_became_active":"2024-09-19T05:04:07.976902+0000","last_became_peered":"2024-09-19T05:04:07.976902+0000","last_unstale":"2024-09-19T05:05:53.689249+0000","last_undegraded":"2024-09-19T05:05:53.689249+0000","last_fullsized":"2024-09-19T05:05:53.689249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:09.001224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"79'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971123+0000","last_change":"2024-09-19T05:04:07.974885+0000","last_active":"2024-09-19T05:05:53.971123+0000","last_peered":"2024-09-19T05:05:53.971123+0000","last_clean":"2024-09-19T05:05:53.971123+0000","last_became_active":"2024-09-19T05:04:07.974480+0000","last_became_peered":"2024-09-19T05:04:07.974480+0000","last_unstale":"2024-09-19T05:05:53.971123+0000","last_undegraded":"2024-09-19T05:05:53.971123+0000","last_fullsized":"2024-09-19T05:05:53.971123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:37.853089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.972635+0000","last_change":"2024-09-19T05:04:07.974833+0000","last_active":"2024-09-19T05:05:53.972635+0000","last_peered":"2024-09-19T05:05:53.972635+0000","last_clean":"2024-09-19T05:05:53.972635+0000","last_became_active":"2024-09-19T05:04:07.974533+0000","last_became_peered":"2024-09-19T05:04:07.974533+0000","last_unstale":"2024-09-19T05:05:53.972635+0000","last_undegraded":"2024-09-19T05:05:53.972635+0000","last_fullsized":"2024-09-19T05:05:53.972635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:56:41.838806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945375+0000","last_change":"2024-09-19T05:04:07.977188+0000","last_active":"2024-09-19T05:05:53.945375+0000","last_peered":"2024-09-19T05:05:53.945375+0000","last_clean":"2024-09-19T05:05:53.945375+0000","last_became_active":"2024-09-19T05:04:07.975989+0000","last_became_peered":"2024-09-19T05:04:07.975989+0000","last_unstale":"2024-09-19T05:05:53.945375+0000","last_undegraded":"2024-09-19T05:05:53.945375+0000","last_fullsized":"2024-09-19T05:05:53.945375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.821615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"79'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945424+0000","last_change":"2024-09-19T05:04:07.981285+0000","last_active":"2024-09-19T05:05:53.945424+0000","last_peered":"2024-09-19T05:05:53.945424+0000","last_clean":"2024-09-19T05:05:53.945424+0000","last_became_active":"2024-09-19T05:04:07.980967+0000","last_became_peered":"2024-09-19T05:04:07.980967+0000","last_unstale":"2024-09-19T05:05:53.945424+0000","last_undegraded":"2024-09-19T05:05:53.945424+0000","last_fullsized":"2024-09-19T05:05:53.945424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:53.758786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689079+0000","last_change":"2024-09-19T05:04:07.979175+0000","last_active":"2024-09-19T05:05:53.689079+0000","last_peered":"2024-09-19T05:05:53.689079+0000","last_clean":"2024-09-19T05:05:53.689079+0000","last_became_active":"2024-09-19T05:04:07.978326+0000","last_became_peered":"2024-09-19T05:04:07.978326+0000","last_unstale":"2024-09-19T05:05:53.689079+0000","last_undegraded":"2024-09-19T05:05:53.689079+0000","last_fullsized":"2024-09-19T05:05:53.689079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:25.554298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783659+0000","last_change":"2024-09-19T05:04:07.978604+0000","last_active":"2024-09-19T05:05:53.783659+0000","last_peered":"2024-09-19T05:05:53.783659+0000","last_clean":"2024-09-19T05:05:53.783659+0000","last_became_active":"2024-09-19T05:04:07.976578+0000","last_became_peered":"2024-09-19T05:04:07.976578+0000","last_unstale":"2024-09-19T05:05:53.783659+0000","last_undegraded":"2024-09-19T05:05:53.783659+0000","last_fullsized":"2024-09-19T05:05:53.783659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:39.379794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'9","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.163491+0000","last_change":"2024-09-19T05:04:05.951898+0000","last_active":"2024-09-19T05:05:54.163491+0000","last_peered":"2024-09-19T05:05:54.163491+0000","last_clean":"2024-09-19T05:05:54.163491+0000","last_became_active":"2024-09-19T05:04:05.951551+0000","last_became_peered":"2024-09-19T05:04:05.951551+0000","last_unstale":"2024-09-19T05:05:54.163491+0000","last_undegraded":"2024-09-19T05:05:54.163491+0000","last_fullsized":"2024-09-19T05:05:54.163491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:10.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2044,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781306+0000","last_change":"2024-09-19T05:04:20.219825+0000","last_active":"2024-09-19T05:05:53.781306+0000","last_peered":"2024-09-19T05:05:53.781306+0000","last_clean":"2024-09-19T05:05:53.781306+0000","last_became_active":"2024-09-19T05:04:20.219459+0000","last_became_peered":"2024-09-19T05:04:20.219459+0000","last_unstale":"2024-09-19T05:05:53.781306+0000","last_undegraded":"2024-09-19T05:05:53.781306+0000","last_fullsized":"2024-09-19T05:05:53.781306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:01.404412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691365+0000","last_change":"2024-09-19T05:04:05.947145+0000","last_active":"2024-09-19T05:05:53.691365+0000","last_peered":"2024-09-19T05:05:53.691365+0000","last_clean":"2024-09-19T05:05:53.691365+0000","last_became_active":"2024-09-19T05:04:05.945551+0000","last_became_peered":"2024-09-19T05:04:05.945551+0000","last_unstale":"2024-09-19T05:05:53.691365+0000","last_undegraded":"2024-09-19T05:05:53.691365+0000","last_fullsized":"2024-09-19T05:05:53.691365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:39.957500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971411+0000","last_change":"2024-09-19T05:04:05.946286+0000","last_active":"2024-09-19T05:05:53.971411+0000","last_peered":"2024-09-19T05:05:53.971411+0000","last_clean":"2024-09-19T05:05:53.971411+0000","last_became_active":"2024-09-19T05:04:05.945926+0000","last_became_peered":"2024-09-19T05:04:05.945926+0000","last_unstale":"2024-09-19T05:05:53.971411+0000","last_undegraded":"2024-09-19T05:05:53.971411+0000","last_fullsized":"2024-09-19T05:05:53.971411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:38.446441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689842+0000","last_change":"2024-09-19T05:04:07.968178+0000","last_active":"2024-09-19T05:05:53.689842+0000","last_peered":"2024-09-19T05:05:53.689842+0000","last_clean":"2024-09-19T05:05:53.689842+0000","last_became_active":"2024-09-19T05:04:07.967871+0000","last_became_peered":"2024-09-19T05:04:07.967871+0000","last_unstale":"2024-09-19T05:05:53.689842+0000","last_undegraded":"2024-09-19T05:05:53.689842+0000","last_fullsized":"2024-09-19T05:05:53.689842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:07.374639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789504+0000","last_change":"2024-09-19T05:04:05.950910+0000","last_active":"2024-09-19T05:05:53.789504+0000","last_peered":"2024-09-19T05:05:53.789504+0000","last_clean":"2024-09-19T05:05:53.789504+0000","last_became_active":"2024-09-19T05:04:05.950354+0000","last_became_peered":"2024-09-19T05:04:05.950354+0000","last_unstale":"2024-09-19T05:05:53.789504+0000","last_undegraded":"2024-09-19T05:05:53.789504+0000","last_fullsized":"2024-09-19T05:05:53.789504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:20.568770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781090+0000","last_change":"2024-09-19T05:04:07.978960+0000","last_active":"2024-09-19T05:05:53.781090+0000","last_peered":"2024-09-19T05:05:53.781090+0000","last_clean":"2024-09-19T05:05:53.781090+0000","last_became_active":"2024-09-19T05:04:07.978628+0000","last_became_peered":"2024-09-19T05:04:07.978628+0000","last_unstale":"2024-09-19T05:05:53.781090+0000","last_undegraded":"2024-09-19T05:05:53.781090+0000","last_fullsized":"2024-09-19T05:05:53.781090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:41.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691578+0000","last_change":"2024-09-19T05:04:05.948045+0000","last_active":"2024-09-19T05:05:53.691578+0000","last_peered":"2024-09-19T05:05:53.691578+0000","last_clean":"2024-09-19T05:05:53.691578+0000","last_became_active":"2024-09-19T05:04:05.947738+0000","last_became_peered":"2024-09-19T05:04:05.947738+0000","last_unstale":"2024-09-19T05:05:53.691578+0000","last_undegraded":"2024-09-19T05:05:53.691578+0000","last_fullsized":"2024-09-19T05:05:53.691578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:28:46.528119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"79'10","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948853+0000","last_change":"2024-09-19T05:04:07.981461+0000","last_active":"2024-09-19T05:05:53.948853+0000","last_peered":"2024-09-19T05:05:53.948853+0000","last_clean":"2024-09-19T05:05:53.948853+0000","last_became_active":"2024-09-19T05:04:07.981107+0000","last_became_peered":"2024-09-19T05:04:07.981107+0000","last_unstale":"2024-09-19T05:05:53.948853+0000","last_undegraded":"2024-09-19T05:05:53.948853+0000","last_fullsized":"2024-09-19T05:05:53.948853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:50.269033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946143+0000","last_change":"2024-09-19T05:04:20.223302+0000","last_active":"2024-09-19T05:05:53.946143+0000","last_peered":"2024-09-19T05:05:53.946143+0000","last_clean":"2024-09-19T05:05:53.946143+0000","last_became_active":"2024-09-19T05:04:20.220488+0000","last_became_peered":"2024-09-19T05:04:20.220488+0000","last_unstale":"2024-09-19T05:05:53.946143+0000","last_undegraded":"2024-09-19T05:05:53.946143+0000","last_fullsized":"2024-09-19T05:05:53.946143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:04.092221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"79'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973668+0000","last_change":"2024-09-19T05:04:20.044799+0000","last_active":"2024-09-19T05:05:53.973668+0000","last_peered":"2024-09-19T05:05:53.973668+0000","last_clean":"2024-09-19T05:05:53.973668+0000","last_became_active":"2024-09-19T05:04:20.044380+0000","last_became_peered":"2024-09-19T05:04:20.044380+0000","last_unstale":"2024-09-19T05:05:53.973668+0000","last_undegraded":"2024-09-19T05:05:53.973668+0000","last_fullsized":"2024-09-19T05:05:53.973668+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:24.072688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789562+0000","last_change":"2024-09-19T05:04:05.973146+0000","last_active":"2024-09-19T05:05:53.789562+0000","last_peered":"2024-09-19T05:05:53.789562+0000","last_clean":"2024-09-19T05:05:53.789562+0000","last_became_active":"2024-09-19T05:04:05.972754+0000","last_became_peered":"2024-09-19T05:04:05.972754+0000","last_unstale":"2024-09-19T05:05:53.789562+0000","last_undegraded":"2024-09-19T05:05:53.789562+0000","last_fullsized":"2024-09-19T05:05:53.789562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:39.492656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"79'11","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948706+0000","last_change":"2024-09-19T05:04:07.975780+0000","last_active":"2024-09-19T05:05:53.948706+0000","last_peered":"2024-09-19T05:05:53.948706+0000","last_clean":"2024-09-19T05:05:53.948706+0000","last_became_active":"2024-09-19T05:04:07.975427+0000","last_became_peered":"2024-09-19T05:04:07.975427+0000","last_unstale":"2024-09-19T05:05:53.948706+0000","last_undegraded":"2024-09-19T05:05:53.948706+0000","last_fullsized":"2024-09-19T05:05:53.948706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:46.398497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'37","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.166232+0000","last_change":"2024-09-19T05:04:05.950019+0000","last_active":"2024-09-19T05:05:54.166232+0000","last_peered":"2024-09-19T05:05:54.166232+0000","last_clean":"2024-09-19T05:05:54.166232+0000","last_became_active":"2024-09-19T05:04:05.949662+0000","last_became_peered":"2024-09-19T05:04:05.949662+0000","last_unstale":"2024-09-19T05:05:54.166232+0000","last_undegraded":"2024-09-19T05:05:54.166232+0000","last_fullsized":"2024-09-19T05:05:54.166232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:20.061637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67761,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":169,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783871+0000","last_change":"2024-09-19T05:04:07.975798+0000","last_active":"2024-09-19T05:05:53.783871+0000","last_peered":"2024-09-19T05:05:53.783871+0000","last_clean":"2024-09-19T05:05:53.783871+0000","last_became_active":"2024-09-19T05:04:07.975505+0000","last_became_peered":"2024-09-19T05:04:07.975505+0000","last_unstale":"2024-09-19T05:05:53.783871+0000","last_undegraded":"2024-09-19T05:05:53.783871+0000","last_fullsized":"2024-09-19T05:05:53.783871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:14.334214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689009+0000","last_change":"2024-09-19T05:04:20.058790+0000","last_active":"2024-09-19T05:05:53.689009+0000","last_peered":"2024-09-19T05:05:53.689009+0000","last_clean":"2024-09-19T05:05:53.689009+0000","last_became_active":"2024-09-19T05:04:20.048002+0000","last_became_peered":"2024-09-19T05:04:20.048002+0000","last_unstale":"2024-09-19T05:05:53.689009+0000","last_undegraded":"2024-09-19T05:05:53.689009+0000","last_fullsized":"2024-09-19T05:05:53.689009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:16.562605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971016+0000","last_change":"2024-09-19T05:04:07.972924+0000","last_active":"2024-09-19T05:05:53.971016+0000","last_peered":"2024-09-19T05:05:53.971016+0000","last_clean":"2024-09-19T05:05:53.971016+0000","last_became_active":"2024-09-19T05:04:07.972661+0000","last_became_peered":"2024-09-19T05:04:07.972661+0000","last_unstale":"2024-09-19T05:05:53.971016+0000","last_undegraded":"2024-09-19T05:05:53.971016+0000","last_fullsized":"2024-09-19T05:05:53.971016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:22.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789394+0000","last_change":"2024-09-19T05:04:05.951980+0000","last_active":"2024-09-19T05:05:53.789394+0000","last_peered":"2024-09-19T05:05:53.789394+0000","last_clean":"2024-09-19T05:05:53.789394+0000","last_became_active":"2024-09-19T05:04:05.951279+0000","last_became_peered":"2024-09-19T05:04:05.951279+0000","last_unstale":"2024-09-19T05:05:53.789394+0000","last_undegraded":"2024-09-19T05:05:53.789394+0000","last_fullsized":"2024-09-19T05:05:53.789394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:24.186421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945962+0000","last_change":"2024-09-19T05:04:07.974896+0000","last_active":"2024-09-19T05:05:53.945962+0000","last_peered":"2024-09-19T05:05:53.945962+0000","last_clean":"2024-09-19T05:05:53.945962+0000","last_became_active":"2024-09-19T05:04:07.974345+0000","last_became_peered":"2024-09-19T05:04:07.974345+0000","last_unstale":"2024-09-19T05:05:53.945962+0000","last_undegraded":"2024-09-19T05:05:53.945962+0000","last_fullsized":"2024-09-19T05:05:53.945962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:43.190273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:54.183533+0000","last_change":"2024-09-19T05:04:05.949955+0000","last_active":"2024-09-19T05:05:54.183533+0000","last_peered":"2024-09-19T05:05:54.183533+0000","last_clean":"2024-09-19T05:05:54.183533+0000","last_became_active":"2024-09-19T05:04:05.949565+0000","last_became_peered":"2024-09-19T05:04:05.949565+0000","last_unstale":"2024-09-19T05:05:54.183533+0000","last_undegraded":"2024-09-19T05:05:54.183533+0000","last_fullsized":"2024-09-19T05:05:54.183533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:46.765008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945636+0000","last_change":"2024-09-19T05:04:07.981306+0000","last_active":"2024-09-19T05:05:53.945636+0000","last_peered":"2024-09-19T05:05:53.945636+0000","last_clean":"2024-09-19T05:05:53.945636+0000","last_became_active":"2024-09-19T05:04:07.980952+0000","last_became_peered":"2024-09-19T05:04:07.980952+0000","last_unstale":"2024-09-19T05:05:53.945636+0000","last_undegraded":"2024-09-19T05:05:53.945636+0000","last_fullsized":"2024-09-19T05:05:53.945636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.487060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783541+0000","last_change":"2024-09-19T05:04:05.973319+0000","last_active":"2024-09-19T05:05:53.783541+0000","last_peered":"2024-09-19T05:05:53.783541+0000","last_clean":"2024-09-19T05:05:53.783541+0000","last_became_active":"2024-09-19T05:04:05.972900+0000","last_became_peered":"2024-09-19T05:04:05.972900+0000","last_unstale":"2024-09-19T05:05:53.783541+0000","last_undegraded":"2024-09-19T05:05:53.783541+0000","last_fullsized":"2024-09-19T05:05:53.783541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:31.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"79'6","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.780938+0000","last_change":"2024-09-19T05:04:20.220550+0000","last_active":"2024-09-19T05:05:53.780938+0000","last_peered":"2024-09-19T05:05:53.780938+0000","last_clean":"2024-09-19T05:05:53.780938+0000","last_became_active":"2024-09-19T05:04:20.220204+0000","last_became_peered":"2024-09-19T05:04:20.220204+0000","last_unstale":"2024-09-19T05:05:53.780938+0000","last_undegraded":"2024-09-19T05:05:53.780938+0000","last_fullsized":"2024-09-19T05:05:53.780938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:29.477353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.784003+0000","last_change":"2024-09-19T05:04:05.950714+0000","last_active":"2024-09-19T05:05:53.784003+0000","last_peered":"2024-09-19T05:05:53.784003+0000","last_clean":"2024-09-19T05:05:53.784003+0000","last_became_active":"2024-09-19T05:04:05.950386+0000","last_became_peered":"2024-09-19T05:04:05.950386+0000","last_unstale":"2024-09-19T05:05:53.784003+0000","last_undegraded":"2024-09-19T05:05:53.784003+0000","last_fullsized":"2024-09-19T05:05:53.784003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:00.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689229+0000","last_change":"2024-09-19T05:04:07.978501+0000","last_active":"2024-09-19T05:05:53.689229+0000","last_peered":"2024-09-19T05:05:53.689229+0000","last_clean":"2024-09-19T05:05:53.689229+0000","last_became_active":"2024-09-19T05:04:07.977962+0000","last_became_peered":"2024-09-19T05:04:07.977962+0000","last_unstale":"2024-09-19T05:05:53.689229+0000","last_undegraded":"2024-09-19T05:05:53.689229+0000","last_fullsized":"2024-09-19T05:05:53.689229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:57:04.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971302+0000","last_change":"2024-09-19T05:04:05.950060+0000","last_active":"2024-09-19T05:05:53.971302+0000","last_peered":"2024-09-19T05:05:53.971302+0000","last_clean":"2024-09-19T05:05:53.971302+0000","last_became_active":"2024-09-19T05:04:05.949626+0000","last_became_peered":"2024-09-19T05:04:05.949626+0000","last_unstale":"2024-09-19T05:05:53.971302+0000","last_undegraded":"2024-09-19T05:05:53.971302+0000","last_fullsized":"2024-09-19T05:05:53.971302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:14.324977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945960+0000","last_change":"2024-09-19T05:04:07.976773+0000","last_active":"2024-09-19T05:05:53.945960+0000","last_peered":"2024-09-19T05:05:53.945960+0000","last_clean":"2024-09-19T05:05:53.945960+0000","last_became_active":"2024-09-19T05:04:07.976217+0000","last_became_peered":"2024-09-19T05:04:07.976217+0000","last_unstale":"2024-09-19T05:05:53.945960+0000","last_undegraded":"2024-09-19T05:05:53.945960+0000","last_fullsized":"2024-09-19T05:05:53.945960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:12.864754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'39","reported_seq":71,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.894173+0000","last_change":"2024-09-19T05:04:05.948163+0000","last_active":"2024-09-19T05:05:53.894173+0000","last_peered":"2024-09-19T05:05:53.894173+0000","last_clean":"2024-09-19T05:05:53.894173+0000","last_became_active":"2024-09-19T05:04:05.947789+0000","last_became_peered":"2024-09-19T05:04:05.947789+0000","last_unstale":"2024-09-19T05:05:53.894173+0000","last_undegraded":"2024-09-19T05:05:53.894173+0000","last_fullsized":"2024-09-19T05:05:53.894173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:11.783737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945898+0000","last_change":"2024-09-19T05:04:07.984166+0000","last_active":"2024-09-19T05:05:53.945898+0000","last_peered":"2024-09-19T05:05:53.945898+0000","last_clean":"2024-09-19T05:05:53.945898+0000","last_became_active":"2024-09-19T05:04:07.983830+0000","last_became_peered":"2024-09-19T05:04:07.983830+0000","last_unstale":"2024-09-19T05:05:53.945898+0000","last_undegraded":"2024-09-19T05:05:53.945898+0000","last_fullsized":"2024-09-19T05:05:53.945898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:53.740724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945656+0000","last_change":"2024-09-19T05:04:05.972400+0000","last_active":"2024-09-19T05:05:53.945656+0000","last_peered":"2024-09-19T05:05:53.945656+0000","last_clean":"2024-09-19T05:05:53.945656+0000","last_became_active":"2024-09-19T05:04:05.972055+0000","last_became_peered":"2024-09-19T05:04:05.972055+0000","last_unstale":"2024-09-19T05:05:53.945656+0000","last_undegraded":"2024-09-19T05:05:53.945656+0000","last_fullsized":"2024-09-19T05:05:53.945656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:24.895394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946106+0000","last_change":"2024-09-19T05:04:07.988598+0000","last_active":"2024-09-19T05:05:53.946106+0000","last_peered":"2024-09-19T05:05:53.946106+0000","last_clean":"2024-09-19T05:05:53.946106+0000","last_became_active":"2024-09-19T05:04:07.988196+0000","last_became_peered":"2024-09-19T05:04:07.988196+0000","last_unstale":"2024-09-19T05:05:53.946106+0000","last_undegraded":"2024-09-19T05:05:53.946106+0000","last_fullsized":"2024-09-19T05:05:53.946106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:09.476211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":346,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945800+0000","last_change":"2024-09-19T05:02:57.637506+0000","last_active":"2024-09-19T05:05:53.945800+0000","last_peered":"2024-09-19T05:05:53.945800+0000","last_clean":"2024-09-19T05:05:53.945800+0000","last_became_active":"2024-09-19T05:02:57.636984+0000","last_became_peered":"2024-09-19T05:02:57.636984+0000","last_unstale":"2024-09-19T05:05:53.945800+0000","last_undegraded":"2024-09-19T05:05:53.945800+0000","last_fullsized":"2024-09-19T05:05:53.945800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:59:35.270026+0000","last_clean_scrub_stamp":"2024-09-19T04:59:35.270026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.108922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"77'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946523+0000","last_change":"2024-09-19T05:04:05.950945+0000","last_active":"2024-09-19T05:05:53.946523+0000","last_peered":"2024-09-19T05:05:53.946523+0000","last_clean":"2024-09-19T05:05:53.946523+0000","last_became_active":"2024-09-19T05:04:05.950603+0000","last_became_peered":"2024-09-19T05:04:05.950603+0000","last_unstale":"2024-09-19T05:05:53.946523+0000","last_undegraded":"2024-09-19T05:05:53.946523+0000","last_fullsized":"2024-09-19T05:05:53.946523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:55.630729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690501+0000","last_change":"2024-09-19T05:04:07.978303+0000","last_active":"2024-09-19T05:05:53.690501+0000","last_peered":"2024-09-19T05:05:53.690501+0000","last_clean":"2024-09-19T05:05:53.690501+0000","last_became_active":"2024-09-19T05:04:07.977717+0000","last_became_peered":"2024-09-19T05:04:07.977717+0000","last_unstale":"2024-09-19T05:05:53.690501+0000","last_undegraded":"2024-09-19T05:05:53.690501+0000","last_fullsized":"2024-09-19T05:05:53.690501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:29.883755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945921+0000","last_change":"2024-09-19T05:04:20.048907+0000","last_active":"2024-09-19T05:05:53.945921+0000","last_peered":"2024-09-19T05:05:53.945921+0000","last_clean":"2024-09-19T05:05:53.945921+0000","last_became_active":"2024-09-19T05:04:20.044137+0000","last_became_peered":"2024-09-19T05:04:20.044137+0000","last_unstale":"2024-09-19T05:05:53.945921+0000","last_undegraded":"2024-09-19T05:05:53.945921+0000","last_fullsized":"2024-09-19T05:05:53.945921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.364784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.970734+0000","last_change":"2024-09-19T05:04:07.974413+0000","last_active":"2024-09-19T05:05:53.970734+0000","last_peered":"2024-09-19T05:05:53.970734+0000","last_clean":"2024-09-19T05:05:53.970734+0000","last_became_active":"2024-09-19T05:04:07.974046+0000","last_became_peered":"2024-09-19T05:04:07.974046+0000","last_unstale":"2024-09-19T05:05:53.970734+0000","last_undegraded":"2024-09-19T05:05:53.970734+0000","last_fullsized":"2024-09-19T05:05:53.970734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:05.364591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":126,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971558+0000","last_change":"2024-09-19T05:04:05.972516+0000","last_active":"2024-09-19T05:05:53.971558+0000","last_peered":"2024-09-19T05:05:53.971558+0000","last_clean":"2024-09-19T05:05:53.971558+0000","last_became_active":"2024-09-19T05:04:05.972085+0000","last_became_peered":"2024-09-19T05:04:05.972085+0000","last_unstale":"2024-09-19T05:05:53.971558+0000","last_undegraded":"2024-09-19T05:05:53.971558+0000","last_fullsized":"2024-09-19T05:05:53.971558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.772755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789355+0000","last_change":"2024-09-19T05:04:07.985333+0000","last_active":"2024-09-19T05:05:53.789355+0000","last_peered":"2024-09-19T05:05:53.789355+0000","last_clean":"2024-09-19T05:05:53.789355+0000","last_became_active":"2024-09-19T05:04:07.984919+0000","last_became_peered":"2024-09-19T05:04:07.984919+0000","last_unstale":"2024-09-19T05:05:53.789355+0000","last_undegraded":"2024-09-19T05:05:53.789355+0000","last_fullsized":"2024-09-19T05:05:53.789355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945849+0000","last_change":"2024-09-19T05:04:05.946609+0000","last_active":"2024-09-19T05:05:53.945849+0000","last_peered":"2024-09-19T05:05:53.945849+0000","last_clean":"2024-09-19T05:05:53.945849+0000","last_became_active":"2024-09-19T05:04:05.946266+0000","last_became_peered":"2024-09-19T05:04:05.946266+0000","last_unstale":"2024-09-19T05:05:53.945849+0000","last_undegraded":"2024-09-19T05:05:53.945849+0000","last_fullsized":"2024-09-19T05:05:53.945849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:40.799283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"79'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691485+0000","last_change":"2024-09-19T05:04:07.985713+0000","last_active":"2024-09-19T05:05:53.691485+0000","last_peered":"2024-09-19T05:05:53.691485+0000","last_clean":"2024-09-19T05:05:53.691485+0000","last_became_active":"2024-09-19T05:04:07.985274+0000","last_became_peered":"2024-09-19T05:04:07.985274+0000","last_unstale":"2024-09-19T05:05:53.691485+0000","last_undegraded":"2024-09-19T05:05:53.691485+0000","last_fullsized":"2024-09-19T05:05:53.691485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:51.977488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781445+0000","last_change":"2024-09-19T05:04:05.973555+0000","last_active":"2024-09-19T05:05:53.781445+0000","last_peered":"2024-09-19T05:05:53.781445+0000","last_clean":"2024-09-19T05:05:53.781445+0000","last_became_active":"2024-09-19T05:04:05.973016+0000","last_became_peered":"2024-09-19T05:04:05.973016+0000","last_unstale":"2024-09-19T05:05:53.781445+0000","last_undegraded":"2024-09-19T05:05:53.781445+0000","last_fullsized":"2024-09-19T05:05:53.781445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:46.509294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.691488+0000","last_change":"2024-09-19T05:04:07.981758+0000","last_active":"2024-09-19T05:05:53.691488+0000","last_peered":"2024-09-19T05:05:53.691488+0000","last_clean":"2024-09-19T05:05:53.691488+0000","last_became_active":"2024-09-19T05:04:07.981452+0000","last_became_peered":"2024-09-19T05:04:07.981452+0000","last_unstale":"2024-09-19T05:05:53.691488+0000","last_undegraded":"2024-09-19T05:05:53.691488+0000","last_fullsized":"2024-09-19T05:05:53.691488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:16.509398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"77'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783918+0000","last_change":"2024-09-19T05:04:05.973307+0000","last_active":"2024-09-19T05:05:53.783918+0000","last_peered":"2024-09-19T05:05:53.783918+0000","last_clean":"2024-09-19T05:05:53.783918+0000","last_became_active":"2024-09-19T05:04:05.972501+0000","last_became_peered":"2024-09-19T05:04:05.972501+0000","last_unstale":"2024-09-19T05:05:53.783918+0000","last_undegraded":"2024-09-19T05:05:53.783918+0000","last_fullsized":"2024-09-19T05:05:53.783918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:11.301832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.971243+0000","last_change":"2024-09-19T05:04:07.972481+0000","last_active":"2024-09-19T05:05:53.971243+0000","last_peered":"2024-09-19T05:05:53.971243+0000","last_clean":"2024-09-19T05:05:53.971243+0000","last_became_active":"2024-09-19T05:04:07.972138+0000","last_became_peered":"2024-09-19T05:04:07.972138+0000","last_unstale":"2024-09-19T05:05:53.971243+0000","last_undegraded":"2024-09-19T05:05:53.971243+0000","last_fullsized":"2024-09-19T05:05:53.971243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:30.004842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'47","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.901455+0000","last_change":"2024-09-19T05:04:05.945313+0000","last_active":"2024-09-19T05:05:53.901455+0000","last_peered":"2024-09-19T05:05:53.901455+0000","last_clean":"2024-09-19T05:05:53.901455+0000","last_became_active":"2024-09-19T05:04:05.944983+0000","last_became_peered":"2024-09-19T05:04:05.944983+0000","last_unstale":"2024-09-19T05:05:53.901455+0000","last_undegraded":"2024-09-19T05:05:53.901455+0000","last_fullsized":"2024-09-19T05:05:53.901455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:38.497805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690050+0000","last_change":"2024-09-19T05:04:07.970110+0000","last_active":"2024-09-19T05:05:53.690050+0000","last_peered":"2024-09-19T05:05:53.690050+0000","last_clean":"2024-09-19T05:05:53.690050+0000","last_became_active":"2024-09-19T05:04:07.969800+0000","last_became_peered":"2024-09-19T05:04:07.969800+0000","last_unstale":"2024-09-19T05:05:53.690050+0000","last_undegraded":"2024-09-19T05:05:53.690050+0000","last_fullsized":"2024-09-19T05:05:53.690050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:10.149705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.946983+0000","last_change":"2024-09-19T05:04:05.946109+0000","last_active":"2024-09-19T05:05:53.946983+0000","last_peered":"2024-09-19T05:05:53.946983+0000","last_clean":"2024-09-19T05:05:53.946983+0000","last_became_active":"2024-09-19T05:04:05.945781+0000","last_became_peered":"2024-09-19T05:04:05.945781+0000","last_unstale":"2024-09-19T05:05:53.946983+0000","last_undegraded":"2024-09-19T05:05:53.946983+0000","last_fullsized":"2024-09-19T05:05:53.946983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:09.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"79'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973426+0000","last_change":"2024-09-19T05:04:20.047866+0000","last_active":"2024-09-19T05:05:53.973426+0000","last_peered":"2024-09-19T05:05:53.973426+0000","last_clean":"2024-09-19T05:05:53.973426+0000","last_became_active":"2024-09-19T05:04:20.047449+0000","last_became_peered":"2024-09-19T05:04:20.047449+0000","last_unstale":"2024-09-19T05:05:53.973426+0000","last_undegraded":"2024-09-19T05:05:53.973426+0000","last_fullsized":"2024-09-19T05:05:53.973426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:02.893503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948732+0000","last_change":"2024-09-19T05:04:05.972928+0000","last_active":"2024-09-19T05:05:53.948732+0000","last_peered":"2024-09-19T05:05:53.948732+0000","last_clean":"2024-09-19T05:05:53.948732+0000","last_became_active":"2024-09-19T05:04:05.972675+0000","last_became_peered":"2024-09-19T05:04:05.972675+0000","last_unstale":"2024-09-19T05:05:53.948732+0000","last_undegraded":"2024-09-19T05:05:53.948732+0000","last_fullsized":"2024-09-19T05:05:53.948732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:35.264508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"79'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783541+0000","last_change":"2024-09-19T05:04:07.975133+0000","last_active":"2024-09-19T05:05:53.783541+0000","last_peered":"2024-09-19T05:05:53.783541+0000","last_clean":"2024-09-19T05:05:53.783541+0000","last_became_active":"2024-09-19T05:04:07.974828+0000","last_became_peered":"2024-09-19T05:04:07.974828+0000","last_unstale":"2024-09-19T05:05:53.783541+0000","last_undegraded":"2024-09-19T05:05:53.783541+0000","last_fullsized":"2024-09-19T05:05:53.783541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:03.100725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689947+0000","last_change":"2024-09-19T05:04:05.945025+0000","last_active":"2024-09-19T05:05:53.689947+0000","last_peered":"2024-09-19T05:05:53.689947+0000","last_clean":"2024-09-19T05:05:53.689947+0000","last_became_active":"2024-09-19T05:04:05.944662+0000","last_became_peered":"2024-09-19T05:04:05.944662+0000","last_unstale":"2024-09-19T05:05:53.689947+0000","last_undegraded":"2024-09-19T05:05:53.689947+0000","last_fullsized":"2024-09-19T05:05:53.689947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:19.880156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689258+0000","last_change":"2024-09-19T05:04:07.972812+0000","last_active":"2024-09-19T05:05:53.689258+0000","last_peered":"2024-09-19T05:05:53.689258+0000","last_clean":"2024-09-19T05:05:53.689258+0000","last_became_active":"2024-09-19T05:04:07.972451+0000","last_became_peered":"2024-09-19T05:04:07.972451+0000","last_unstale":"2024-09-19T05:05:53.689258+0000","last_undegraded":"2024-09-19T05:05:53.689258+0000","last_fullsized":"2024-09-19T05:05:53.689258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:08.545116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945400+0000","last_change":"2024-09-19T05:04:05.948971+0000","last_active":"2024-09-19T05:05:53.945400+0000","last_peered":"2024-09-19T05:05:53.945400+0000","last_clean":"2024-09-19T05:05:53.945400+0000","last_became_active":"2024-09-19T05:04:05.948327+0000","last_became_peered":"2024-09-19T05:04:05.948327+0000","last_unstale":"2024-09-19T05:05:53.945400+0000","last_undegraded":"2024-09-19T05:05:53.945400+0000","last_fullsized":"2024-09-19T05:05:53.945400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:55.315815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"79'8","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689907+0000","last_change":"2024-09-19T05:04:07.978786+0000","last_active":"2024-09-19T05:05:53.689907+0000","last_peered":"2024-09-19T05:05:53.689907+0000","last_clean":"2024-09-19T05:05:53.689907+0000","last_became_active":"2024-09-19T05:04:07.978497+0000","last_became_peered":"2024-09-19T05:04:07.978497+0000","last_unstale":"2024-09-19T05:05:53.689907+0000","last_undegraded":"2024-09-19T05:05:53.689907+0000","last_fullsized":"2024-09-19T05:05:53.689907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:21:50.766675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783621+0000","last_change":"2024-09-19T05:04:05.952203+0000","last_active":"2024-09-19T05:05:53.783621+0000","last_peered":"2024-09-19T05:05:53.783621+0000","last_clean":"2024-09-19T05:05:53.783621+0000","last_became_active":"2024-09-19T05:04:05.951835+0000","last_became_peered":"2024-09-19T05:04:05.951835+0000","last_unstale":"2024-09-19T05:05:53.783621+0000","last_undegraded":"2024-09-19T05:05:53.783621+0000","last_fullsized":"2024-09-19T05:05:53.783621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:18.316908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.945782+0000","last_change":"2024-09-19T05:04:07.987183+0000","last_active":"2024-09-19T05:05:53.945782+0000","last_peered":"2024-09-19T05:05:53.945782+0000","last_clean":"2024-09-19T05:05:53.945782+0000","last_became_active":"2024-09-19T05:04:07.986819+0000","last_became_peered":"2024-09-19T05:04:07.986819+0000","last_unstale":"2024-09-19T05:05:53.945782+0000","last_undegraded":"2024-09-19T05:05:53.945782+0000","last_fullsized":"2024-09-19T05:05:53.945782+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:34.665191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"79'15","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.789418+0000","last_change":"2024-09-19T05:04:07.979471+0000","last_active":"2024-09-19T05:05:53.789418+0000","last_peered":"2024-09-19T05:05:53.789418+0000","last_clean":"2024-09-19T05:05:53.789418+0000","last_became_active":"2024-09-19T05:04:07.978923+0000","last_became_peered":"2024-09-19T05:04:07.978923+0000","last_unstale":"2024-09-19T05:05:53.789418+0000","last_undegraded":"2024-09-19T05:05:53.789418+0000","last_fullsized":"2024-09-19T05:05:53.789418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:42.213328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.783794+0000","last_change":"2024-09-19T05:04:05.948541+0000","last_active":"2024-09-19T05:05:53.783794+0000","last_peered":"2024-09-19T05:05:53.783794+0000","last_clean":"2024-09-19T05:05:53.783794+0000","last_became_active":"2024-09-19T05:04:05.948172+0000","last_became_peered":"2024-09-19T05:04:05.948172+0000","last_unstale":"2024-09-19T05:05:53.783794+0000","last_undegraded":"2024-09-19T05:05:53.783794+0000","last_fullsized":"2024-09-19T05:05:53.783794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:33.068343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"79'9","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.948663+0000","last_change":"2024-09-19T05:04:07.983740+0000","last_active":"2024-09-19T05:05:53.948663+0000","last_peered":"2024-09-19T05:05:53.948663+0000","last_clean":"2024-09-19T05:05:53.948663+0000","last_became_active":"2024-09-19T05:04:07.983380+0000","last_became_peered":"2024-09-19T05:04:07.983380+0000","last_unstale":"2024-09-19T05:05:53.948663+0000","last_undegraded":"2024-09-19T05:05:53.948663+0000","last_fullsized":"2024-09-19T05:05:53.948663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:35.120191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.689087+0000","last_change":"2024-09-19T05:04:20.052380+0000","last_active":"2024-09-19T05:05:53.689087+0000","last_peered":"2024-09-19T05:05:53.689087+0000","last_clean":"2024-09-19T05:05:53.689087+0000","last_became_active":"2024-09-19T05:04:20.048074+0000","last_became_peered":"2024-09-19T05:04:20.048074+0000","last_unstale":"2024-09-19T05:05:53.689087+0000","last_undegraded":"2024-09-19T05:05:53.689087+0000","last_fullsized":"2024-09-19T05:05:53.689087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:46.109688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"79'6","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.973779+0000","last_change":"2024-09-19T05:04:07.985866+0000","last_active":"2024-09-19T05:05:53.973779+0000","last_peered":"2024-09-19T05:05:53.973779+0000","last_clean":"2024-09-19T05:05:53.973779+0000","last_became_active":"2024-09-19T05:04:07.985576+0000","last_became_peered":"2024-09-19T05:04:07.985576+0000","last_unstale":"2024-09-19T05:05:53.973779+0000","last_undegraded":"2024-09-19T05:05:53.973779+0000","last_fullsized":"2024-09-19T05:05:53.973779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:47.779713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690350+0000","last_change":"2024-09-19T05:04:05.972433+0000","last_active":"2024-09-19T05:05:53.690350+0000","last_peered":"2024-09-19T05:05:53.690350+0000","last_clean":"2024-09-19T05:05:53.690350+0000","last_became_active":"2024-09-19T05:04:05.972059+0000","last_became_peered":"2024-09-19T05:04:05.972059+0000","last_unstale":"2024-09-19T05:05:53.690350+0000","last_undegraded":"2024-09-19T05:05:53.690350+0000","last_fullsized":"2024-09-19T05:05:53.690350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:16:16.486977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"79'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.690198+0000","last_change":"2024-09-19T05:04:07.978249+0000","last_active":"2024-09-19T05:05:53.690198+0000","last_peered":"2024-09-19T05:05:53.690198+0000","last_clean":"2024-09-19T05:05:53.690198+0000","last_became_active":"2024-09-19T05:04:07.977376+0000","last_became_peered":"2024-09-19T05:04:07.977376+0000","last_unstale":"2024-09-19T05:05:53.690198+0000","last_undegraded":"2024-09-19T05:05:53.690198+0000","last_fullsized":"2024-09-19T05:05:53.690198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:06.944664+0000","last_clean_scrub_stamp":"2024-09-19T05:04:06.944664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:54.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T05:05:53.781831+0000","last_change":"2024-09-19T05:04:05.945949+0000","last_active":"2024-09-19T05:05:53.781831+0000","last_peered":"2024-09-19T05:05:53.781831+0000","last_clean":"2024-09-19T05:05:53.781831+0000","last_became_active":"2024-09-19T05:04:05.945585+0000","last_became_peered":"2024-09-19T05:04:05.945585+0000","last_unstale":"2024-09-19T05:05:53.781831+0000","last_undegraded":"2024-09-19T05:05:53.781831+0000","last_fullsized":"2024-09-19T05:05:53.781831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:04:04.930405+0000","last_clean_scrub_stamp":"2024-09-19T05:04:04.930405+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:38.898155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598497,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":709,"num_read_kb":465,"num_write":295,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":34,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1795491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678050,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":431785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1724416,"data_stored":1223481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1294336,"data_stored":825203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712700,"statfs":{"total":95995035648,"available":95961804800,"internally_reserved":0,"allocated":1294336,"data_stored":825123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528373,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1478656,"data_stored":1015985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":431376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888127,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1642496,"data_stored":1156152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":892928,"data_stored":432701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712568,"statfs":{"total":95995035648,"available":95961669632,"internally_reserved":0,"allocated":1433600,"data_stored":956503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":501668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":498930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":909312,"data_stored":432647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:05:59.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:59.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:05:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:05:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:58 smithi103 ceph-mon[34446]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-19T05:05:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:59.793 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:59 smithi064 ceph-mon[30960]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:59.793 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:05:59 smithi064 ceph-mon[30960]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:05:59.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 -- 172.21.15.64:0/284829976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc100310 msgr2=0x7f2bfc1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/284829976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc100310 0x7f2bfc1006f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009900 tx=0x7f2bf802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 -- 172.21.15.64:0/284829976 shutdown_connections 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/284829976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bfc106f30 0x7f2bfc10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/284829976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bfc100cc0 0x7f2bfc106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/284829976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc100310 0x7f2bfc1006f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 -- 172.21.15.64:0/284829976 >> 172.21.15.64:0/284829976 conn(0x7f2bfc077210 msgr2=0x7f2bfc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 -- 172.21.15.64:0/284829976 shutdown_connections 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.874+0000 7f2c02c6d640 1 -- 172.21.15.64:0/284829976 wait complete. 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 Processor -- start 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 -- start start 2024-09-19T05:05:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bfc100310 0x7f2bfc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bfc100cc0 0x7f2bfc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 0x7f2bfc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc111e60 con 0x7f2bfc106f30 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc111ce0 con 0x7f2bfc100310 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.875+0000 7f2c02c6d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc111fe0 con 0x7f2bfc100cc0 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 0x7f2bfc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c00a09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bfc100310 0x7f2bfc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 0x7f2bfc19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42558/0 (socket says 172.21.15.64:42558) 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 -- 172.21.15.64:0/2641899585 learned_addr learned my addr 172.21.15.64:0/2641899585 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2bf3fff640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bfc100cc0 0x7f2bfc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 -- 172.21.15.64:0/2641899585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bfc100cc0 msgr2=0x7f2bfc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bfc100cc0 0x7f2bfc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 -- 172.21.15.64:0/2641899585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bfc100310 msgr2=0x7f2bfc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bfc100310 0x7f2bfc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 -- 172.21.15.64:0/2641899585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc19d3d0 con 0x7f2bfc106f30 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c0120a640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 0x7f2bfc19ccf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400eaa0 tx=0x7f2bf400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4004d20 con 0x7f2bfc106f30 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bf4005020 con 0x7f2bfc106f30 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40116b0 con 0x7f2bfc106f30 2024-09-19T05:05:59.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.876+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc19d6c0 con 0x7f2bfc106f30 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.877+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1a4fa0 con 0x7f2bfc106f30 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.878+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc1006f0 con 0x7f2bfc106f30 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.879+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f2bf40051d0 con 0x7f2bfc106f30 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.879+0000 7f2bf1ffb640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2be40a3550 0x7f2be40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.879+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2bf40d0170 con 0x7f2bfc106f30 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.879+0000 7f2c00a09640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2be40a3550 0x7f2be40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:59.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.880+0000 7f2c00a09640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2be40a3550 0x7f2be40a5a10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8031440 tx=0x7f2bf8005b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:59.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:05:59.884+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bf4016030 con 0x7f2bfc106f30 2024-09-19T05:06:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:59 smithi174 ceph-mon[34488]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:06:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:05:59 smithi174 ceph-mon[34488]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:06:00.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.098+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bfc02d7c0 con 0x7f2bfc106f30 2024-09-19T05:06:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.099+0000 7f2bf1ffb640 1 -- 172.21.15.64:0/2641899585 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f2bf409c8a0 con 0x7f2bfc106f30 2024-09-19T05:06:00.100 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:06:00.100 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:06:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:06:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2be40a3550 msgr2=0x7f2be40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2be40a3550 0x7f2be40a5a10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8031440 tx=0x7f2bf8005b50 comp rx=0 tx=0).stop 2024-09-19T05:06:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 msgr2=0x7f2bfc19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 0x7f2bfc19ccf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400eaa0 tx=0x7f2bf400eea0 comp rx=0 tx=0).stop 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 shutdown_connections 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2be40a3550 0x7f2be40a5a10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bfc106f30 0x7f2bfc19ccf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bfc100cc0 0x7f2bfc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 --2- 172.21.15.64:0/2641899585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bfc100310 0x7f2bfc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 >> 172.21.15.64:0/2641899585 conn(0x7f2bfc077210 msgr2=0x7f2bfc105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 shutdown_connections 2024-09-19T05:06:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:00.101+0000 7f2c02c6d640 1 -- 172.21.15.64:0/2641899585 wait complete. 2024-09-19T05:06:00.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:59 smithi103 ceph-mon[34446]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:06:00.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:05:59 smithi103 ceph-mon[34446]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:06:00.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:06:00.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:00 smithi064 ceph-mon[30960]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:06:00.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:00 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2641899585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:01.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:00 smithi174 ceph-mon[34488]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:06:01.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:00 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2641899585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:01.033 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:01.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:00 smithi103 ceph-mon[34446]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:06:01.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:00 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2641899585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:01.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 -- 172.21.15.64:0/1368070388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108106f10 msgr2=0x7fa10810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 --2- 172.21.15.64:0/1368070388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108106f10 0x7fa10810ea90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009ca0 tx=0x7fa0f802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 -- 172.21.15.64:0/1368070388 shutdown_connections 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 --2- 172.21.15.64:0/1368070388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108106f10 0x7fa10810ea90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 --2- 172.21.15.64:0/1368070388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108102790 0x7fa1081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 --2- 172.21.15.64:0/1368070388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa108101de0 0x7fa1081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 -- 172.21.15.64:0/1368070388 >> 172.21.15.64:0/1368070388 conn(0x7fa1080fd8c0 msgr2=0x7fa1080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 -- 172.21.15.64:0/1368070388 shutdown_connections 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.490+0000 7fa10d857640 1 -- 172.21.15.64:0/1368070388 wait complete. 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 Processor -- start 2024-09-19T05:06:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 -- start start 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 0x7fa1081982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa108102790 0x7fa1081987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108106f10 0x7fa10819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111e20 con 0x7fa108101de0 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111ca0 con 0x7fa108102790 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa10d857640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111fa0 con 0x7fa108106f10 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 0x7fa1081982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 0x7fa1081982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42580/0 (socket says 172.21.15.64:42580) 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa106ffd640 1 -- 172.21.15.64:0/93218307 learned_addr learned my addr 172.21.15.64:0/93218307 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.491+0000 7fa1067fc640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa108102790 0x7fa1081987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa1077fe640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108106f10 0x7fa10819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa106ffd640 1 -- 172.21.15.64:0/93218307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108106f10 msgr2=0x7fa10819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa106ffd640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108106f10 0x7fa10819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa106ffd640 1 -- 172.21.15.64:0/93218307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa108102790 msgr2=0x7fa1081987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa106ffd640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa108102790 0x7fa1081987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa106ffd640 1 -- 172.21.15.64:0/93218307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819d250 con 0x7fa108101de0 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa1077fe640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108106f10 0x7fa10819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa106ffd640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 0x7fa1081982a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c920 tx=0x7fa10000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100007c70 con 0x7fa108101de0 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa100007e10 con 0x7fa108101de0 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819d540 con 0x7fa108101de0 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100018820 con 0x7fa108101de0 2024-09-19T05:06:01.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.492+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a4e20 con 0x7fa108101de0 2024-09-19T05:06:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.493+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1081021c0 con 0x7fa108101de0 2024-09-19T05:06:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.494+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7fa1000189c0 con 0x7fa108101de0 2024-09-19T05:06:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.495+0000 7fa10c855640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa0f40a3550 0x7fa0f40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.495+0000 7fa1067fc640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa0f40a3550 0x7fa0f40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.495+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa1000d00a0 con 0x7fa108101de0 2024-09-19T05:06:01.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.496+0000 7fa1067fc640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa0f40a3550 0x7fa0f40a5a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1080fdce0 tx=0x7fa0f000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:01.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.499+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa100014030 con 0x7fa108101de0 2024-09-19T05:06:01.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.674+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa108109b60 con 0x7fa108101de0 2024-09-19T05:06:01.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.675+0000 7fa10c855640 1 -- 172.21.15.64:0/93218307 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+23198 (secure 0 0 0) 0x7fa10009c830 con 0x7fa108101de0 2024-09-19T05:06:01.677 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:06:01.677 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":80,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:05:53.680165+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:6800/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/868126904":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/2000236557":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/2343862077":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/1076683694":"2024-09-20T05:05:53.680120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa0f40a3550 msgr2=0x7fa0f40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa0f40a3550 0x7fa0f40a5a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1080fdce0 tx=0x7fa0f000b390 comp rx=0 tx=0).stop 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 msgr2=0x7fa1081982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 0x7fa1081982a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c920 tx=0x7fa10000cde0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 shutdown_connections 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa0f40a3550 0x7fa0f40a5a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa108106f10 0x7fa10819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa108102790 0x7fa1081987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 --2- 172.21.15.64:0/93218307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108101de0 0x7fa1081982a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.677+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 >> 172.21.15.64:0/93218307 conn(0x7fa1080fd8c0 msgr2=0x7fa1080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.678+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 shutdown_connections 2024-09-19T05:06:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:01.678+0000 7fa10d857640 1 -- 172.21.15.64:0/93218307 wait complete. 2024-09-19T05:06:01.988 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:01 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/93218307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:06:02.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:01 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/93218307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:06:02.120 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:01 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/93218307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:06:02.277 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T05:06:02.277 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T05:06:02.288 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:06:02.288 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:06:02.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T05:06:02.288 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T05:06:02.289 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:06:02.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:06:02.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 -- 172.21.15.64:0/2355719252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1069c0 msgr2=0x7f618c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 --2- 172.21.15.64:0/2355719252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1069c0 0x7f618c106da0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f617c009cd0 tx=0x7f617c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 -- 172.21.15.64:0/2355719252 shutdown_connections 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 --2- 172.21.15.64:0/2355719252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f618c101350 0x7f618c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 --2- 172.21.15.64:0/2355719252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1009b0 0x7f618c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 --2- 172.21.15.64:0/2355719252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1069c0 0x7f618c106da0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 -- 172.21.15.64:0/2355719252 >> 172.21.15.64:0/2355719252 conn(0x7f618c077990 msgr2=0x7f618c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 -- 172.21.15.64:0/2355719252 shutdown_connections 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.424+0000 7f6191df8640 1 -- 172.21.15.64:0/2355719252 wait complete. 2024-09-19T05:06:02.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 Processor -- start 2024-09-19T05:06:02.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 -- start start 2024-09-19T05:06:02.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 0x7f618c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f618c101350 0x7f618c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1069c0 0x7f618c1a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f618c110cb0 con 0x7f618c1009b0 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f618c110b30 con 0x7f618c1069c0 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f6191df8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f618c110e30 con 0x7f618c101350 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 0x7f618c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 0x7f618c1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42598/0 (socket says 172.21.15.64:42598) 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 -- 172.21.15.64:0/595563867 learned_addr learned my addr 172.21.15.64:0/595563867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618affd640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f618c101350 0x7f618c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618bfff640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1069c0 0x7f618c1a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 -- 172.21.15.64:0/595563867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f618c101350 msgr2=0x7f618c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f618c101350 0x7f618c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 -- 172.21.15.64:0/595563867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1069c0 msgr2=0x7f618c1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.425+0000 7f618b7fe640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1069c0 0x7f618c1a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f618b7fe640 1 -- 172.21.15.64:0/595563867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c1a4660 con 0x7f618c1009b0 2024-09-19T05:06:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f618bfff640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1069c0 0x7f618c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f618b7fe640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 0x7f618c1990a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f617c02ae00 tx=0x7f617c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c004240 con 0x7f618c1009b0 2024-09-19T05:06:02.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f617c0305a0 con 0x7f618c1009b0 2024-09-19T05:06:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c1a4830 con 0x7f618c1009b0 2024-09-19T05:06:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0466e0 con 0x7f618c1009b0 2024-09-19T05:06:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.426+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1a4cf0 con 0x7f618c1009b0 2024-09-19T05:06:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.427+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c100e10 con 0x7f618c1009b0 2024-09-19T05:06:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.428+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f617c030750 con 0x7f618c1009b0 2024-09-19T05:06:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.429+0000 7f6188ff9640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f61740a3580 0x7f61740a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.429+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f617c02e030 con 0x7f618c1009b0 2024-09-19T05:06:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.429+0000 7f618affd640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f61740a3580 0x7f61740a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.429+0000 7f618affd640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f61740a3580 0x7f61740a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f618c06a3f0 tx=0x7f618000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.433+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f617c04b060 con 0x7f618c1009b0 2024-09-19T05:06:02.579 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:02.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.618+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f618c108990 con 0x7f618c1009b0 2024-09-19T05:06:02.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.622+0000 7f6188ff9640 1 -- 172.21.15.64:0/595563867 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7f617c0bde30 con 0x7f618c1009b0 2024-09-19T05:06:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.624+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f61740a3580 msgr2=0x7f61740a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.624+0000 7f6191df8640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f61740a3580 0x7f61740a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f618c06a3f0 tx=0x7f618000b390 comp rx=0 tx=0).stop 2024-09-19T05:06:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.624+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 msgr2=0x7f618c1990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.624+0000 7f6191df8640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 0x7f618c1990a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f617c02ae00 tx=0x7f617c003ba0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 shutdown_connections 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f61740a3580 0x7f61740a5a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f618c1069c0 0x7f618c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f618c101350 0x7f618c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 --2- 172.21.15.64:0/595563867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f618c1009b0 0x7f618c1990a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 >> 172.21.15.64:0/595563867 conn(0x7f618c077990 msgr2=0x7f618c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 shutdown_connections 2024-09-19T05:06:02.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.625+0000 7f6191df8640 1 -- 172.21.15.64:0/595563867 wait complete. 2024-09-19T05:06:02.639 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 -- 172.21.15.64:0/2665535087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684100860 msgr2=0x7f9684100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 --2- 172.21.15.64:0/2665535087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684100860 0x7f9684100c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9680009900 tx=0x7f968002a410 comp rx=0 tx=0).stop 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 -- 172.21.15.64:0/2665535087 shutdown_connections 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 --2- 172.21.15.64:0/2665535087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 0x7f9684110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 --2- 172.21.15.64:0/2665535087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684101210 0x7f968410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 --2- 172.21.15.64:0/2665535087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684100860 0x7f9684100c40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 -- 172.21.15.64:0/2665535087 >> 172.21.15.64:0/2665535087 conn(0x7f9684077990 msgr2=0x7f96840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 -- 172.21.15.64:0/2665535087 shutdown_connections 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 -- 172.21.15.64:0/2665535087 wait complete. 2024-09-19T05:06:02.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.747+0000 7f968c3d8640 1 Processor -- start 2024-09-19T05:06:02.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 -- start start 2024-09-19T05:06:02.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684100860 0x7f968419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 0x7f968419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 0x7f968419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9684113f90 con 0x7f9684101210 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9684113e10 con 0x7f968410e410 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968c3d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9684114110 con 0x7f9684100860 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 0x7f968419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 0x7f968419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42616/0 (socket says 172.21.15.64:42616) 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968a174640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684100860 0x7f968419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 -- 172.21.15.64:0/4226215530 learned_addr learned my addr 172.21.15.64:0/4226215530 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968a975640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 0x7f968419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 -- 172.21.15.64:0/4226215530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684100860 msgr2=0x7f968419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684100860 0x7f968419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 -- 172.21.15.64:0/4226215530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 msgr2=0x7f968419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 0x7f968419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f9689973640 1 -- 172.21.15.64:0/4226215530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968419f440 con 0x7f9684101210 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968a975640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 0x7f968419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.748+0000 7f968a174640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684100860 0x7f968419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.749+0000 7f9689973640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 0x7f968419a990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f967400e9f0 tx=0x7f967400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.749+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967400cd80 con 0x7f9684101210 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.749+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96740054d0 con 0x7f9684101210 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.749+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968419f730 con 0x7f9684101210 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.749+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674010490 con 0x7f9684101210 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.749+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96841a6fc0 con 0x7f9684101210 2024-09-19T05:06:02.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.750+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9684100c40 con 0x7f9684101210 2024-09-19T05:06:02.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.751+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f9674005000 con 0x7f9684101210 2024-09-19T05:06:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.752+0000 7f967b7fe640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96700a3580 0x7f96700a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.752+0000 7f968a174640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96700a3580 0x7f96700a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.752+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f96740d03a0 con 0x7f9684101210 2024-09-19T05:06:02.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.753+0000 7f968a174640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96700a3580 0x7f96700a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9680005ab0 tx=0x7f9680005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:02.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.756+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96740052f0 con 0x7f9684101210 2024-09-19T05:06:02.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:02 smithi064 ceph-mon[30960]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 881 B/s wr, 22 op/s 2024-09-19T05:06:02.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:02 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/595563867' entity='client.admin' 2024-09-19T05:06:02.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:02 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:02.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:02 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:02.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:02 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:02.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:02 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:02.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.935+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f9684109600 con 0x7f9684101210 2024-09-19T05:06:02.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.938+0000 7f967b7fe640 1 -- 172.21.15.64:0/4226215530 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f967409cb30 con 0x7f9684101210 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96700a3580 msgr2=0x7f96700a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96700a3580 0x7f96700a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9680005ab0 tx=0x7f9680005a20 comp rx=0 tx=0).stop 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 msgr2=0x7f968419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 0x7f968419a990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f967400e9f0 tx=0x7f967400edf0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 shutdown_connections 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96700a3580 0x7f96700a5a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f968410e410 0x7f968419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9684101210 0x7f968419a990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 --2- 172.21.15.64:0/4226215530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9684100860 0x7f968419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:02.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.940+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 >> 172.21.15.64:0/4226215530 conn(0x7f9684077990 msgr2=0x7f96840ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:02.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.941+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 shutdown_connections 2024-09-19T05:06:02.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:02.941+0000 7f968c3d8640 1 -- 172.21.15.64:0/4226215530 wait complete. 2024-09-19T05:06:02.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T05:06:03.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:02 smithi174 ceph-mon[34488]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 881 B/s wr, 22 op/s 2024-09-19T05:06:03.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:02 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/595563867' entity='client.admin' 2024-09-19T05:06:03.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:02 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:03.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:02 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:03.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:02 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:03.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:02 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 -- 172.21.15.64:0/3984680640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 msgr2=0x7fba7410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 --2- 172.21.15.64:0/3984680640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7410e790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba64009ca0 tx=0x7fba6402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 -- 172.21.15.64:0/3984680640 shutdown_connections 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 --2- 172.21.15.64:0/3984680640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7410e790 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 --2- 172.21.15.64:0/3984680640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba741037a0 0x7fba7410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 --2- 172.21.15.64:0/3984680640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba74102df0 0x7fba741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.056+0000 7fba7b397640 1 -- 172.21.15.64:0/3984680640 >> 172.21.15.64:0/3984680640 conn(0x7fba740fc7d0 msgr2=0x7fba740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 -- 172.21.15.64:0/3984680640 shutdown_connections 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 -- 172.21.15.64:0/3984680640 wait complete. 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 Processor -- start 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 -- start start 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba74102df0 0x7fba741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba741037a0 0x7fba74198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.057+0000 7fba7b397640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fba74111fa0 con 0x7fba7410c2f0 2024-09-19T05:06:03.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba7b397640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba74111e20 con 0x7fba741037a0 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba7b397640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba74112120 con 0x7fba74102df0 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba79133640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba74102df0 0x7fba741984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba79934640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba741037a0 0x7fba74198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba79934640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42642/0 (socket says 172.21.15.64:42642) 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba79133640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba74102df0 0x7fba741984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:36864/0 (socket says 172.21.15.64:36864) 2024-09-19T05:06:03.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba79934640 1 -- 172.21.15.64:0/3330845572 learned_addr learned my addr 172.21.15.64:0/3330845572 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 -- 172.21.15.64:0/3330845572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba74102df0 msgr2=0x7fba741984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba74102df0 0x7fba741984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 -- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 msgr2=0x7fba7419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 -- 172.21.15.64:0/3330845572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7419d490 con 0x7fba741037a0 2024-09-19T05:06:03.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba79934640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba78932640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba741037a0 0x7fba74198a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba68005fd0 tx=0x7fba6800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba727fc640 1 -- 172.21.15.64:0/3330845572 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68009c00 con 0x7fba741037a0 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.058+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7419d780 con 0x7fba741037a0 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.059+0000 7fba727fc640 1 -- 172.21.15.64:0/3330845572 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba68009da0 con 0x7fba741037a0 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.059+0000 7fba727fc640 1 -- 172.21.15.64:0/3330845572 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68003db0 con 0x7fba741037a0 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.059+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba741a5060 con 0x7fba741037a0 2024-09-19T05:06:03.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.061+0000 7fba7d770740 1 Processor -- start 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.061+0000 7fba7d770740 1 -- start start 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.061+0000 7fba7d770740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e028f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.061+0000 7fba7d770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e033800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba7d770740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e036130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba7d770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56171df8e280 con 0x56171e029480 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba7d770740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56171df35b10 con 0x56171e028b60 2024-09-19T05:06:03.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba7d770740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56171df8a240 con 0x56171e033d40 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e028f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba727fc640 1 -- 172.21.15.64:0/3330845572 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7fba680035f0 con 0x7fba741037a0 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e028f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:46650/0 (socket says 172.21.15.64:46650) 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 -- 172.21.15.64:0/4169459680 learned_addr learned my addr 172.21.15.64:0/4169459680 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79133640 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e033800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba78932640 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e036130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 -- 172.21.15.64:0/4169459680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 msgr2=0x56171e036130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e036130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 -- 172.21.15.64:0/4169459680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 msgr2=0x56171e033800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e033800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 -- 172.21.15.64:0/4169459680 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171df421f0 con 0x56171e028b60 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79133640 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e033800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.062+0000 7fba79934640 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e028f40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba640098e0 tx=0x7fba64009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12fb99a23c93e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.063+0000 7fba5b7fe640 1 -- 172.21.15.64:0/4169459680 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64031cc0 con 0x56171e028b60 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.063+0000 7fba5b7fe640 1 -- 172.21.15.64:0/4169459680 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba64036030 con 0x56171e028b60 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.063+0000 7fba5b7fe640 1 -- 172.21.15.64:0/4169459680 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64040e60 con 0x56171e028b60 2024-09-19T05:06:03.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.063+0000 7fba727fc640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba5c0a3550 0x7fba5c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.063+0000 7fba79133640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba5c0a3550 0x7fba5c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.063+0000 7fba727fc640 1 -- 172.21.15.64:0/3330845572 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fba680cfa10 con 0x7fba741037a0 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 -- 172.21.15.64:0/4169459680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 msgr2=0x56171e028f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e028f40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba640098e0 tx=0x7fba64009910 comp rx=0 tx=0).stop 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 -- 172.21.15.64:0/4169459680 shutdown_connections 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e036130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e033800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 --2- 172.21.15.64:0/4169459680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e028f40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 -- 172.21.15.64:0/4169459680 >> 172.21.15.64:0/4169459680 conn(0x56171e0250f0 msgr2=0x56171e0254d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba79133640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba5c0a3550 0x7fba5c0a5a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56171e025f50 tx=0x7fba6c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 -- 172.21.15.64:0/4169459680 shutdown_connections 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.064+0000 7fba7d770740 1 -- 172.21.15.64:0/4169459680 wait complete. 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 -- start start 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 Processor -- start 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e030bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e029480 0x56171e031100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 0x56171e031640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56171e036790 con 0x56171e028b60 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56171df48f10 con 0x56171e033d40 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56171e036910 con 0x56171e029480 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba7d770740 10 client.0 _ll_register_callbacks cb 0x56171df11a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba79934640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 0x56171e031640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba79934640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 0x56171e031640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:46660/0 (socket says 172.21.15.64:46660) 2024-09-19T05:06:03.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba79934640 1 -- 172.21.15.64:0/1280679551 learned_addr learned my addr 172.21.15.64:0/1280679551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba78932640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e029480 0x56171e031100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.065+0000 7fba79133640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e030bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79934640 1 -- 172.21.15.64:0/1280679551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e029480 msgr2=0x56171e031100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79934640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e029480 0x56171e031100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79934640 1 -- 172.21.15.64:0/1280679551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 msgr2=0x56171e030bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79934640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e030bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79934640 1 -- 172.21.15.64:0/1280679551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171dcff6a0 con 0x56171e033d40 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79133640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e030bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba79934640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 0x56171e031640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fba64005e30 tx=0x7fba64003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf49eaa99e75a48c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:06:03.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba58ff9640 1 -- 172.21.15.64:0/1280679551 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6404e040 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba64036030 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba58ff9640 1 -- 172.21.15.64:0/1280679551 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fba64004240 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba58ff9640 1 -- 172.21.15.64:0/1280679551 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba64062050 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba7d770740 10 client.25016 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.066+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56171e036790 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba58ff9640 1 -- 172.21.15.64:0/1280679551 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba64036030 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba58ff9640 1 -- 172.21.15.64:0/1280679551 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fba64051030 con 0x56171e033d40 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba7d770740 10 client.25016 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba7d770740 10 client.25016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba7d770740 20 client.25016 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:03.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba7d770740 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0435d0 0x56171e0459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba7d770740 4 client.25016 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:03.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56171df9b7f0 con 0x56171e0435d0 2024-09-19T05:06:03.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.067+0000 7fba79133640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0435d0 0x56171e0459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 -- 172.21.15.64:0/2793712818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f84108d30 msgr2=0x7f0f8410cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/2793712818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f84108d30 0x7f0f8410cd70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009860 tx=0x7f0f7402a340 comp rx=0 tx=0).stop 2024-09-19T05:06:03.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7fba58ff9640 10 client.25016 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:03.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 -- 172.21.15.64:0/2793712818 shutdown_connections 2024-09-19T05:06:03.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/2793712818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f8410d4d0 0x7f0f84115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/2793712818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f84108d30 0x7f0f8410cd70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/2793712818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f84108380 0x7f0f84108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7fba79133640 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0435d0 0x56171e0459c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1676eb3fd8c21932 server_cookie=d7e4561e4015ee42 in_seq=0 out_seq=0 2024-09-19T05:06:03.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 -- 172.21.15.64:0/2793712818 >> 172.21.15.64:0/2793712818 conn(0x7f0f84077b20 msgr2=0x7f0f84079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 -- 172.21.15.64:0/2793712818 shutdown_connections 2024-09-19T05:06:03.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 -- 172.21.15.64:0/2793712818 wait complete. 2024-09-19T05:06:03.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7fba58ff9640 1 -- 172.21.15.64:0/1280679551 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56171df9b7f0 con 0x56171e0435d0 2024-09-19T05:06:03.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7fba58ff9640 10 client.25016 handle_command_reply: tid=0 2024-09-19T05:06:03.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7fba7d770740 1 client.25016 shutdown 2024-09-19T05:06:03.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.068+0000 7f0f8bc27640 1 Processor -- start 2024-09-19T05:06:03.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 -- start start 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 msgr2=0x56171e031640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 0x56171e031640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fba64005e30 tx=0x7fba64003c20 comp rx=0 tx=0).stop 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 shutdown_connections 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e033d40 0x56171e031640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e029480 0x56171e031100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0435d0 0x56171e0459c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 --2- 172.21.15.64:0/1280679551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e030bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 0x7f0f841a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f8410d4d0 0x7f0f841a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 >> 172.21.15.64:0/1280679551 conn(0x56171e0250f0 msgr2=0x56171e024270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba58ff9640 0 client.25016 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f8419e780 0x7f0f8419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841183f0 con 0x7f0f84108380 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84118270 con 0x7f0f8410d4d0 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8bc27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84118570 con 0x7f0f8419e780 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f899c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 0x7f0f841a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f899c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 0x7f0f841a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42686/0 (socket says 172.21.15.64:42686) 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f899c3640 1 -- 172.21.15.64:0/3551134107 learned_addr learned my addr 172.21.15.64:0/3551134107 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8a1c4640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f8419e780 0x7f0f8419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f891c2640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f8410d4d0 0x7f0f841a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 shutdown_connections 2024-09-19T05:06:03.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 1 -- 172.21.15.64:0/1280679551 wait complete. 2024-09-19T05:06:03.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7fba7d770740 20 client.25016 trim_cache size 0 max 16384 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8a1c4640 1 -- 172.21.15.64:0/3551134107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f8410d4d0 msgr2=0x7f0f841a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8a1c4640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f8410d4d0 0x7f0f841a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8a1c4640 1 -- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 msgr2=0x7f0f841a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8a1c4640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 0x7f0f841a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f8a1c4640 1 -- 172.21.15.64:0/3551134107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8419f310 con 0x7f0f8419e780 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f899c3640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 0x7f0f841a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.069+0000 7f0f891c2640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f8410d4d0 0x7f0f841a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.070+0000 7f0f8a1c4640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f8419e780 0x7f0f8419ec30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00ef00 tx=0x7f0f7c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.070+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c003970 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.070+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8419f600 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.070+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f7c004020 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.070+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00b860 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.070+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841ab6a0 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.071+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84109850 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.072+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f0f7c01d020 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.073+0000 7f0f7affd640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0f700a35d0 0x7f0f700a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.073+0000 7f0f899c3640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0f700a35d0 0x7f0f700a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.073+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0f7c0cfd40 con 0x7f0f8419e780 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.073+0000 7f0f899c3640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0f700a35d0 0x7f0f700a5a90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f80011500 tx=0x7f0f80005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.077+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0f8419f600 con 0x7f0f8419e780 2024-09-19T05:06:03.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 Processor -- start 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 -- start start 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e04b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e04b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e023a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fba741addb0 con 0x56171e029480 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba741ab680 con 0x56171e028b60 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba7d770740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba741a94d0 con 0x56171e033d40 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba78932640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e04b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba79133640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e04b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba78932640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e04b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42692/0 (socket says 172.21.15.64:42692) 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba78932640 1 -- 172.21.15.64:0/3066318147 learned_addr learned my addr 172.21.15.64:0/3066318147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.099+0000 7fba79934640 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e023a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba78932640 1 -- 172.21.15.64:0/3066318147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 msgr2=0x56171e023a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba78932640 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e023a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba78932640 1 -- 172.21.15.64:0/3066318147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 msgr2=0x56171e04b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba78932640 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e04b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba78932640 1 -- 172.21.15.64:0/3066318147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171e0292b0 con 0x56171e029480 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba79934640 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e023a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba79133640 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e04b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba78932640 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e04b7d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fba6809ba80 tx=0x7fba680045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d46c4984d0f8f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba5affd640 1 -- 172.21.15.64:0/3066318147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68020040 con 0x56171e029480 2024-09-19T05:06:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba5affd640 1 -- 172.21.15.64:0/3066318147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba6809fdf0 con 0x56171e029480 2024-09-19T05:06:03.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.100+0000 7fba5affd640 1 -- 172.21.15.64:0/3066318147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680cd260 con 0x56171e029480 2024-09-19T05:06:03.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 -- 172.21.15.64:0/3066318147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 msgr2=0x56171e04b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e04b7d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fba6809ba80 tx=0x7fba680045e0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 -- 172.21.15.64:0/3066318147 shutdown_connections 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e023a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e029480 0x56171e04b7d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 --2- 172.21.15.64:0/3066318147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e028b60 0x56171e04b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 -- 172.21.15.64:0/3066318147 >> 172.21.15.64:0/3066318147 conn(0x56171e0250f0 msgr2=0x56171e032b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 -- 172.21.15.64:0/3066318147 shutdown_connections 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 -- 172.21.15.64:0/3066318147 wait complete. 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.101+0000 7fba7d770740 1 -- start start 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 Processor -- start 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e02dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e029480 0x56171e02e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e02e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fba741a9b40 con 0x56171e028b60 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba741aafb0 con 0x56171e029480 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba741aa210 con 0x56171e033d40 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba7d770740 10 client.0 _ll_register_callbacks cb 0x7fba64009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba79133640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e02dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba79934640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e02e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba79133640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e02dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42694/0 (socket says 172.21.15.64:42694) 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba78932640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e029480 0x56171e02e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.102+0000 7fba79133640 1 -- 172.21.15.64:0/2468739251 learned_addr learned my addr 172.21.15.64:0/2468739251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79133640 1 -- 172.21.15.64:0/2468739251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 msgr2=0x56171e02e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79133640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e02e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79133640 1 -- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e029480 msgr2=0x56171e02e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79133640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e029480 0x56171e02e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79133640 1 -- 172.21.15.64:0/2468739251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171e0ad3e0 con 0x56171e028b60 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79934640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e02e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba78932640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e029480 0x56171e02e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba79133640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e02dd60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x56171e046230 tx=0x7fba6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becc7d794b544286 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba58ff9640 1 -- 172.21.15.64:0/2468739251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c01dcc0 con 0x56171e028b60 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba58ff9640 1 -- 172.21.15.64:0/2468739251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fba6c02a390 con 0x56171e028b60 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6809fdf0 con 0x56171e028b60 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba58ff9640 1 -- 172.21.15.64:0/2468739251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba6c01de60 con 0x56171e028b60 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba7d770740 10 client.15291 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.103+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56171e0ae780 con 0x56171e028b60 2024-09-19T05:06:03.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba58ff9640 1 -- 172.21.15.64:0/2468739251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba6809fdf0 con 0x56171e028b60 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba58ff9640 1 -- 172.21.15.64:0/2468739251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fba6c032840 con 0x56171e028b60 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba7d770740 10 client.15291 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba7d770740 10 client.15291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba7d770740 20 client.15291 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba7d770740 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0aef00 0x56171e0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba7d770740 4 client.15291 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56171df95d00 con 0x56171e0aef00 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.104+0000 7fba78932640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0aef00 0x56171e0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.105+0000 7fba78932640 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0aef00 0x56171e0b1320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5237369f84eac389 server_cookie=70ee6aac258295cd in_seq=0 out_seq=0 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.105+0000 7fba58ff9640 10 client.15291 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.105+0000 7fba58ff9640 1 -- 172.21.15.64:0/2468739251 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56171df95d00 con 0x56171e0aef00 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.105+0000 7fba58ff9640 10 client.15291 handle_command_reply: tid=0 2024-09-19T05:06:03.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 client.15291 shutdown 2024-09-19T05:06:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 msgr2=0x56171e02dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e02dd60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x56171e046230 tx=0x7fba6c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 shutdown_connections 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56171e033d40 0x56171e02e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56171e029480 0x56171e02e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56171e0aef00 0x56171e0b1320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 --2- 172.21.15.64:0/2468739251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56171e028b60 0x56171e02dd60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba58ff9640 0 client.15291 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 >> 172.21.15.64:0/2468739251 conn(0x56171e0250f0 msgr2=0x56171e0322e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 shutdown_connections 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 1 -- 172.21.15.64:0/2468739251 wait complete. 2024-09-19T05:06:03.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.106+0000 7fba7d770740 20 client.15291 trim_cache size 0 max 16384 2024-09-19T05:06:03.113 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:06:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:06:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "95d34c67-3db3-4db2-be7b-39e919d9b173", 2024-09-19T05:06:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:06:03.113 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba5c0a3550 msgr2=0x7fba5c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba5c0a3550 0x7fba5c0a5a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56171e025f50 tx=0x7fba6c005f90 comp rx=0 tx=0).stop 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba741037a0 msgr2=0x7fba74198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba741037a0 0x7fba74198a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba68005fd0 tx=0x7fba6800dde0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 shutdown_connections 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba5c0a3550 0x7fba5c0a5a10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba7410c2f0 0x7fba7419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba741037a0 0x7fba74198a20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 --2- 172.21.15.64:0/3330845572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba74102df0 0x7fba741984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 >> 172.21.15.64:0/3330845572 conn(0x7fba740fc7d0 msgr2=0x7fba740fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 shutdown_connections 2024-09-19T05:06:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.107+0000 7fba7b397640 1 -- 172.21.15.64:0/3330845572 wait complete. 2024-09-19T05:06:03.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:02 smithi103 ceph-mon[34446]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 881 B/s wr, 22 op/s 2024-09-19T05:06:03.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:02 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/595563867' entity='client.admin' 2024-09-19T05:06:03.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:02 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:03.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:02 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:03.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:02 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:03.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:02 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.279+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f0f84108d30 con 0x7f0f8419e780 2024-09-19T05:06:03.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.284+0000 7f0f7affd640 1 -- 172.21.15.64:0/3551134107 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f0f7c09c4d0 con 0x7f0f8419e780 2024-09-19T05:06:03.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0f700a35d0 msgr2=0x7f0f700a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0f700a35d0 0x7f0f700a5a90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f80011500 tx=0x7f0f80005f90 comp rx=0 tx=0).stop 2024-09-19T05:06:03.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f8419e780 msgr2=0x7f0f8419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f8419e780 0x7f0f8419ec30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00ef00 tx=0x7f0f7c00c740 comp rx=0 tx=0).stop 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 shutdown_connections 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0f700a35d0 0x7f0f700a5a90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f8419e780 0x7f0f8419ec30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f8410d4d0 0x7f0f841a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 --2- 172.21.15.64:0/3551134107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f84108380 0x7f0f841a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 >> 172.21.15.64:0/3551134107 conn(0x7f0f84077b20 msgr2=0x7f0f84078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 shutdown_connections 2024-09-19T05:06:03.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.287+0000 7f0f8bc27640 1 -- 172.21.15.64:0/3551134107 wait complete. 2024-09-19T05:06:03.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T05:06:03.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 -- 172.21.15.64:0/2307198958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 msgr2=0x7fdc6010ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 --2- 172.21.15.64:0/2307198958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc6010ba80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402a480 comp rx=0 tx=0).stop 2024-09-19T05:06:03.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 -- 172.21.15.64:0/2307198958 shutdown_connections 2024-09-19T05:06:03.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 --2- 172.21.15.64:0/2307198958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc6010c0d0 0x7fdc6010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 --2- 172.21.15.64:0/2307198958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc6010ba80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 --2- 172.21.15.64:0/2307198958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc60102c80 0x7fdc60103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 -- 172.21.15.64:0/2307198958 >> 172.21.15.64:0/2307198958 conn(0x7fdc60077990 msgr2=0x7fdc600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 -- 172.21.15.64:0/2307198958 shutdown_connections 2024-09-19T05:06:03.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 -- 172.21.15.64:0/2307198958 wait complete. 2024-09-19T05:06:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 Processor -- start 2024-09-19T05:06:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.408+0000 7fdc64981640 1 -- start start 2024-09-19T05:06:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc60102c80 0x7fdc60198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc601987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc6010c0d0 0x7fdc6019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111db0 con 0x7fdc601035a0 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111c30 con 0x7fdc6010c0d0 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111f30 con 0x7fdc60102c80 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc601987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc601987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42714/0 (socket says 172.21.15.64:42714) 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 -- 172.21.15.64:0/3733180600 learned_addr learned my addr 172.21.15.64:0/3733180600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5e59c640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc60102c80 0x7fdc60198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5ed9d640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc6010c0d0 0x7fdc6019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 -- 172.21.15.64:0/3733180600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc60102c80 msgr2=0x7fdc60198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc60102c80 0x7fdc60198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 -- 172.21.15.64:0/3733180600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc6010c0d0 msgr2=0x7fdc6019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc6010c0d0 0x7fdc6019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 -- 172.21.15.64:0/3733180600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6019d270 con 0x7fdc601035a0 2024-09-19T05:06:03.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc5dd9b640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc601987c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005ab0 tx=0x7fdc54003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5403f070 con 0x7fdc601035a0 2024-09-19T05:06:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc54031b50 con 0x7fdc601035a0 2024-09-19T05:06:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.409+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6019d500 con 0x7fdc601035a0 2024-09-19T05:06:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.410+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54035030 con 0x7fdc601035a0 2024-09-19T05:06:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.410+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a4df0 con 0x7fdc601035a0 2024-09-19T05:06:03.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.411+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc60103060 con 0x7fdc601035a0 2024-09-19T05:06:03.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.412+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7fdc54042070 con 0x7fdc601035a0 2024-09-19T05:06:03.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.412+0000 7fdc4f7fe640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdc480a3600 0x7fdc480a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.412+0000 7fdc5e59c640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdc480a3600 0x7fdc480a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.412+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdc540f0430 con 0x7fdc601035a0 2024-09-19T05:06:03.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.413+0000 7fdc5e59c640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdc480a3600 0x7fdc480a5ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009bd0 tx=0x7fdc50005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.416+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc540bc640 con 0x7fdc601035a0 2024-09-19T05:06:03.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.619+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdc60062630 con 0x7fdc480a3600 2024-09-19T05:06:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.680+0000 7fdc4f7fe640 1 -- 172.21.15.64:0/3733180600 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc60062630 con 0x7fdc480a3600 2024-09-19T05:06:03.683 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path / 2024-09-19T05:06:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdc480a3600 msgr2=0x7fdc480a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdc480a3600 0x7fdc480a5ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009bd0 tx=0x7fdc50005f90 comp rx=0 tx=0).stop 2024-09-19T05:06:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 msgr2=0x7fdc601987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc601987c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005ab0 tx=0x7fdc54003b60 comp rx=0 tx=0).stop 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 shutdown_connections 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdc480a3600 0x7fdc480a5ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc6010c0d0 0x7fdc6019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc601035a0 0x7fdc601987c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 --2- 172.21.15.64:0/3733180600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc60102c80 0x7fdc60198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 >> 172.21.15.64:0/3733180600 conn(0x7fdc60077990 msgr2=0x7fdc600fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 shutdown_connections 2024-09-19T05:06:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.683+0000 7fdc64981640 1 -- 172.21.15.64:0/3733180600 wait complete. 2024-09-19T05:06:03.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 -- 172.21.15.64:0/600337402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd22010c110 msgr2=0x7fd22010e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 --2- 172.21.15.64:0/600337402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd22010c110 0x7fd22010e590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd218009ca0 tx=0x7fd21802a950 comp rx=0 tx=0).stop 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 -- 172.21.15.64:0/600337402 shutdown_connections 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 --2- 172.21.15.64:0/600337402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd22010c110 0x7fd22010e590 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 --2- 172.21.15.64:0/600337402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2201035e0 0x7fd22010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 --2- 172.21.15.64:0/600337402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd220102cc0 0x7fd2201030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 -- 172.21.15.64:0/600337402 >> 172.21.15.64:0/600337402 conn(0x7fd2200fc6b0 msgr2=0x7fd2200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:03.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 -- 172.21.15.64:0/600337402 shutdown_connections 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.801+0000 7fd227f8d640 1 -- 172.21.15.64:0/600337402 wait complete. 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 Processor -- start 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 -- start start 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd220102cc0 0x7fd2201981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 0x7fd220198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22010c110 0x7fd22019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd220111df0 con 0x7fd2201035e0 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd220111c70 con 0x7fd220102cc0 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd227f8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd220111f70 con 0x7fd22010c110 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd225528640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 0x7fd220198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd225528640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 0x7fd220198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42734/0 (socket says 172.21.15.64:42734) 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd225528640 1 -- 172.21.15.64:0/3393962974 learned_addr learned my addr 172.21.15.64:0/3393962974 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:03.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.802+0000 7fd225d29640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd220102cc0 0x7fd2201981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd22652a640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22010c110 0x7fd22019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd225528640 1 -- 172.21.15.64:0/3393962974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22010c110 msgr2=0x7fd22019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd225528640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22010c110 0x7fd22019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd225528640 1 -- 172.21.15.64:0/3393962974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd220102cc0 msgr2=0x7fd2201981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd225528640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd220102cc0 0x7fd2201981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd225528640 1 -- 172.21.15.64:0/3393962974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22019d1b0 con 0x7fd2201035e0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd225528640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 0x7fd220198700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c970 tx=0x7fd21000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210007bb0 con 0x7fd2201035e0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd210007d50 con 0x7fd2201035e0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22019d4a0 con 0x7fd2201035e0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210018780 con 0x7fd2201035e0 2024-09-19T05:06:03.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.803+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2201a4d30 con 0x7fd2201035e0 2024-09-19T05:06:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.804+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2201030a0 con 0x7fd2201035e0 2024-09-19T05:06:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.806+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7fd21000f450 con 0x7fd2201035e0 2024-09-19T05:06:03.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.806+0000 7fd216ffd640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd20c0a3580 0x7fd20c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:03.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.806+0000 7fd225d29640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd20c0a3580 0x7fd20c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:03.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.806+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fd2100d02c0 con 0x7fd2201035e0 2024-09-19T05:06:03.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.807+0000 7fd225d29640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd20c0a3580 0x7fd20c0a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0058c0 tx=0x7fd21c00a250 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:03.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:03.809+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd210014030 con 0x7fd2201035e0 2024-09-19T05:06:03.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:06:04.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.013+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd2200fe9d0 con 0x7fd20c0a3580 2024-09-19T05:06:04.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.075+0000 7fd216ffd640 1 -- 172.21.15.64:0/3393962974 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd2200fe9d0 con 0x7fd20c0a3580 2024-09-19T05:06:04.077 INFO:teuthology.orchestra.run.smithi064.stdout:Retention added to path / 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd20c0a3580 msgr2=0x7fd20c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd20c0a3580 0x7fd20c0a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0058c0 tx=0x7fd21c00a250 comp rx=0 tx=0).stop 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 msgr2=0x7fd220198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 0x7fd220198700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c970 tx=0x7fd21000ce30 comp rx=0 tx=0).stop 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 shutdown_connections 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd20c0a3580 0x7fd20c0a5a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22010c110 0x7fd22019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2201035e0 0x7fd220198700 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 --2- 172.21.15.64:0/3393962974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd220102cc0 0x7fd2201981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 >> 172.21.15.64:0/3393962974 conn(0x7fd2200fc6b0 msgr2=0x7fd2200fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 shutdown_connections 2024-09-19T05:06:04.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.077+0000 7fd227f8d640 1 -- 172.21.15.64:0/3393962974 wait complete. 2024-09-19T05:06:04.090 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4226215530' entity='client.admin' 2024-09-19T05:06:04.090 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:04.090 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:04.090 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='client.? ' entity='client.admin' 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:04.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:03 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:04.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T05:06:04.109 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:04.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.201+0000 7f20a3e27640 1 -- 172.21.15.64:0/412747258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 msgr2=0x7f209c10e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.201+0000 7f20a3e27640 1 --2- 172.21.15.64:0/412747258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c10e550 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2094009ca0 tx=0x7f209402a950 comp rx=0 tx=0).stop 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- 172.21.15.64:0/412747258 shutdown_connections 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 --2- 172.21.15.64:0/412747258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c10e550 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 --2- 172.21.15.64:0/412747258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209c1035a0 0x7f209c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 --2- 172.21.15.64:0/412747258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209c102c80 0x7f209c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- 172.21.15.64:0/412747258 >> 172.21.15.64:0/412747258 conn(0x7f209c077990 msgr2=0x7f209c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- 172.21.15.64:0/412747258 shutdown_connections 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- 172.21.15.64:0/412747258 wait complete. 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 Processor -- start 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- start start 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209c102c80 0x7f209c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209c1035a0 0x7f209c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f209c111d20 con 0x7f209c10c0d0 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f209c111ba0 con 0x7f209c102c80 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.202+0000 7f20a3e27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f209c111ea0 con 0x7f209c1035a0 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42754/0 (socket says 172.21.15.64:42754) 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 -- 172.21.15.64:0/1005309629 learned_addr learned my addr 172.21.15.64:0/1005309629 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a1bc3640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209c102c80 0x7f209c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a13c2640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209c1035a0 0x7f209c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 -- 172.21.15.64:0/1005309629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209c1035a0 msgr2=0x7f209c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209c1035a0 0x7f209c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 -- 172.21.15.64:0/1005309629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209c102c80 msgr2=0x7f209c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209c102c80 0x7f209c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 -- 172.21.15.64:0/1005309629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c19d240 con 0x7f209c10c0d0 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a23c4640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c19cb10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2094004af0 tx=0x7f209402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094034040 con 0x7f209c10c0d0 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209402fe80 con 0x7f209c10c0d0 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c19d4d0 con 0x7f209c10c0d0 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20940046a0 con 0x7f209c10c0d0 2024-09-19T05:06:04.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.203+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c1a4dc0 con 0x7f209c10c0d0 2024-09-19T05:06:04.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.204+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c103060 con 0x7f209c10c0d0 2024-09-19T05:06:04.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.206+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f2094037020 con 0x7f209c10c0d0 2024-09-19T05:06:04.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.206+0000 7f2092ffd640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f20880a3580 0x7f20880a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.207+0000 7f20a1bc3640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f20880a3580 0x7f20880a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.207+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f20940f0de0 con 0x7f209c10c0d0 2024-09-19T05:06:04.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.207+0000 7f20a1bc3640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f20880a3580 0x7f20880a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2098009bd0 tx=0x7f2098005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.210+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20940bd680 con 0x7f209c10c0d0 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4226215530' entity='client.admin' 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='client.? ' entity='client.admin' 2024-09-19T05:06:04.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:04.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:04.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:04.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:03 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4226215530' entity='client.admin' 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='client.? ' entity='client.admin' 2024-09-19T05:06:04.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:04.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:04.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:04.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:03 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:06:04.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.409+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f209c107190 con 0x7f20880a3580 2024-09-19T05:06:04.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.417+0000 7f2092ffd640 1 -- 172.21.15.64:0/1005309629 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f209801b3f0 con 0x7f20880a3580 2024-09-19T05:06:04.419 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T05:06:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T05:06:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f20880a3580 msgr2=0x7f20880a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f20880a3580 0x7f20880a5a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2098009bd0 tx=0x7f2098005f90 comp rx=0 tx=0).stop 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 msgr2=0x7f209c19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c19cb10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2094004af0 tx=0x7f209402f5d0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 shutdown_connections 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f20880a3580 0x7f20880a5a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f209c10c0d0 0x7f209c19cb10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209c1035a0 0x7f209c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 --2- 172.21.15.64:0/1005309629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209c102c80 0x7f209c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 >> 172.21.15.64:0/1005309629 conn(0x7f209c077990 msgr2=0x7f209c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 shutdown_connections 2024-09-19T05:06:04.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.419+0000 7f20a3e27640 1 -- 172.21.15.64:0/1005309629 wait complete. 2024-09-19T05:06:04.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T05:06:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 -- 172.21.15.64:0/3575983849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1082a0 msgr2=0x7fe68c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 --2- 172.21.15.64:0/3575983849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1082a0 0x7fe68c10ec40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe684009cd0 tx=0x7fe68402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 -- 172.21.15.64:0/3575983849 shutdown_connections 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 --2- 172.21.15.64:0/3575983849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1082a0 0x7fe68c10ec40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 --2- 172.21.15.64:0/3575983849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe68c1036e0 0x7fe68c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 --2- 172.21.15.64:0/3575983849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe68c102d30 0x7fe68c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.534+0000 7fe693e16640 1 -- 172.21.15.64:0/3575983849 >> 172.21.15.64:0/3575983849 conn(0x7fe68c0fc7d0 msgr2=0x7fe68c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.535+0000 7fe693e16640 1 -- 172.21.15.64:0/3575983849 shutdown_connections 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.535+0000 7fe693e16640 1 -- 172.21.15.64:0/3575983849 wait complete. 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.535+0000 7fe693e16640 1 Processor -- start 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.535+0000 7fe693e16640 1 -- start start 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.535+0000 7fe693e16640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe68c102d30 0x7fe68c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe693e16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 0x7fe68c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe693e16640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe68c1082a0 0x7fe68c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe693e16640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c111f50 con 0x7fe68c1036e0 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe693e16640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c111dd0 con 0x7fe68c102d30 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe693e16640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1120d0 con 0x7fe68c1082a0 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 0x7fe68c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 0x7fe68c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44048/0 (socket says 172.21.15.64:44048) 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 -- 172.21.15.64:0/4013773301 learned_addr learned my addr 172.21.15.64:0/4013773301 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6923b3640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe68c1082a0 0x7fe68c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe691bb2640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe68c102d30 0x7fe68c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 -- 172.21.15.64:0/4013773301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe68c1082a0 msgr2=0x7fe68c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe68c1082a0 0x7fe68c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 -- 172.21.15.64:0/4013773301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe68c102d30 msgr2=0x7fe68c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe68c102d30 0x7fe68c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 -- 172.21.15.64:0/4013773301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c19d450 con 0x7fe68c1036e0 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe6913b1640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 0x7fe68c1989e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe68800e9c0 tx=0x7fe68800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe682ffd640 1 -- 172.21.15.64:0/4013773301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68800ccc0 con 0x7fe68c1036e0 2024-09-19T05:06:04.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.536+0000 7fe682ffd640 1 -- 172.21.15.64:0/4013773301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6880054d0 con 0x7fe68c1036e0 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.537+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c19d740 con 0x7fe68c1036e0 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.537+0000 7fe682ffd640 1 -- 172.21.15.64:0/4013773301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688010430 con 0x7fe68c1036e0 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.537+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a5020 con 0x7fe68c1036e0 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.539+0000 7fe682ffd640 1 -- 172.21.15.64:0/4013773301 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7fe6880105d0 con 0x7fe68c1036e0 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.539+0000 7fe6961ef740 1 Processor -- start 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.539+0000 7fe6961ef740 1 -- start start 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c21e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c229140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6961ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55997c1842e0 con 0x55997c21e4a0 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6961ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55997c12bce0 con 0x55997c21edc0 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6961ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55997c1802a0 con 0x55997c229680 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c21e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c21e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44054/0 (socket says 172.21.15.64:44054) 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 -- 172.21.15.64:0/1380176735 learned_addr learned my addr 172.21.15.64:0/1380176735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe682ffd640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe6780a3550 0x7fe6780a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe691bb2640 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c229140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe6780a3550 0x7fe6780a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe682ffd640 1 -- 172.21.15.64:0/4013773301 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe6880d0460 con 0x7fe68c1036e0 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 -- 172.21.15.64:0/1380176735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 msgr2=0x55997c22ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 -- 172.21.15.64:0/1380176735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 msgr2=0x55997c229140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c229140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 -- 172.21.15.64:0/1380176735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55997c110440 con 0x55997c21e4a0 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.540+0000 7fe6923b3640 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c21e880 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe684004af0 tx=0x7fe684031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758781c5d0959609 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe677fff640 1 -- 172.21.15.64:0/1380176735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684031b40 con 0x55997c21e4a0 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe677fff640 1 -- 172.21.15.64:0/1380176735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe684036040 con 0x55997c21e4a0 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe677fff640 1 -- 172.21.15.64:0/1380176735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684048c30 con 0x55997c21e4a0 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe6923b3640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe6780a3550 0x7fe6780a5a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe684053000 tx=0x7fe68404f000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe6961ef740 1 -- 172.21.15.64:0/1380176735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 msgr2=0x55997c21e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe6961ef740 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c21e880 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe684004af0 tx=0x7fe684031950 comp rx=0 tx=0).stop 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.541+0000 7fe6961ef740 1 -- 172.21.15.64:0/1380176735 shutdown_connections 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c229140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 --2- 172.21.15.64:0/1380176735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c21e880 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 -- 172.21.15.64:0/1380176735 >> 172.21.15.64:0/1380176735 conn(0x55997c21aa30 msgr2=0x55997c21ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 -- 172.21.15.64:0/1380176735 shutdown_connections 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 -- 172.21.15.64:0/1380176735 wait complete. 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 -- start start 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.542+0000 7fe6961ef740 1 Processor -- start 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21e4a0 0x55997c229060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 0x55997c226310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c2268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55997c22c250 con 0x55997c21edc0 2024-09-19T05:06:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55997c13f050 con 0x55997c21e4a0 2024-09-19T05:06:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55997c22c3d0 con 0x55997c229680 2024-09-19T05:06:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6961ef740 10 client.0 _ll_register_callbacks cb 0x55997c17f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 0x55997c226310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 0x55997c226310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44056/0 (socket says 172.21.15.64:44056) 2024-09-19T05:06:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 -- 172.21.15.64:0/433113449 learned_addr learned my addr 172.21.15.64:0/433113449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6923b3640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c2268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe691bb2640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21e4a0 0x55997c229060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 -- 172.21.15.64:0/433113449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 msgr2=0x55997c2268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c2268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 -- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21e4a0 msgr2=0x55997c229060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21e4a0 0x55997c229060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 -- 172.21.15.64:0/433113449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55997bef53c0 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6913b1640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 0x55997c226310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe6880ce1b0 tx=0x7fe688004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7ace1266f6e004 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.543+0000 7fe6757fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6757fa640 1 -- 172.21.15.64:0/433113449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880b5060 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6757fa640 1 -- 172.21.15.64:0/433113449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe6880b5200 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684036040 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6757fa640 1 -- 172.21.15.64:0/433113449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe688005680 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 10 client.15321 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55997c237cf0 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6757fa640 1 -- 172.21.15.64:0/433113449 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe684036040 con 0x55997c21edc0 2024-09-19T05:06:04.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6757fa640 1 -- 172.21.15.64:0/433113449 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe68809b940 con 0x55997c21edc0 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 10 client.15321 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 20 client.15321 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c239100 0x55997c23b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 4 client.15321 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55997c191850 con 0x55997c239100 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.544+0000 7fe691bb2640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c239100 0x55997c23b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.545+0000 7fe691bb2640 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c239100 0x55997c23b4f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ab5c64bbd1d72f server_cookie=88e5eff23c4289eb in_seq=0 out_seq=0 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.545+0000 7fe6757fa640 10 client.15321 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6757fa640 1 -- 172.21.15.64:0/433113449 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55997c191850 con 0x55997c239100 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6757fa640 10 client.15321 handle_command_reply: tid=0 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 client.15321 shutdown 2024-09-19T05:06:04.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 msgr2=0x55997c226310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 0x55997c226310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe6880ce1b0 tx=0x7fe688004840 comp rx=0 tx=0).stop 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 shutdown_connections 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c2268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c239100 0x55997c23b4f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21edc0 0x55997c226310 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 --2- 172.21.15.64:0/433113449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21e4a0 0x55997c229060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6757fa640 0 client.15321 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 >> 172.21.15.64:0/433113449 conn(0x55997c21aa30 msgr2=0x55997c220760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 shutdown_connections 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 1 -- 172.21.15.64:0/433113449 wait complete. 2024-09-19T05:06:04.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.546+0000 7fe6961ef740 20 client.15321 trim_cache size 0 max 16384 2024-09-19T05:06:04.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.563+0000 7fe6961ef740 1 Processor -- start 2024-09-19T05:06:04.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 -- start start 2024-09-19T05:06:04.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c21e4a0 0x55997c2354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c224be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 0x55997c240d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1ade30 con 0x55997c229680 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1ab700 con 0x55997c21edc0 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6961ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1a9550 con 0x55997c21e4a0 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 0x55997c240d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 0x55997c240d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44058/0 (socket says 172.21.15.64:44058) 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 -- 172.21.15.64:0/539115303 learned_addr learned my addr 172.21.15.64:0/539115303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe691bb2640 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c21e4a0 0x55997c2354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6913b1640 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c224be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 -- 172.21.15.64:0/539115303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c21e4a0 msgr2=0x55997c2354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c21e4a0 0x55997c2354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 -- 172.21.15.64:0/539115303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 msgr2=0x55997c224be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c224be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6923b3640 1 -- 172.21.15.64:0/539115303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55997c23b860 con 0x55997c229680 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.564+0000 7fe6913b1640 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c224be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.565+0000 7fe6923b3640 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 0x55997c240d50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe6840488a0 tx=0x7fe68402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf729cccfd610bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.565+0000 7fe6777fe640 1 -- 172.21.15.64:0/539115303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684048940 con 0x55997c229680 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.565+0000 7fe6777fe640 1 -- 172.21.15.64:0/539115303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe684067070 con 0x55997c229680 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.565+0000 7fe6777fe640 1 -- 172.21.15.64:0/539115303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684048480 con 0x55997c229680 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 -- 172.21.15.64:0/539115303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 msgr2=0x55997c240d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 0x55997c240d50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe6840488a0 tx=0x7fe68402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 -- 172.21.15.64:0/539115303 shutdown_connections 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c229680 0x55997c240d50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c224be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 --2- 172.21.15.64:0/539115303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c21e4a0 0x55997c2354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 -- 172.21.15.64:0/539115303 >> 172.21.15.64:0/539115303 conn(0x55997c21aa30 msgr2=0x55997c228390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 -- 172.21.15.64:0/539115303 shutdown_connections 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 -- 172.21.15.64:0/539115303 wait complete. 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 -- start start 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.566+0000 7fe6961ef740 1 Processor -- start 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c231780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c231cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1a9bc0 con 0x55997c21e4a0 2024-09-19T05:06:04.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1ab030 con 0x55997c21edc0 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1aa290 con 0x55997c229680 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6961ef740 10 client.0 _ll_register_callbacks cb 0x7fe6880096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c231780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c231780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44070/0 (socket says 172.21.15.64:44070) 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 -- 172.21.15.64:0/3056170020 learned_addr learned my addr 172.21.15.64:0/3056170020 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6923b3640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6913b1640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c231cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 -- 172.21.15.64:0/3056170020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 msgr2=0x55997c22db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 -- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 msgr2=0x55997c231cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c231cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 -- 172.21.15.64:0/3056170020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55997c2293d0 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe691bb2640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c231780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55997c23bc90 tx=0x7fe67c00bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55cfc5902064a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6757fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.567+0000 7fe6757fa640 1 -- 172.21.15.64:0/3056170020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c01fcd0 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684067070 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6757fa640 1 -- 172.21.15.64:0/3056170020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe67c005020 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6757fa640 1 -- 172.21.15.64:0/3056170020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe67c00be80 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 10 client.15327 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55997c2a4700 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6757fa640 1 -- 172.21.15.64:0/3056170020 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe684067070 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6757fa640 1 -- 172.21.15.64:0/3056170020 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe67c0035f0 con 0x55997c21e4a0 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 10 client.15327 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:04.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 10 client.15327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 20 client.15327 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c2a4e20 0x55997c2a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 4 client.15327 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55997c18bd60 con 0x55997c2a4e20 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.568+0000 7fe6913b1640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c2a4e20 0x55997c2a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6757fa640 10 client.15327 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6913b1640 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c2a4e20 0x55997c2a7210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb9f089b1d84337 server_cookie=10392ceef2859c61 in_seq=0 out_seq=0 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6757fa640 1 -- 172.21.15.64:0/3056170020 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55997c18bd60 con 0x55997c2a4e20 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6757fa640 10 client.15327 handle_command_reply: tid=0 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 client.15327 shutdown 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 msgr2=0x55997c231780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c231780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55997c23bc90 tx=0x7fe67c00bb70 comp rx=0 tx=0).stop 2024-09-19T05:06:04.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 shutdown_connections 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55997c229680 0x55997c22db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55997c21edc0 0x55997c231cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55997c2a4e20 0x55997c2a7210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 --2- 172.21.15.64:0/3056170020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55997c21e4a0 0x55997c231780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6757fa640 0 client.15327 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 >> 172.21.15.64:0/3056170020 conn(0x55997c21aa30 msgr2=0x55997c21fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.569+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 shutdown_connections 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe6961ef740 1 -- 172.21.15.64:0/3056170020 wait complete. 2024-09-19T05:06:04.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe6961ef740 20 client.15327 trim_cache size 0 max 16384 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: "95d34c67-3db3-4db2-be7b-39e919d9b173": { 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "95d34c67-3db3-4db2-be7b-39e919d9b173", 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:06:04.586 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:06:04.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe6780a3550 msgr2=0x7fe6780a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe6780a3550 0x7fe6780a5a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe684053000 tx=0x7fe68404f000 comp rx=0 tx=0).stop 2024-09-19T05:06:04.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 msgr2=0x7fe68c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 0x7fe68c1989e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe68800e9c0 tx=0x7fe68800ee80 comp rx=0 tx=0).stop 2024-09-19T05:06:04.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 shutdown_connections 2024-09-19T05:06:04.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe6780a3550 0x7fe6780a5a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe68c1082a0 0x7fe68c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe68c1036e0 0x7fe68c1989e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 --2- 172.21.15.64:0/4013773301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe68c102d30 0x7fe68c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 >> 172.21.15.64:0/4013773301 conn(0x7fe68c0fc7d0 msgr2=0x7fe68c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 shutdown_connections 2024-09-19T05:06:04.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.570+0000 7fe693e16640 1 -- 172.21.15.64:0/4013773301 wait complete. 2024-09-19T05:06:04.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 -- 172.21.15.64:0/3255076172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 msgr2=0x7fbc8c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 --2- 172.21.15.64:0/3255076172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c106dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009930 tx=0x7fbc7c02a420 comp rx=0 tx=0).stop 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 -- 172.21.15.64:0/3255076172 shutdown_connections 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 --2- 172.21.15.64:0/3255076172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c101370 0x7fbc8c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 --2- 172.21.15.64:0/3255076172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c1009d0 0x7fbc8c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 --2- 172.21.15.64:0/3255076172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c106dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 -- 172.21.15.64:0/3255076172 >> 172.21.15.64:0/3255076172 conn(0x7fbc8c0fc6f0 msgr2=0x7fbc8c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 -- 172.21.15.64:0/3255076172 shutdown_connections 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 -- 172.21.15.64:0/3255076172 wait complete. 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.584+0000 7fbc91442640 1 Processor -- start 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 -- start start 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1009d0 0x7fbc8c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c101370 0x7fbc8c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c111de0 con 0x7fbc8c1069e0 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c111c60 con 0x7fbc8c101370 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc91442640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c111f60 con 0x7fbc8c1009d0 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44090/0 (socket says 172.21.15.64:44090) 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 -- 172.21.15.64:0/3010292869 learned_addr learned my addr 172.21.15.64:0/3010292869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8affd640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1009d0 0x7fbc8c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8a7fc640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c101370 0x7fbc8c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 -- 172.21.15.64:0/3010292869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1009d0 msgr2=0x7fbc8c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1009d0 0x7fbc8c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 -- 172.21.15.64:0/3010292869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c101370 msgr2=0x7fbc8c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c101370 0x7fbc8c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8b7fe640 1 -- 172.21.15.64:0/3010292869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c19d280 con 0x7fbc8c1069e0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.585+0000 7fbc8a7fc640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c101370 0x7fbc8c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.586+0000 7fbc8b7fe640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c19cb50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400eb10 tx=0x7fbc8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.586+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004b20 con 0x7fbc8c1069e0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.586+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc84004cc0 con 0x7fbc8c1069e0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.586+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c19d570 con 0x7fbc8c1069e0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.586+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84010690 con 0x7fbc8c1069e0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.586+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a4e00 con 0x7fbc8c1069e0 2024-09-19T05:06:04.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.587+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c100e30 con 0x7fbc8c1069e0 2024-09-19T05:06:04.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.589+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7fbc84005000 con 0x7fbc8c1069e0 2024-09-19T05:06:04.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.589+0000 7fbc73fff640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc740a3580 0x7fbc740a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:04.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.590+0000 7fbc8affd640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc740a3580 0x7fbc740a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:04.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.590+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fbc840d0a10 con 0x7fbc8c1069e0 2024-09-19T05:06:04.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.590+0000 7fbc8affd640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc740a3580 0x7fbc740a5a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009900 tx=0x7fbc7c000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:04.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.593+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc8409d1a0 con 0x7fbc8c1069e0 2024-09-19T05:06:04.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.792+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fbc8c062630 con 0x7fbc740a3580 2024-09-19T05:06:04.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.799+0000 7fbc73fff640 1 -- 172.21.15.64:0/3010292869 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fbc8c062630 con 0x7fbc740a3580 2024-09-19T05:06:04.802 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1m 6m3h 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc740a3580 msgr2=0x7fbc740a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc740a3580 0x7fbc740a5a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009900 tx=0x7fbc7c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 msgr2=0x7fbc8c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c19cb50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400eb10 tx=0x7fbc8400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 shutdown_connections 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc740a3580 0x7fbc740a5a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8c1069e0 0x7fbc8c19cb50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc8c101370 0x7fbc8c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 --2- 172.21.15.64:0/3010292869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1009d0 0x7fbc8c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 >> 172.21.15.64:0/3010292869 conn(0x7fbc8c0fc6f0 msgr2=0x7fbc8c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:04.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 shutdown_connections 2024-09-19T05:06:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:04.801+0000 7fbc91442640 1 -- 172.21.15.64:0/3010292869 wait complete. 2024-09-19T05:06:04.815 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T05:06:04.846 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T05:06:04.857 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T05:06:04.875 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:06:04.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T05:06:04.889 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T05:06:04.902 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:06:04.903 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T05:06:04.903 INFO:tasks.workunit:timeout=3h 2024-09-19T05:06:04.903 INFO:tasks.workunit:cleanup=True 2024-09-19T05:06:04.903 DEBUG:teuthology.orchestra.run.smithi064:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T05:06:04.919 INFO:teuthology.orchestra.run.smithi064.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-19 05:04:23.260579862 +0000 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-19 05:04:23.260579862 +0000 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-19 05:05:52.272286618 +0000 2024-09-19T05:06:04.920 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-09-19T05:06:04.920 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T05:06:04.921 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T05:06:04.993 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:06:05.050 INFO:tasks.workunit.client.0.smithi064.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T05:06:05.303 INFO:tasks.cephfs.filesystem:scrub status for tag:95d34c67-3db3-4db2-be7b-39e919d9b173 - {'path': '/', 'tag': '95d34c67-3db3-4db2-be7b-39e919d9b173', 'options': 'recursive,force'} 2024-09-19T05:06:05.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:05 smithi064 ceph-mon[30960]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:05 smithi064 ceph-mon[30960]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 685 B/s wr, 17 op/s 2024-09-19T05:06:05.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:05 smithi064 ceph-mon[30960]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:05 smithi064 ceph-mon[30960]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:05 smithi103 ceph-mon[34446]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:05 smithi103 ceph-mon[34446]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 685 B/s wr, 17 op/s 2024-09-19T05:06:05.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:05 smithi103 ceph-mon[34446]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:05 smithi103 ceph-mon[34446]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:05 smithi174 ceph-mon[34488]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:05 smithi174 ceph-mon[34488]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 685 B/s wr, 17 op/s 2024-09-19T05:06:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:05 smithi174 ceph-mon[34488]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:05 smithi174 ceph-mon[34488]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:06.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:06 smithi064 ceph-mon[30960]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:06.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:06 smithi103 ceph-mon[34446]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:06.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:06 smithi174 ceph-mon[34488]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:06:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:07 smithi064 ceph-mon[30960]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-09-19T05:06:07.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:07 smithi103 ceph-mon[34446]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-09-19T05:06:07.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:07 smithi174 ceph-mon[34488]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-09-19T05:06:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:09 smithi064 ceph-mon[30960]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-19T05:06:09.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:09 smithi103 ceph-mon[34446]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-19T05:06:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:09 smithi174 ceph-mon[34488]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-19T05:06:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:11 smithi064 ceph-mon[30960]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-19T05:06:11.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:11 smithi103 ceph-mon[34446]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-19T05:06:11.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:11 smithi174 ceph-mon[34488]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-19T05:06:13.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:13 smithi174 ceph-mon[34488]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-19T05:06:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:13 smithi064 ceph-mon[30960]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-19T05:06:13.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:13 smithi103 ceph-mon[34446]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-19T05:06:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:14 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:06:14.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:14 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:06:14.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:14 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:06:15.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:15 smithi064 ceph-mon[30960]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-09-19T05:06:15.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:15 smithi103 ceph-mon[34446]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-09-19T05:06:15.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:15 smithi174 ceph-mon[34488]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-09-19T05:06:17.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:17 smithi064 ceph-mon[30960]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-09-19T05:06:17.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:17 smithi103 ceph-mon[34446]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-09-19T05:06:17.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:17 smithi174 ceph-mon[34488]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-09-19T05:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 -- 172.21.15.64:0/1150419876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c101ae0 msgr2=0x7ff47c101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 --2- 172.21.15.64:0/1150419876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c101ae0 0x7ff47c101f40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff470009930 tx=0x7ff47002a420 comp rx=0 tx=0).stop 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 -- 172.21.15.64:0/1150419876 shutdown_connections 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 --2- 172.21.15.64:0/1150419876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c102480 0x7ff47c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 --2- 172.21.15.64:0/1150419876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c101ae0 0x7ff47c101f40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 --2- 172.21.15.64:0/1150419876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff47c107b80 0x7ff47c107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 -- 172.21.15.64:0/1150419876 >> 172.21.15.64:0/1150419876 conn(0x7ff47c0fd7e0 msgr2=0x7ff47c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 -- 172.21.15.64:0/1150419876 shutdown_connections 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 -- 172.21.15.64:0/1150419876 wait complete. 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.032+0000 7ff482eae640 1 Processor -- start 2024-09-19T05:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 -- start start 2024-09-19T05:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff47c101ae0 0x7ff47c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 0x7ff47c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c107b80 0x7ff47c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c111de0 con 0x7ff47c102480 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c111c60 con 0x7ff47c101ae0 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff482eae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c111f60 con 0x7ff47c107b80 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 0x7ff47c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 0x7ff47c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48666/0 (socket says 172.21.15.64:48666) 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 -- 172.21.15.64:0/1091988889 learned_addr learned my addr 172.21.15.64:0/1091988889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff480c4a640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff47c101ae0 0x7ff47c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff48144b640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c107b80 0x7ff47c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 -- 172.21.15.64:0/1091988889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c107b80 msgr2=0x7ff47c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c107b80 0x7ff47c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 -- 172.21.15.64:0/1091988889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff47c101ae0 msgr2=0x7ff47c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff47c101ae0 0x7ff47c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 -- 172.21.15.64:0/1091988889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c19d2a0 con 0x7ff47c102480 2024-09-19T05:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.033+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 0x7ff47c1987f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff470005940 tx=0x7ff4700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.034+0000 7ff479ffb640 1 -- 172.21.15.64:0/1091988889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47003f030 con 0x7ff47c102480 2024-09-19T05:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.034+0000 7ff479ffb640 1 -- 172.21.15.64:0/1091988889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4700318a0 con 0x7ff47c102480 2024-09-19T05:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.034+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c19d530 con 0x7ff47c102480 2024-09-19T05:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.034+0000 7ff479ffb640 1 -- 172.21.15.64:0/1091988889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470035070 con 0x7ff47c102480 2024-09-19T05:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.034+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c1a4e20 con 0x7ff47c102480 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.036+0000 7ff479ffb640 1 -- 172.21.15.64:0/1091988889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7ff470042030 con 0x7ff47c102480 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 Processor -- start 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 -- start start 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff479ffb640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff4640a3600 0x7ff4640a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf645db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf654b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 0x55d5bf6574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d5bf5aa2f0 con 0x55d5bf6550c0 2024-09-19T05:06:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5bf552270 con 0x55d5bf6462f0 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff485c48740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5bf5a62b0 con 0x55d5bf6459d0 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff479ffb640 1 -- 172.21.15.64:0/1091988889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff4700f0620 con 0x7ff47c102480 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff48144b640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff4640a3600 0x7ff4640a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff48144b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 0x55d5bf6574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff48144b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 0x55d5bf6574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48678/0 (socket says 172.21.15.64:48678) 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff48144b640 1 -- 172.21.15.64:0/2005478287 learned_addr learned my addr 172.21.15.64:0/2005478287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.037+0000 7ff47bfff640 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf654b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 -- 172.21.15.64:0/2005478287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 msgr2=0x55d5bf645db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf645db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 -- 172.21.15.64:0/2005478287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 msgr2=0x55d5bf654b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf654b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 -- 172.21.15.64:0/2005478287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bf551340 con 0x55d5bf6550c0 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff47bfff640 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf654b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 0x55d5bf6574b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff474000b60 tx=0x7ff474009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dc542432c2272f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff48144b640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff4640a3600 0x7ff4640a5ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff474012c80 tx=0x7ff474009e80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff462ffd640 1 -- 172.21.15.64:0/2005478287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474012e30 con 0x55d5bf6550c0 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff462ffd640 1 -- 172.21.15.64:0/2005478287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff474009980 con 0x55d5bf6550c0 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.038+0000 7ff462ffd640 1 -- 172.21.15.64:0/2005478287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474005500 con 0x55d5bf6550c0 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 -- 172.21.15.64:0/2005478287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 msgr2=0x55d5bf6574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 0x55d5bf6574b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff474000b60 tx=0x7ff474009600 comp rx=0 tx=0).stop 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 -- 172.21.15.64:0/2005478287 shutdown_connections 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6550c0 0x55d5bf6574b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf654b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 --2- 172.21.15.64:0/2005478287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf645db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 -- 172.21.15.64:0/2005478287 >> 172.21.15.64:0/2005478287 conn(0x55d5bf642060 msgr2=0x55d5bf642440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 -- 172.21.15.64:0/2005478287 shutdown_connections 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 -- 172.21.15.64:0/2005478287 wait complete. 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 -- start start 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.039+0000 7ff485c48740 1 Processor -- start 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf6520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6462f0 0x55d5bf652610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf652b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d5bf657c30 con 0x55d5bf6459d0 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5bf533290 con 0x55d5bf6550c0 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5bf657db0 con 0x55d5bf6462f0 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff485c48740 10 client.0 _ll_register_callbacks cb 0x55d5bf564140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf6520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf6520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48686/0 (socket says 172.21.15.64:48686) 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 -- 172.21.15.64:0/1487277765 learned_addr learned my addr 172.21.15.64:0/1487277765 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6462f0 0x55d5bf652610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 -- 172.21.15.64:0/1487277765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6462f0 msgr2=0x55d5bf652610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6462f0 0x55d5bf652610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 -- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 msgr2=0x55d5bf652b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf652b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 -- 172.21.15.64:0/1487277765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bf2bd210 con 0x55d5bf6459d0 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6462f0 0x55d5bf652610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff480c4a640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf6520d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff46c011690 tx=0x7ff46c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc7cbec791ebde7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.040+0000 7ff45bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff45bfff640 1 -- 172.21.15.64:0/1487277765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c004250 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff45bfff640 1 -- 172.21.15.64:0/1487277765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff46c0043f0 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff474009980 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff45bfff640 1 -- 172.21.15.64:0/1487277765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff46c011e90 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 10 client.15345 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5bf389620 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff45bfff640 1 -- 172.21.15.64:0/1487277765 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff474009980 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff45bfff640 1 -- 172.21.15.64:0/1487277765 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff46c003b50 con 0x55d5bf6459d0 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 10 client.15345 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 20 client.15345 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6650a0 0x55d5bf667510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 4 client.15345 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5bf5b7a20 con 0x55d5bf6650a0 2024-09-19T05:06:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.041+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6650a0 0x55d5bf667510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.042+0000 7ff45bfff640 10 client.15345 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.042+0000 7ff47bfff640 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6650a0 0x55d5bf667510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9e8725a2fda033 server_cookie=261eaa5cb60b3ea3 in_seq=0 out_seq=0 2024-09-19T05:06:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff45bfff640 1 -- 172.21.15.64:0/1487277765 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5bf5b7a20 con 0x55d5bf6650a0 2024-09-19T05:06:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff45bfff640 10 client.15345 handle_command_reply: tid=0 2024-09-19T05:06:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 client.15345 shutdown 2024-09-19T05:06:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 msgr2=0x55d5bf6520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf6520d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff46c011690 tx=0x7ff46c011b50 comp rx=0 tx=0).stop 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 shutdown_connections 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf652b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6462f0 0x55d5bf652610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6650a0 0x55d5bf667510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 --2- 172.21.15.64:0/1487277765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf6520d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff45bfff640 0 client.15345 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 >> 172.21.15.64:0/1487277765 conn(0x55d5bf642060 msgr2=0x55d5bf647dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 shutdown_connections 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 1 -- 172.21.15.64:0/1487277765 wait complete. 2024-09-19T05:06:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.043+0000 7ff485c48740 20 client.15345 trim_cache size 0 max 16384 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 Processor -- start 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 -- start start 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf6410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 0x55d5bf66ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf66d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1addf0 con 0x55d5bf6462f0 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1ab6c0 con 0x55d5bf6550c0 2024-09-19T05:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff485c48740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1a9510 con 0x55d5bf6459d0 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 0x55d5bf66ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 0x55d5bf66ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48702/0 (socket says 172.21.15.64:48702) 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 -- 172.21.15.64:0/3183116402 learned_addr learned my addr 172.21.15.64:0/3183116402 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff480c4a640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf6410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 -- 172.21.15.64:0/3183116402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 msgr2=0x55d5bf6410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf6410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff48144b640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf66d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 -- 172.21.15.64:0/3183116402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 msgr2=0x55d5bf66d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf66d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff47bfff640 1 -- 172.21.15.64:0/3183116402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bf660e90 con 0x55d5bf6462f0 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff480c4a640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf6410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.058+0000 7ff48144b640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf66d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff47bfff640 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 0x55d5bf66ce70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff470009340 tx=0x7ff4700a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984cb4e9c6c27168 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff4627fc640 1 -- 172.21.15.64:0/3183116402 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470049040 con 0x55d5bf6462f0 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff4627fc640 1 -- 172.21.15.64:0/3183116402 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4700cf090 con 0x55d5bf6462f0 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff4627fc640 1 -- 172.21.15.64:0/3183116402 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4700b8df0 con 0x55d5bf6462f0 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 -- 172.21.15.64:0/3183116402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 msgr2=0x55d5bf66ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 0x55d5bf66ce70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff470009340 tx=0x7ff4700a7040 comp rx=0 tx=0).stop 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 -- 172.21.15.64:0/3183116402 shutdown_connections 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6550c0 0x55d5bf66d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6462f0 0x55d5bf66ce70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 --2- 172.21.15.64:0/3183116402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6459d0 0x55d5bf6410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 -- 172.21.15.64:0/3183116402 >> 172.21.15.64:0/3183116402 conn(0x55d5bf642060 msgr2=0x55d5bf6409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 -- 172.21.15.64:0/3183116402 shutdown_connections 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.059+0000 7ff485c48740 1 -- 172.21.15.64:0/3183116402 wait complete. 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 -- start start 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 Processor -- start 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf658d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf6592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6550c0 0x55d5bf659810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1a9b80 con 0x55d5bf6459d0 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1aaff0 con 0x55d5bf6462f0 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1aa250 con 0x55d5bf6550c0 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff485c48740 10 client.0 _ll_register_callbacks cb 0x7ff46c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff480c4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf658d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff480c4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf658d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48716/0 (socket says 172.21.15.64:48716) 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff480c4a640 1 -- 172.21.15.64:0/463552047 learned_addr learned my addr 172.21.15.64:0/463552047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.060+0000 7ff48144b640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6550c0 0x55d5bf659810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff47bfff640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf6592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff480c4a640 1 -- 172.21.15.64:0/463552047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6550c0 msgr2=0x55d5bf659810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff480c4a640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6550c0 0x55d5bf659810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff480c4a640 1 -- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 msgr2=0x55d5bf6592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff480c4a640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf6592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff480c4a640 1 -- 172.21.15.64:0/463552047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bf6cee90 con 0x55d5bf6459d0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff480c4a640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf658d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff46c011690 tx=0x7ff46c011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7fd971df3c9c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff45bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff45bfff640 1 -- 172.21.15.64:0/463552047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c009cd0 con 0x55d5bf6459d0 2024-09-19T05:06:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff45bfff640 1 -- 172.21.15.64:0/463552047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff46c00f8c0 con 0x55d5bf6459d0 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4700cf090 con 0x55d5bf6459d0 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff45bfff640 1 -- 172.21.15.64:0/463552047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff46c00fac0 con 0x55d5bf6459d0 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff485c48740 10 client.15351 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5bf6d00f0 con 0x55d5bf6459d0 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.061+0000 7ff45bfff640 1 -- 172.21.15.64:0/463552047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4700cf090 con 0x55d5bf6459d0 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff45bfff640 1 -- 172.21.15.64:0/463552047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff46c003680 con 0x55d5bf6459d0 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff485c48740 10 client.15351 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff485c48740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff485c48740 20 client.15351 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff485c48740 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6d0870 0x55d5bf6d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff485c48740 4 client.15351 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5bf5b1d70 con 0x55d5bf6d0870 2024-09-19T05:06:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff47bfff640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6d0870 0x55d5bf6d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff47bfff640 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6d0870 0x55d5bf6d2c90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6f8250bdeef276 server_cookie=a0196e544ad191b1 in_seq=0 out_seq=0 2024-09-19T05:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.062+0000 7ff45bfff640 10 client.15351 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff45bfff640 1 -- 172.21.15.64:0/463552047 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5bf5b1d70 con 0x55d5bf6d0870 2024-09-19T05:06:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff45bfff640 10 client.15351 handle_command_reply: tid=0 2024-09-19T05:06:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 client.15351 shutdown 2024-09-19T05:06:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 msgr2=0x55d5bf658d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf658d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff46c011690 tx=0x7ff46c011fd0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 shutdown_connections 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5bf6550c0 0x55d5bf659810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5bf6462f0 0x55d5bf6592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d5bf6d0870 0x55d5bf6d2c90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 --2- 172.21.15.64:0/463552047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5bf6459d0 0x55d5bf658d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 >> 172.21.15.64:0/463552047 conn(0x55d5bf642060 msgr2=0x55d5bf65fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff45bfff640 0 client.15351 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 shutdown_connections 2024-09-19T05:06:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 1 -- 172.21.15.64:0/463552047 wait complete. 2024-09-19T05:06:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.073+0000 7ff485c48740 20 client.15351 trim_cache size 0 max 16384 2024-09-19T05:06:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:06:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:06:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:06:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff4640a3600 msgr2=0x7ff4640a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff4640a3600 0x7ff4640a5ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff474012c80 tx=0x7ff474009e80 comp rx=0 tx=0).stop 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 msgr2=0x7ff47c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 0x7ff47c1987f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff470005940 tx=0x7ff4700042f0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 shutdown_connections 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff4640a3600 0x7ff4640a5ac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c107b80 0x7ff47c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff47c102480 0x7ff47c1987f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 --2- 172.21.15.64:0/1091988889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff47c101ae0 0x7ff47c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 >> 172.21.15.64:0/1091988889 conn(0x7ff47c0fd7e0 msgr2=0x7ff47c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 shutdown_connections 2024-09-19T05:06:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:18.074+0000 7ff482eae640 1 -- 172.21.15.64:0/1091988889 wait complete. 2024-09-19T05:06:19.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:19 smithi064 ceph-mon[30960]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T05:06:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:19 smithi103 ceph-mon[34446]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T05:06:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:19 smithi174 ceph-mon[34488]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T05:06:21.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:21 smithi064 ceph-mon[30960]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T05:06:21.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:21 smithi103 ceph-mon[34446]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T05:06:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:21 smithi174 ceph-mon[34488]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T05:06:23.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:23 smithi064 ceph-mon[30960]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T05:06:23.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:23 smithi103 ceph-mon[34446]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T05:06:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:23 smithi174 ceph-mon[34488]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-19T05:06:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:25 smithi064 ceph-mon[30960]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-19T05:06:25.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:25 smithi103 ceph-mon[34446]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-19T05:06:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:25 smithi174 ceph-mon[34488]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-19T05:06:27.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:27 smithi064 ceph-mon[30960]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T05:06:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:27 smithi103 ceph-mon[34446]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T05:06:27.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:27 smithi174 ceph-mon[34488]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T05:06:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:29 smithi064 ceph-mon[30960]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-19T05:06:29.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:29 smithi103 ceph-mon[34446]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-19T05:06:29.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:29 smithi174 ceph-mon[34488]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-19T05:06:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 -- 172.21.15.64:0/1821265037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b01069c0 msgr2=0x7ff5b0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 --2- 172.21.15.64:0/1821265037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b01069c0 0x7ff5b0106da0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009930 tx=0x7ff5a002a420 comp rx=0 tx=0).stop 2024-09-19T05:06:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 -- 172.21.15.64:0/1821265037 shutdown_connections 2024-09-19T05:06:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 --2- 172.21.15.64:0/1821265037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b0101350 0x7ff5b010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 --2- 172.21.15.64:0/1821265037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01009b0 0x7ff5b0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 --2- 172.21.15.64:0/1821265037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b01069c0 0x7ff5b0106da0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 -- 172.21.15.64:0/1821265037 >> 172.21.15.64:0/1821265037 conn(0x7ff5b0077990 msgr2=0x7ff5b00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 -- 172.21.15.64:0/1821265037 shutdown_connections 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.181+0000 7ff5b668d640 1 -- 172.21.15.64:0/1821265037 wait complete. 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 Processor -- start 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 -- start start 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009b0 0x7ff5b01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 0x7ff5b0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069c0 0x7ff5b019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110d00 con 0x7ff5b0101350 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110b80 con 0x7ff5b01069c0 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5b668d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110e80 con 0x7ff5b01009b0 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 0x7ff5b0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 0x7ff5b0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54260/0 (socket says 172.21.15.64:54260) 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.182+0000 7ff5af7fe640 1 -- 172.21.15.64:0/3808080286 learned_addr learned my addr 172.21.15.64:0/3808080286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069c0 0x7ff5b019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5affff640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009b0 0x7ff5b01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5af7fe640 1 -- 172.21.15.64:0/3808080286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009b0 msgr2=0x7ff5b01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009b0 0x7ff5b01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5af7fe640 1 -- 172.21.15.64:0/3808080286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069c0 msgr2=0x7ff5b019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069c0 0x7ff5b019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5af7fe640 1 -- 172.21.15.64:0/3808080286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b019d2d0 con 0x7ff5b0101350 2024-09-19T05:06:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5affff640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009b0 0x7ff5b01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 0x7ff5b0198820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d950 tx=0x7ff5a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5ad7fa640 1 -- 172.21.15.64:0/3808080286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4017070 con 0x7ff5b0101350 2024-09-19T05:06:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b019d5c0 con 0x7ff5b0101350 2024-09-19T05:06:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5ad7fa640 1 -- 172.21.15.64:0/3808080286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a4009c30 con 0x7ff5b0101350 2024-09-19T05:06:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5ad7fa640 1 -- 172.21.15.64:0/3808080286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400f620 con 0x7ff5b0101350 2024-09-19T05:06:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.183+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a4e50 con 0x7ff5b0101350 2024-09-19T05:06:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.186+0000 7ff5ad7fa640 1 -- 172.21.15.64:0/3808080286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7ff5a40109a0 con 0x7ff5b0101350 2024-09-19T05:06:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.186+0000 7ff5b9428740 1 Processor -- start 2024-09-19T05:06:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.186+0000 7ff5b9428740 1 -- start start 2024-09-19T05:06:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422d8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b9428740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ba4222e2f0 con 0x55ba422ca2f0 2024-09-19T05:06:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b9428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ba421d6270 con 0x55ba422d90c0 2024-09-19T05:06:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b9428740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba4222a2b0 con 0x55ba422c99d0 2024-09-19T05:06:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422d8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422d8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54276/0 (socket says 172.21.15.64:54276) 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 -- 172.21.15.64:0/1692229120 learned_addr learned my addr 172.21.15.64:0/1692229120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 -- 172.21.15.64:0/1692229120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 msgr2=0x55ba422c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 -- 172.21.15.64:0/1692229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 msgr2=0x55ba422db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5ad7fa640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5980a3600 0x7ff5980a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422db4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 -- 172.21.15.64:0/1692229120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba421d5340 con 0x55ba422ca2f0 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5980a3600 0x7ff5980a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5ad7fa640 1 -- 172.21.15.64:0/3808080286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff5a40cfc40 con 0x7ff5b0101350 2024-09-19T05:06:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.187+0000 7ff5affff640 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422d8b80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009a50 tx=0x7ff5a0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff0277bbfa050ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5967fc640 1 -- 172.21.15.64:0/1692229120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a003f020 con 0x55ba422ca2f0 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5967fc640 1 -- 172.21.15.64:0/1692229120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a00046c0 con 0x55ba422ca2f0 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5967fc640 1 -- 172.21.15.64:0/1692229120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0035070 con 0x55ba422ca2f0 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b9428740 1 -- 172.21.15.64:0/1692229120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 msgr2=0x55ba422d8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422d8b80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009a50 tx=0x7ff5a0004390 comp rx=0 tx=0).stop 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5980a3600 0x7ff5980a5ac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005cc0 tx=0x7ff5a800a2b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b9428740 1 -- 172.21.15.64:0/1692229120 shutdown_connections 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422d8b80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.188+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1692229120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- 172.21.15.64:0/1692229120 >> 172.21.15.64:0/1692229120 conn(0x55ba422c6060 msgr2=0x55ba422c6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- 172.21.15.64:0/1692229120 shutdown_connections 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- 172.21.15.64:0/1692229120 wait complete. 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- start start 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 Processor -- start 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ba422dbc90 con 0x55ba422d90c0 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ba421b7290 con 0x55ba422c99d0 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba422dbe10 con 0x55ba422ca2f0 2024-09-19T05:06:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.189+0000 7ff5b9428740 10 client.0 _ll_register_callbacks cb 0x55ba421e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422d6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54280/0 (socket says 172.21.15.64:54280) 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/287245271 learned_addr learned my addr 172.21.15.64:0/287245271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5affff640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/287245271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 msgr2=0x55ba422d65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422d65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 msgr2=0x55ba422d60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422d60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/287245271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba41f41210 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422d6b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8010af0 tx=0x7ff5a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e9fa5dfcc1d5d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff58f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff58f7fe640 1 -- 172.21.15.64:0/287245271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8021070 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff58f7fe640 1 -- 172.21.15.64:0/287245271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff5a800fcd0 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.190+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a00046c0 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff58f7fe640 1 -- 172.21.15.64:0/287245271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5a802d050 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 10 client.15363 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba4200d620 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff58f7fe640 1 -- 172.21.15.64:0/287245271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a00046c0 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff58f7fe640 1 -- 172.21.15.64:0/287245271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5a80254d0 con 0x55ba422d90c0 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 10 client.15363 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 20 client.15363 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba422eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 4 client.15363 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba4223ba20 con 0x55ba422e90f0 2024-09-19T05:06:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.191+0000 7ff5affff640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba422eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.192+0000 7ff5affff640 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba422eb540 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87942fef055f7eb3 server_cookie=659277e54fb287a0 in_seq=0 out_seq=0 2024-09-19T05:06:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.192+0000 7ff58f7fe640 10 client.15363 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.192+0000 7ff58f7fe640 1 -- 172.21.15.64:0/287245271 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba4223ba20 con 0x55ba422e90f0 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.192+0000 7ff58f7fe640 10 client.15363 handle_command_reply: tid=0 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.192+0000 7ff5b9428740 1 client.15363 shutdown 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 msgr2=0x55ba422d6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422d6b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8010af0 tx=0x7ff5a8004ab0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 shutdown_connections 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba422eb540 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422d6b30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff58f7fe640 0 client.15363 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 --2- 172.21.15.64:0/287245271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422d60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 >> 172.21.15.64:0/287245271 conn(0x55ba422c6060 msgr2=0x55ba422cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 shutdown_connections 2024-09-19T05:06:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 1 -- 172.21.15.64:0/287245271 wait complete. 2024-09-19T05:06:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.193+0000 7ff5b9428740 20 client.15363 trim_cache size 0 max 16384 2024-09-19T05:06:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 Processor -- start 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 -- start start 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01addf0 con 0x55ba422ca2f0 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01ab6c0 con 0x55ba422d90c0 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b9428740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a9510 con 0x55ba422c99d0 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422f13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54284/0 (socket says 172.21.15.64:54284) 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 -- 172.21.15.64:0/1690819782 learned_addr learned my addr 172.21.15.64:0/1690819782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5affff640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 -- 172.21.15.64:0/1690819782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 msgr2=0x55ba422f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 -- 172.21.15.64:0/1690819782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 msgr2=0x55ba422d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422d32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5af7fe640 1 -- 172.21.15.64:0/1690819782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba422e7cb0 con 0x55ba422ca2f0 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.208+0000 7ff5affff640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.209+0000 7ff5af7fe640 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422f13a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40cd320 tx=0x7ff5a40b70b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1820a395ad2565c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.209+0000 7ff595ffb640 1 -- 172.21.15.64:0/1690819782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4020070 con 0x55ba422ca2f0 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.209+0000 7ff595ffb640 1 -- 172.21.15.64:0/1690819782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a409bcd0 con 0x55ba422ca2f0 2024-09-19T05:06:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.209+0000 7ff595ffb640 1 -- 172.21.15.64:0/1690819782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40cd750 con 0x55ba422ca2f0 2024-09-19T05:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 -- 172.21.15.64:0/1690819782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 msgr2=0x55ba422f13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422f13a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40cd320 tx=0x7ff5a40b70b0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 -- 172.21.15.64:0/1690819782 shutdown_connections 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422d90c0 0x55ba422d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422ca2f0 0x55ba422f13a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 --2- 172.21.15.64:0/1690819782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422c99d0 0x55ba422f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 -- 172.21.15.64:0/1690819782 >> 172.21.15.64:0/1690819782 conn(0x55ba422c6060 msgr2=0x55ba422d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 -- 172.21.15.64:0/1690819782 shutdown_connections 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 -- 172.21.15.64:0/1690819782 wait complete. 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 -- start start 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.210+0000 7ff5b9428740 1 Processor -- start 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a9b80 con 0x55ba422d90c0 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01aaff0 con 0x55ba422c99d0 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01aa250 con 0x55ba422ca2f0 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b9428740 10 client.0 _ll_register_callbacks cb 0x7ff5a8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54294/0 (socket says 172.21.15.64:54294) 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/2709275969 learned_addr learned my addr 172.21.15.64:0/2709275969 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5affff640 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/2709275969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 msgr2=0x55ba422dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 msgr2=0x55ba422dcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422dcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 -- 172.21.15.64:0/2709275969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba42352ee0 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff5b4c2a640 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422dd920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005e10 tx=0x7ff5a800a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7938dc3ca324a539 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.211+0000 7ff58f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff58f7fe640 1 -- 172.21.15.64:0/2709275969 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8021040 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff58f7fe640 1 -- 172.21.15.64:0/2709275969 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff5a80255b0 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a409bcd0 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff58f7fe640 1 -- 172.21.15.64:0/2709275969 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5a800fad0 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 10 client.15369 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba423541a0 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff58f7fe640 1 -- 172.21.15.64:0/2709275969 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a409bcd0 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff58f7fe640 1 -- 172.21.15.64:0/2709275969 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5a801dc10 con 0x55ba422d90c0 2024-09-19T05:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 10 client.15369 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 10 client.15369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 20 client.15369 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba42356950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 4 client.15369 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.212+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba42235d70 con 0x55ba422e90f0 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.213+0000 7ff5affff640 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba42356950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.213+0000 7ff5affff640 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba42356950 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199ec375a5ea3e0b server_cookie=54f347b78c06ff47 in_seq=0 out_seq=0 2024-09-19T05:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.213+0000 7ff58f7fe640 10 client.15369 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff58f7fe640 1 -- 172.21.15.64:0/2709275969 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba42235d70 con 0x55ba422e90f0 2024-09-19T05:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff58f7fe640 10 client.15369 handle_command_reply: tid=0 2024-09-19T05:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 client.15369 shutdown 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 msgr2=0x55ba422dd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422dd920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005e10 tx=0x7ff5a800a590 comp rx=0 tx=0).stop 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 shutdown_connections 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba422e90f0 0x55ba42356950 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba422d90c0 0x55ba422dd920 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba422ca2f0 0x55ba422dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 --2- 172.21.15.64:0/2709275969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba422c99d0 0x55ba422dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 >> 172.21.15.64:0/2709275969 conn(0x55ba422c6060 msgr2=0x55ba422c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff58f7fe640 0 client.15369 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 shutdown_connections 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 1 -- 172.21.15.64:0/2709275969 wait complete. 2024-09-19T05:06:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.217+0000 7ff5b9428740 20 client.15369 trim_cache size 0 max 16384 2024-09-19T05:06:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:06:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:06:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:06:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5980a3600 msgr2=0x7ff5980a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5980a3600 0x7ff5980a5ac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005cc0 tx=0x7ff5a800a2b0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 msgr2=0x7ff5b0198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 0x7ff5b0198820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d950 tx=0x7ff5a400de10 comp rx=0 tx=0).stop 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 shutdown_connections 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5980a3600 0x7ff5980a5ac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5b01069c0 0x7ff5b019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5b0101350 0x7ff5b0198820 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 --2- 172.21.15.64:0/3808080286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5b01009b0 0x7ff5b01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 >> 172.21.15.64:0/3808080286 conn(0x7ff5b0077990 msgr2=0x7ff5b0069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 shutdown_connections 2024-09-19T05:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:31.218+0000 7ff5b668d640 1 -- 172.21.15.64:0/3808080286 wait complete. 2024-09-19T05:06:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:31 smithi064 ceph-mon[30960]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-19T05:06:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:31 smithi103 ceph-mon[34446]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-19T05:06:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:31 smithi174 ceph-mon[34488]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-19T05:06:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:33 smithi064 ceph-mon[30960]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-19T05:06:33.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:33 smithi103 ceph-mon[34446]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-19T05:06:33.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:33 smithi174 ceph-mon[34488]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-19T05:06:35.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:06:35.605 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:35 smithi064 ceph-mon[30960]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:06:35.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:35 smithi103 ceph-mon[34446]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:06:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:35 smithi174 ceph-mon[34488]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:06:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 -- 172.21.15.64:0/739730312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0106f10 msgr2=0x7f5de010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 --2- 172.21.15.64:0/739730312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0106f10 0x7f5de010ea90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 -- 172.21.15.64:0/739730312 shutdown_connections 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 --2- 172.21.15.64:0/739730312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0106f10 0x7f5de010ea90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 --2- 172.21.15.64:0/739730312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de0102790 0x7f5de01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 --2- 172.21.15.64:0/739730312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0101de0 0x7f5de01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 -- 172.21.15.64:0/739730312 >> 172.21.15.64:0/739730312 conn(0x7f5de00fd8e0 msgr2=0x7f5de00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 -- 172.21.15.64:0/739730312 shutdown_connections 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.240+0000 7f5de82b7640 1 -- 172.21.15.64:0/739730312 wait complete. 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.241+0000 7f5de82b7640 1 Processor -- start 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.241+0000 7f5de82b7640 1 -- start start 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.241+0000 7f5de82b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 0x7f5de0073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.241+0000 7f5de82b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de0102790 0x7f5de006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de82b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0106f10 0x7f5de006d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de82b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0111e20 con 0x7f5de0101de0 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de82b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0111ca0 con 0x7f5de0106f10 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de82b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0111fa0 con 0x7f5de0102790 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6053640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 0x7f5de0073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6854640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0106f10 0x7f5de006d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6053640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 0x7f5de0073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43810/0 (socket says 172.21.15.64:43810) 2024-09-19T05:06:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6053640 1 -- 172.21.15.64:0/4253278485 learned_addr learned my addr 172.21.15.64:0/4253278485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6854640 1 -- 172.21.15.64:0/4253278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de0102790 msgr2=0x7f5de006cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6854640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de0102790 0x7f5de006cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6854640 1 -- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 msgr2=0x7f5de0073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.242+0000 7f5de6854640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 0x7f5de0073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5de6854640 1 -- 172.21.15.64:0/4253278485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de006d650 con 0x7f5de0106f10 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5de6053640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 0x7f5de0073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5de6854640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0106f10 0x7f5de006d010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd8032ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5dd77fe640 1 -- 172.21.15.64:0/4253278485 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003c30 con 0x7f5de0106f10 2024-09-19T05:06:36.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de006d8e0 con 0x7f5de0106f10 2024-09-19T05:06:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5dd77fe640 1 -- 172.21.15.64:0/4253278485 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd8003dd0 con 0x7f5de0106f10 2024-09-19T05:06:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5dd77fe640 1 -- 172.21.15.64:0/4253278485 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8030b10 con 0x7f5de0106f10 2024-09-19T05:06:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.243+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de0071900 con 0x7f5de0106f10 2024-09-19T05:06:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.247+0000 7f5dd77fe640 1 -- 172.21.15.64:0/4253278485 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f5dd8030cb0 con 0x7f5de0106f10 2024-09-19T05:06:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.247+0000 7f5dea690740 1 Processor -- start 2024-09-19T05:06:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.247+0000 7f5dea690740 1 -- start start 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.247+0000 7f5dea690740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7baf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.247+0000 7f5dea690740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5dea690740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bbca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5dea690740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5577b7b152e0 con 0x5577b7baf4a0 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5dea690740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577b7abcce0 con 0x5577b7bba680 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5dea690740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5577b7b112a0 con 0x5577b7bafdc0 2024-09-19T05:06:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7baf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7baf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43814/0 (socket says 172.21.15.64:43814) 2024-09-19T05:06:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 -- 172.21.15.64:0/3883142552 learned_addr learned my addr 172.21.15.64:0/3883142552 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bbca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de6053640 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 -- 172.21.15.64:0/3883142552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 msgr2=0x5577b7bba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 -- 172.21.15.64:0/3883142552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 msgr2=0x5577b7bbca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bbca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 -- 172.21.15.64:0/3883142552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b7aa1440 con 0x5577b7baf4a0 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de6053640 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5dd77fe640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5dcc0a3550 0x7f5dcc0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.248+0000 7f5de5852640 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7baf880 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0004650 tx=0x7f5dd000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0aeb098f6f76fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5dd77fe640 1 -- 172.21.15.64:0/4253278485 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5dd80f12f0 con 0x7f5de0106f10 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5de6053640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5dcc0a3550 0x7f5dcc0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5dd4dea640 1 -- 172.21.15.64:0/3883142552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005220 con 0x5577b7baf4a0 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5dd4dea640 1 -- 172.21.15.64:0/3883142552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd0004c50 con 0x5577b7baf4a0 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5dd4dea640 1 -- 172.21.15.64:0/3883142552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0012630 con 0x5577b7baf4a0 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5dea690740 1 -- 172.21.15.64:0/3883142552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 msgr2=0x5577b7baf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5dea690740 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7baf880 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0004650 tx=0x7f5dd000b260 comp rx=0 tx=0).stop 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.249+0000 7f5de6053640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5dcc0a3550 0x7f5dcc0a5a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5577b7bac890 tx=0x7f5ddc005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 -- 172.21.15.64:0/3883142552 shutdown_connections 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bbca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 --2- 172.21.15.64:0/3883142552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7baf880 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 -- 172.21.15.64:0/3883142552 >> 172.21.15.64:0/3883142552 conn(0x5577b7baba30 msgr2=0x5577b7babe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 -- 172.21.15.64:0/3883142552 shutdown_connections 2024-09-19T05:06:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 -- 172.21.15.64:0/3883142552 wait complete. 2024-09-19T05:06:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.250+0000 7f5dea690740 1 -- start start 2024-09-19T05:06:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 Processor -- start 2024-09-19T05:06:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bb7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bba680 0x5577b7bb7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5577b7bbd250 con 0x5577b7bba680 2024-09-19T05:06:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5de6053640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5de6053640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48188/0 (socket says 172.21.15.64:48188) 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5de6053640 1 -- 172.21.15.64:0/3338755239 learned_addr learned my addr 172.21.15.64:0/3338755239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5de6854640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bba680 0x5577b7bb7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577b7ad0050 con 0x5577b7baf4a0 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.251+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5577b7bbd3d0 con 0x5577b7bafdc0 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5dea690740 10 client.0 _ll_register_callbacks cb 0x5577b7b10160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6854640 1 -- 172.21.15.64:0/3338755239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 msgr2=0x5577b7bb7360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de5852640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bb7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6854640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bb7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6854640 1 -- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 msgr2=0x5577b7bba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6854640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6854640 1 -- 172.21.15.64:0/3338755239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b78863c0 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd0004c50 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6053640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5de6854640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bba680 0x5577b7bb7af0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ee9d0 tx=0x7f5dd80bd0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e73ed2c7fe2b207 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.252+0000 7f5dc9ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.253+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/3338755239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80d6060 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.253+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/3338755239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5dd80bd980 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.253+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/3338755239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dd80c18d0 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.253+0000 7f5dea690740 10 client.15381 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.253+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5577b7bd1390 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/3338755239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5dd80d6200 con 0x5577b7bba680 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dea690740 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dea690740 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dea690740 20 client.15381 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dea690740 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7bd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dea690740 4 client.15381 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b7b22850 con 0x5577b7bd27f0 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.254+0000 7f5de6053640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7bd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.255+0000 7f5dc9ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.255+0000 7f5de6053640 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7bd4c30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99839adbb84bc09b server_cookie=cc334a8a45fb8bd6 in_seq=0 out_seq=0 2024-09-19T05:06:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/3338755239 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577b7b22850 con 0x5577b7bd27f0 2024-09-19T05:06:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dc9ffb640 10 client.15381 handle_command_reply: tid=0 2024-09-19T05:06:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 client.15381 shutdown 2024-09-19T05:06:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bba680 msgr2=0x5577b7bb7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bba680 0x5577b7bb7af0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ee9d0 tx=0x7f5dd80bd0e0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 shutdown_connections 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7bd4c30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bba680 0x5577b7bb7af0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dc9ffb640 0 client.15381 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bb7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 --2- 172.21.15.64:0/3338755239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 >> 172.21.15.64:0/3338755239 conn(0x5577b7baba30 msgr2=0x5577b7bb1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 shutdown_connections 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.256+0000 7f5dea690740 1 -- 172.21.15.64:0/3338755239 wait complete. 2024-09-19T05:06:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.257+0000 7f5dea690740 20 client.15381 trim_cache size 0 max 16384 2024-09-19T05:06:36.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.274+0000 7f5dea690740 1 Processor -- start 2024-09-19T05:06:36.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.274+0000 7f5dea690740 1 -- start start 2024-09-19T05:06:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5dea690740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7bb5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5dea690740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bda500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5dea690740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bdaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5dea690740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b66c0 con 0x5577b7baf4a0 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5dea690740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b3f90 con 0x5577b7bba680 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5dea690740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b1de0 con 0x5577b7bafdc0 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de5852640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bda500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6854640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bdaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de5852640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bda500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:56020/0 (socket says 172.21.15.64:56020) 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de5852640 1 -- 172.21.15.64:0/1109123041 learned_addr learned my addr 172.21.15.64:0/1109123041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6053640 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7bb5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6854640 1 -- 172.21.15.64:0/1109123041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 msgr2=0x5577b7bda500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6854640 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bda500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6854640 1 -- 172.21.15.64:0/1109123041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 msgr2=0x5577b7bb5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6854640 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7bb5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.275+0000 7f5de6854640 1 -- 172.21.15.64:0/1109123041 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b7bb6310 con 0x5577b7bba680 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.276+0000 7f5de6854640 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bdaa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ef040 tx=0x7f5dd80f0000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1700cf81906b992e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.276+0000 7f5dcbfff640 1 -- 172.21.15.64:0/1109123041 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80d6040 con 0x5577b7bba680 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.276+0000 7f5dcbfff640 1 -- 172.21.15.64:0/1109123041 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd0004c50 con 0x5577b7bba680 2024-09-19T05:06:36.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.276+0000 7f5dcbfff640 1 -- 172.21.15.64:0/1109123041 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80bd780 con 0x5577b7bba680 2024-09-19T05:06:36.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 -- 172.21.15.64:0/1109123041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 msgr2=0x5577b7bdaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bdaa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ef040 tx=0x7f5dd80f0000 comp rx=0 tx=0).stop 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 -- 172.21.15.64:0/1109123041 shutdown_connections 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7bba680 0x5577b7bdaa40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bafdc0 0x5577b7bda500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 --2- 172.21.15.64:0/1109123041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7baf4a0 0x5577b7bb5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 -- 172.21.15.64:0/1109123041 >> 172.21.15.64:0/1109123041 conn(0x5577b7baba30 msgr2=0x5577b7bb93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 -- 172.21.15.64:0/1109123041 shutdown_connections 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.277+0000 7f5dea690740 1 -- 172.21.15.64:0/1109123041 wait complete. 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 -- start start 2024-09-19T05:06:36.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 Processor -- start 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bc9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 0x5577b7bc9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bba680 0x5577b7c3c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b2450 con 0x5577b7bafdc0 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b38c0 con 0x5577b7baf4a0 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01b2b20 con 0x5577b7bba680 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5dea690740 10 client.0 _ll_register_callbacks cb 0x7f5dd8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5de5852640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 0x5577b7bc9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5de5852640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 0x5577b7bc9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43852/0 (socket says 172.21.15.64:43852) 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5de5852640 1 -- 172.21.15.64:0/890476044 learned_addr learned my addr 172.21.15.64:0/890476044 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.278+0000 7f5de6053640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bc9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de6854640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bba680 0x5577b7c3c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de5852640 1 -- 172.21.15.64:0/890476044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bba680 msgr2=0x5577b7c3c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de5852640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bba680 0x5577b7c3c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de5852640 1 -- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 msgr2=0x5577b7bc9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de5852640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bc9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de5852640 1 -- 172.21.15.64:0/890476044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b7c3ca20 con 0x5577b7bafdc0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5de5852640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 0x5577b7bc9580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000f450 tx=0x7f5dd000f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85a4484bb2e5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5dc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/890476044 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd002c060 con 0x5577b7bafdc0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/890476044 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5dd00049d0 con 0x5577b7bafdc0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.279+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd0004c50 con 0x5577b7bafdc0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/890476044 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dd0004e00 con 0x5577b7bafdc0 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 10 client.15387 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577b7c3db90 con 0x5577b7bafdc0 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/890476044 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd0004c50 con 0x5577b7bafdc0 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/890476044 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5dd001e7e0 con 0x5577b7bafdc0 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 10 client.15387 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 20 client.15387 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7c401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 4 client.15387 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.280+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577b7b1cd60 con 0x5577b7bd27f0 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5de6053640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7c401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5de6053640 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7c401d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bad239a1946c564 server_cookie=8a81a5996139c47d in_seq=0 out_seq=0 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5dc9ffb640 10 client.15387 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:36.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5dc9ffb640 1 -- 172.21.15.64:0/890476044 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5577b7b1cd60 con 0x5577b7bd27f0 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5dc9ffb640 10 client.15387 handle_command_reply: tid=0 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5dea690740 1 client.15387 shutdown 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 msgr2=0x5577b7bc9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.281+0000 7f5dea690740 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 0x5577b7bc9580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000f450 tx=0x7f5dd000f480 comp rx=0 tx=0).stop 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 shutdown_connections 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577b7bba680 0x5577b7c3c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5577b7bd27f0 0x5577b7c401d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577b7bafdc0 0x5577b7bc9580 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 --2- 172.21.15.64:0/890476044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b7baf4a0 0x5577b7bc9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 >> 172.21.15.64:0/890476044 conn(0x5577b7baba30 msgr2=0x5577b7bb93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dc9ffb640 0 client.15387 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 shutdown_connections 2024-09-19T05:06:36.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 1 -- 172.21.15.64:0/890476044 wait complete. 2024-09-19T05:06:36.289 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:06:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:06:36.289 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:06:36.289 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5dea690740 20 client.15387 trim_cache size 0 max 16384 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5dcc0a3550 msgr2=0x7f5dcc0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5de82b7640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5dcc0a3550 0x7f5dcc0a5a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5577b7bac890 tx=0x7f5ddc005f90 comp rx=0 tx=0).stop 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0106f10 msgr2=0x7f5de006d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.282+0000 7f5de82b7640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0106f10 0x7f5de006d010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009ca0 tx=0x7f5dd8032ff0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 shutdown_connections 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5dcc0a3550 0x7f5dcc0a5a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de0106f10 0x7f5de006d010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de0102790 0x7f5de006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 --2- 172.21.15.64:0/4253278485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5de0101de0 0x7f5de0073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 >> 172.21.15.64:0/4253278485 conn(0x7f5de00fd8e0 msgr2=0x7f5de00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 shutdown_connections 2024-09-19T05:06:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:36.283+0000 7f5de82b7640 1 -- 172.21.15.64:0/4253278485 wait complete. 2024-09-19T05:06:37.076 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:06:37.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:06:37.364 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:37 smithi064 ceph-mon[30960]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:06:37.405 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:37.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:37 smithi103 ceph-mon[34446]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:06:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:37 smithi174 ceph-mon[34488]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:06:37.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.907+0000 7f59e92af640 1 -- 172.21.15.64:0/1123049698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 msgr2=0x7f59e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.907+0000 7f59e92af640 1 --2- 172.21.15.64:0/1123049698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 0x7f59e410ec40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009cd0 tx=0x7f59dc02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 -- 172.21.15.64:0/1123049698 shutdown_connections 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 --2- 172.21.15.64:0/1123049698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 0x7f59e410ec40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 --2- 172.21.15.64:0/1123049698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e41036e0 0x7f59e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 --2- 172.21.15.64:0/1123049698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e4102d30 0x7f59e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 -- 172.21.15.64:0/1123049698 >> 172.21.15.64:0/1123049698 conn(0x7f59e40fc7d0 msgr2=0x7f59e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 -- 172.21.15.64:0/1123049698 shutdown_connections 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.908+0000 7f59e92af640 1 -- 172.21.15.64:0/1123049698 wait complete. 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e92af640 1 Processor -- start 2024-09-19T05:06:37.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e92af640 1 -- start start 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e92af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e4102d30 0x7f59e4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e92af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 0x7f59e4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e92af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 0x7f59e419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e4102d30 0x7f59e4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 0x7f59e4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.909+0000 7f59e27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 0x7f59e4198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48214/0 (socket says 172.21.15.64:48214) 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 -- 172.21.15.64:0/831321253 learned_addr learned my addr 172.21.15.64:0/831321253 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4111fa0 con 0x7f59e41082a0 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4111e20 con 0x7f59e41036e0 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4112120 con 0x7f59e4102d30 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e37fe640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 0x7f59e419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:37.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 -- 172.21.15.64:0/831321253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e4102d30 msgr2=0x7f59e4198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e4102d30 0x7f59e4198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 -- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 msgr2=0x7f59e419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 0x7f59e419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e419d530 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.910+0000 7f59e27fc640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 0x7f59e4198ac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e9c0 tx=0x7f59d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.911+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d800ccc0 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.911+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e419d820 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.911+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d80054d0 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.911+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8010430 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.911+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e41a5100 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.912+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e4103110 con 0x7f59e41036e0 2024-09-19T05:06:37.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.914+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f59d8005020 con 0x7f59e41036e0 2024-09-19T05:06:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.915+0000 7f59cbfff640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59cc0a34d0 0x7f59cc0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.915+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f59d80d0550 con 0x7f59e41036e0 2024-09-19T05:06:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.915+0000 7f59e2ffd640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59cc0a34d0 0x7f59cc0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.916+0000 7f59e2ffd640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59cc0a34d0 0x7f59cc0a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009bd0 tx=0x7f59d4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:37.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:37.920+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59d809cce0 con 0x7f59e41036e0 2024-09-19T05:06:38.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.225+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59e410a2b0 con 0x7f59e41036e0 2024-09-19T05:06:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.226+0000 7f59cbfff640 1 -- 172.21.15.64:0/831321253 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f59d809c5e0 con 0x7f59e41036e0 2024-09-19T05:06:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:06:38.230 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59cc0a34d0 msgr2=0x7f59cc0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59cc0a34d0 0x7f59cc0a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009bd0 tx=0x7f59d4005f90 comp rx=0 tx=0).stop 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 msgr2=0x7f59e4198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 0x7f59e4198ac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e9c0 tx=0x7f59d800ee80 comp rx=0 tx=0).stop 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 shutdown_connections 2024-09-19T05:06:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59cc0a34d0 0x7f59cc0a5990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59e41082a0 0x7f59e419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59e41036e0 0x7f59e4198ac0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 --2- 172.21.15.64:0/831321253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59e4102d30 0x7f59e4198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.229+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 >> 172.21.15.64:0/831321253 conn(0x7f59e40fc7d0 msgr2=0x7f59e4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.230+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 shutdown_connections 2024-09-19T05:06:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:38.230+0000 7f59e92af640 1 -- 172.21.15.64:0/831321253 wait complete. 2024-09-19T05:06:38.575 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:38 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/831321253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:38.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:38 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/831321253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:38.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:38 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/831321253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:38.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:06:39.265 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:39.629 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:39 smithi064 ceph-mon[30960]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:39.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:39 smithi103 ceph-mon[34446]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 -- 172.21.15.64:0/1521750476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c101df0 msgr2=0x7f4b6c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 --2- 172.21.15.64:0/1521750476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c101df0 0x7f4b6c1021d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009900 tx=0x7f4b5c02a410 comp rx=0 tx=0).stop 2024-09-19T05:06:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 -- 172.21.15.64:0/1521750476 shutdown_connections 2024-09-19T05:06:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 --2- 172.21.15.64:0/1521750476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c106f20 0x7f4b6c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 --2- 172.21.15.64:0/1521750476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b6c1027a0 0x7f4b6c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 --2- 172.21.15.64:0/1521750476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c101df0 0x7f4b6c1021d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.723+0000 7f4b7219c640 1 -- 172.21.15.64:0/1521750476 >> 172.21.15.64:0/1521750476 conn(0x7f4b6c0fd8f0 msgr2=0x7f4b6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.724+0000 7f4b7219c640 1 -- 172.21.15.64:0/1521750476 shutdown_connections 2024-09-19T05:06:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.724+0000 7f4b7219c640 1 -- 172.21.15.64:0/1521750476 wait complete. 2024-09-19T05:06:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 Processor -- start 2024-09-19T05:06:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 -- start start 2024-09-19T05:06:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 0x7f4b6c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b6c1027a0 0x7f4b6c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c106f20 0x7f4b6c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c111e30 con 0x7f4b6c106f20 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c111cb0 con 0x7f4b6c101df0 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b7219c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c111fb0 con 0x7f4b6c1027a0 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 0x7f4b6c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 0x7f4b6c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48226/0 (socket says 172.21.15.64:48226) 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2972858998 learned_addr learned my addr 172.21.15.64:0/2972858998 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.725+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c106f20 0x7f4b6c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6affd640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b6c1027a0 0x7f4b6c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2972858998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b6c1027a0 msgr2=0x7f4b6c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b6c1027a0 0x7f4b6c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c106f20 msgr2=0x7f4b6c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c106f20 0x7f4b6c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2972858998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1a6830 con 0x7f4b6c101df0 2024-09-19T05:06:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c106f20 0x7f4b6c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 0x7f4b6c0744f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005940 tx=0x7f4b5c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.726+0000 7f4b68ff9640 1 -- 172.21.15.64:0/2972858998 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c03f030 con 0x7f4b6c101df0 2024-09-19T05:06:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.727+0000 7f4b68ff9640 1 -- 172.21.15.64:0/2972858998 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b5c003aa0 con 0x7f4b6c101df0 2024-09-19T05:06:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.727+0000 7f4b68ff9640 1 -- 172.21.15.64:0/2972858998 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c035070 con 0x7f4b6c101df0 2024-09-19T05:06:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.727+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1a6ac0 con 0x7f4b6c101df0 2024-09-19T05:06:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.727+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a6f50 con 0x7f4b6c101df0 2024-09-19T05:06:39.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.730+0000 7f4b68ff9640 1 -- 172.21.15.64:0/2972858998 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f4b5c042030 con 0x7f4b6c101df0 2024-09-19T05:06:39.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.730+0000 7f4b74575740 1 Processor -- start 2024-09-19T05:06:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.730+0000 7f4b74575740 1 -- start start 2024-09-19T05:06:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b74575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fcd48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b74575740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcd5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b74575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcdfb80 0x55b18fce1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b74575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b18fc322f0 con 0x55b18fcd6350 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b74575740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b18fbd9cf0 con 0x55b18fcd4de0 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b74575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b18fc2e2b0 con 0x55b18fcdfb80 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fcd48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fcd48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43884/0 (socket says 172.21.15.64:43884) 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 -- 172.21.15.64:0/4108234262 learned_addr learned my addr 172.21.15.64:0/4108234262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6affd640 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcdfb80 0x55b18fce1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcd5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b68ff9640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b540a34d0 0x7f4b540a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 -- 172.21.15.64:0/4108234262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcdfb80 msgr2=0x55b18fce1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcdfb80 0x55b18fce1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 -- 172.21.15.64:0/4108234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 msgr2=0x55b18fcd5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcd5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.731+0000 7f4b6bfff640 1 -- 172.21.15.64:0/4108234262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18fbbe3a0 con 0x55b18fcd6350 2024-09-19T05:06:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.732+0000 7f4b68ff9640 1 -- 172.21.15.64:0/2972858998 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4b5c0f03b0 con 0x7f4b6c101df0 2024-09-19T05:06:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.732+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b540a34d0 0x7f4b540a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.732+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fcd48a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400f790 tx=0x7f4b6400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaaba0fc6937be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.732+0000 7f4b51ffb640 1 -- 172.21.15.64:0/4108234262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400dd80 con 0x55b18fcd6350 2024-09-19T05:06:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.732+0000 7f4b51ffb640 1 -- 172.21.15.64:0/4108234262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b640108e0 con 0x55b18fcd6350 2024-09-19T05:06:39.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.732+0000 7f4b51ffb640 1 -- 172.21.15.64:0/4108234262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6401aaf0 con 0x55b18fcd6350 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.733+0000 7f4b74575740 1 -- 172.21.15.64:0/4108234262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 msgr2=0x55b18fcd48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.733+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b540a34d0 0x7f4b540a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b6401e000 tx=0x7f4b6401ad50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.733+0000 7f4b74575740 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fcd48a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400f790 tx=0x7f4b6400fc50 comp rx=0 tx=0).stop 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 -- 172.21.15.64:0/4108234262 shutdown_connections 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcdfb80 0x55b18fce1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcd5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 --2- 172.21.15.64:0/4108234262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fcd48a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 -- 172.21.15.64:0/4108234262 >> 172.21.15.64:0/4108234262 conn(0x55b18fc95310 msgr2=0x55b18fcd1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 -- 172.21.15.64:0/4108234262 shutdown_connections 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 -- 172.21.15.64:0/4108234262 wait complete. 2024-09-19T05:06:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 -- start start 2024-09-19T05:06:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.734+0000 7f4b74575740 1 Processor -- start 2024-09-19T05:06:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd4de0 0x55b18fcdf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcdc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fcdcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b18fce2880 con 0x55b18fcd4de0 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd4de0 0x55b18fcdf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b18fbed060 con 0x55b18fcdfb80 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcdc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b18fce2a00 con 0x55b18fcd6350 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcdc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:56100/0 (socket says 172.21.15.64:56100) 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6affd640 1 -- 172.21.15.64:0/266117644 learned_addr learned my addr 172.21.15.64:0/266117644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b74575740 10 client.0 _ll_register_callbacks cb 0x55b18f9408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fcdcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 msgr2=0x55b18fcdc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcdc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/266117644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 msgr2=0x55b18fcdcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fcdcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.735+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/266117644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18f9a33c0 con 0x55b18fcd4de0 2024-09-19T05:06:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd4de0 0x55b18fcdf4f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0ed920 tx=0x7f4b5c0bc280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dae390dfb907135 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:39.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b4affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b4affd640 1 -- 172.21.15.64:0/266117644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c049070 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b4affd640 1 -- 172.21.15.64:0/266117644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4b5c0b8b20 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b640108e0 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b4affd640 1 -- 172.21.15.64:0/266117644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b5c03b460 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b74575740 10 client.15405 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b18fcee2a0 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.736+0000 7f4b4affd640 1 -- 172.21.15.64:0/266117644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b640108e0 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b4affd640 1 -- 172.21.15.64:0/266117644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4b5c04c060 con 0x55b18fcd4de0 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b74575740 10 client.15405 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b74575740 10 client.15405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b74575740 20 client.15405 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b74575740 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fcf1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b74575740 4 client.15405 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18fc3f860 con 0x55b18fcef5f0 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.737+0000 7f4b6affd640 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fcf1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.738+0000 7f4b4affd640 10 client.15405 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.738+0000 7f4b6affd640 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fcf1a40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2be48d36a47ba1d server_cookie=2bda92fe86676959 in_seq=0 out_seq=0 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.738+0000 7f4b4affd640 1 -- 172.21.15.64:0/266117644 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b18fc3f860 con 0x55b18fcef5f0 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.738+0000 7f4b4affd640 10 client.15405 handle_command_reply: tid=0 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.738+0000 7f4b74575740 1 client.15405 shutdown 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd4de0 msgr2=0x55b18fcdf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd4de0 0x55b18fcdf4f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0ed920 tx=0x7f4b5c0bc280 comp rx=0 tx=0).stop 2024-09-19T05:06:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 shutdown_connections 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fcf1a40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b4affd640 0 client.15405 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fcdcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcdc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 --2- 172.21.15.64:0/266117644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd4de0 0x55b18fcdf4f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 >> 172.21.15.64:0/266117644 conn(0x55b18fc95310 msgr2=0x55b18fcd7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 shutdown_connections 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 1 -- 172.21.15.64:0/266117644 wait complete. 2024-09-19T05:06:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.739+0000 7f4b74575740 20 client.15405 trim_cache size 0 max 16384 2024-09-19T05:06:39.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 Processor -- start 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 -- start start 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcc7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcdfb80 0x55b18fcf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1adf10 con 0x55b18fcdfb80 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1ab7e0 con 0x55b18fcd4de0 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b74575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1a9630 con 0x55b18fcd6350 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcdfb80 0x55b18fcf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcdfb80 0x55b18fcf7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43902/0 (socket says 172.21.15.64:43902) 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b6bfff640 1 -- 172.21.15.64:0/3490074343 learned_addr learned my addr 172.21.15.64:0/3490074343 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:39.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.758+0000 7f4b6affd640 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcc7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/3490074343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 msgr2=0x55b18fcc7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcc7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/3490074343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcdfb80 msgr2=0x55b18fcf7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcdfb80 0x55b18fcf7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/3490074343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18fcd16e0 con 0x55b18fcd4de0 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcebb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005e00 tx=0x7f4b5c0ba9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9c79553754f3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b517fa640 1 -- 172.21.15.64:0/3490074343 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0d4070 con 0x55b18fcd4de0 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b517fa640 1 -- 172.21.15.64:0/3490074343 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b5c0f3e50 con 0x55b18fcd4de0 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.759+0000 7f4b517fa640 1 -- 172.21.15.64:0/3490074343 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c049070 con 0x55b18fcd4de0 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 -- 172.21.15.64:0/3490074343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 msgr2=0x55b18fcebb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcebb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005e00 tx=0x7f4b5c0ba9c0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 -- 172.21.15.64:0/3490074343 shutdown_connections 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcdfb80 0x55b18fcf7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd6350 0x55b18fcc7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 --2- 172.21.15.64:0/3490074343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcd4de0 0x55b18fcebb50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 -- 172.21.15.64:0/3490074343 >> 172.21.15.64:0/3490074343 conn(0x55b18fc95310 msgr2=0x55b18fcc7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 -- 172.21.15.64:0/3490074343 shutdown_connections 2024-09-19T05:06:39.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 -- 172.21.15.64:0/3490074343 wait complete. 2024-09-19T05:06:39.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.760+0000 7f4b74575740 1 -- start start 2024-09-19T05:06:39.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 Processor -- start 2024-09-19T05:06:39.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd4de0 0x55b18fce7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fce8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fce4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1a9ca0 con 0x55b18fcd6350 2024-09-19T05:06:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1ab110 con 0x55b18fcdfb80 2024-09-19T05:06:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1aa370 con 0x55b18fcd4de0 2024-09-19T05:06:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b6affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fce8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b74575740 10 client.0 _ll_register_callbacks cb 0x7f4b5c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b6b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd4de0 0x55b18fce7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b6affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fce8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43916/0 (socket says 172.21.15.64:43916) 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b6affd640 1 -- 172.21.15.64:0/2920298684 learned_addr learned my addr 172.21.15.64:0/2920298684 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.761+0000 7f4b6bfff640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fce4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2920298684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 msgr2=0x55b18fce4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fce4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2920298684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 msgr2=0x55b18fce8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fce8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6b7fe640 1 -- 172.21.15.64:0/2920298684 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18fce51d0 con 0x55b18fcd4de0 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6affd640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fce8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:39.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b6b7fe640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd4de0 0x55b18fce7cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005e00 tx=0x7f4b5c0bc280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8244a977ca937435 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b4affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b4affd640 1 -- 172.21.15.64:0/2920298684 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c049070 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b4affd640 1 -- 172.21.15.64:0/2920298684 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4b5c0035f0 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.762+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b5c0f3e50 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b4affd640 1 -- 172.21.15.64:0/2920298684 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b5c0d4070 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 10 client.24994 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b18fce51d0 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b4affd640 1 -- 172.21.15.64:0/2920298684 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b5c0f3e50 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b4affd640 1 -- 172.21.15.64:0/2920298684 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4b5c0edd00 con 0x55b18fcd4de0 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 10 client.24994 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 10 client.24994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 20 client.24994 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:39.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fd5cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 4 client.24994 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.763+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b18fc39d70 con 0x55b18fcef5f0 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.764+0000 7f4b6affd640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fd5cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.764+0000 7f4b6affd640 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fd5cc90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=841e7a3e495f62fc server_cookie=ccd1a082f9bed173 in_seq=0 out_seq=0 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.764+0000 7f4b4affd640 10 client.24994 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.764+0000 7f4b4affd640 1 -- 172.21.15.64:0/2920298684 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b18fc39d70 con 0x55b18fcef5f0 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.764+0000 7f4b4affd640 10 client.24994 handle_command_reply: tid=0 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.764+0000 7f4b74575740 1 client.24994 shutdown 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd4de0 msgr2=0x55b18fce7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd4de0 0x55b18fce7cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005e00 tx=0x7f4b5c0bc280 comp rx=0 tx=0).stop 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 shutdown_connections 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55b18fcef5f0 0x55b18fd5cc90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b18fcdfb80 0x55b18fce4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b4affd640 0 client.24994 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b18fcd6350 0x55b18fce8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 --2- 172.21.15.64:0/2920298684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18fcd4de0 0x55b18fce7cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 >> 172.21.15.64:0/2920298684 conn(0x55b18fc95310 msgr2=0x55b18fcc7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 shutdown_connections 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.765+0000 7f4b74575740 1 -- 172.21.15.64:0/2920298684 wait complete. 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.766+0000 7f4b74575740 20 client.24994 trim_cache size 0 max 16384 2024-09-19T05:06:39.771 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:06:39.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.766+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b540a34d0 msgr2=0x7f4b540a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.766+0000 7f4b7219c640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b540a34d0 0x7f4b540a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b6401e000 tx=0x7f4b6401ad50 comp rx=0 tx=0).stop 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.766+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 msgr2=0x7f4b6c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.766+0000 7f4b7219c640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 0x7f4b6c0744f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005940 tx=0x7f4b5c004270 comp rx=0 tx=0).stop 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 shutdown_connections 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b540a34d0 0x7f4b540a5990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b6c106f20 0x7f4b6c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b6c1027a0 0x7f4b6c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 --2- 172.21.15.64:0/2972858998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c101df0 0x7f4b6c0744f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 >> 172.21.15.64:0/2972858998 conn(0x7f4b6c0fd8f0 msgr2=0x7f4b6c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 shutdown_connections 2024-09-19T05:06:39.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:39.767+0000 7f4b7219c640 1 -- 172.21.15.64:0/2972858998 wait complete. 2024-09-19T05:06:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:39 smithi174 ceph-mon[34488]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:40.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:06:40.856 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 -- 172.21.15.64:0/1752108606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664102d30 msgr2=0x7f2664103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 --2- 172.21.15.64:0/1752108606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664102d30 0x7f2664103110 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2654009930 tx=0x7f265402a420 comp rx=0 tx=0).stop 2024-09-19T05:06:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 -- 172.21.15.64:0/1752108606 shutdown_connections 2024-09-19T05:06:41.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 --2- 172.21.15.64:0/1752108606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641082a0 0x7f266410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 --2- 172.21.15.64:0/1752108606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641036e0 0x7f2664107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 --2- 172.21.15.64:0/1752108606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664102d30 0x7f2664103110 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 -- 172.21.15.64:0/1752108606 >> 172.21.15.64:0/1752108606 conn(0x7f26640fc7d0 msgr2=0x7f26640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 -- 172.21.15.64:0/1752108606 shutdown_connections 2024-09-19T05:06:41.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.443+0000 7f266a570640 1 -- 172.21.15.64:0/1752108606 wait complete. 2024-09-19T05:06:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.444+0000 7f266a570640 1 Processor -- start 2024-09-19T05:06:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 -- start start 2024-09-19T05:06:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641036e0 0x7f2664100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 0x7f2664100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664101410 0x7f26641a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2664111f50 con 0x7f2664101410 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2664111dd0 con 0x7f26641082a0 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f266a570640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26641120d0 con 0x7f26641036e0 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f26637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 0x7f2664100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f2668b0d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664101410 0x7f26641a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f26637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 0x7f2664100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48300/0 (socket says 172.21.15.64:48300) 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f26637fe640 1 -- 172.21.15.64:0/2531552558 learned_addr learned my addr 172.21.15.64:0/2531552558 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.445+0000 7f2663fff640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641036e0 0x7f2664100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26637fe640 1 -- 172.21.15.64:0/2531552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641036e0 msgr2=0x7f2664100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26637fe640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641036e0 0x7f2664100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26637fe640 1 -- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664101410 msgr2=0x7f26641a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26637fe640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664101410 0x7f26641a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26637fe640 1 -- 172.21.15.64:0/2531552558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641a6f70 con 0x7f26641082a0 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f2668b0d640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664101410 0x7f26641a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f2663fff640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641036e0 0x7f2664100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26637fe640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 0x7f2664100ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f265800d770 tx=0x7f265800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26617fa640 1 -- 172.21.15.64:0/2531552558 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658017070 con 0x7f26641082a0 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26617fa640 1 -- 172.21.15.64:0/2531552558 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f265800de80 con 0x7f26641082a0 2024-09-19T05:06:41.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f26617fa640 1 -- 172.21.15.64:0/2531552558 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265800f620 con 0x7f26641082a0 2024-09-19T05:06:41.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.446+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641a7260 con 0x7f26641082a0 2024-09-19T05:06:41.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.447+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641a7690 con 0x7f26641082a0 2024-09-19T05:06:41.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.450+0000 7f26617fa640 1 -- 172.21.15.64:0/2531552558 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f265801a070 con 0x7f26641082a0 2024-09-19T05:06:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.450+0000 7f266c949740 1 Processor -- start 2024-09-19T05:06:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.450+0000 7f266c949740 1 -- start start 2024-09-19T05:06:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.450+0000 7f266c949740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b023d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f266c949740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b023da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f266c949740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b0248370 0x55d4b024a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f266c949740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4b01a32f0 con 0x55d4b023d5d0 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f266c949740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d4b014acf0 con 0x55d4b023eb40 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f266c949740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4b019f2b0 con 0x55d4b0248370 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2663fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b023da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2663fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b023da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43934/0 (socket says 172.21.15.64:43934) 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2663fff640 1 -- 172.21.15.64:0/3539131786 learned_addr learned my addr 172.21.15.64:0/3539131786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2668b0d640 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b023d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2668b0d640 1 -- 172.21.15.64:0/3539131786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b0248370 msgr2=0x55d4b024a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2668b0d640 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b0248370 0x55d4b024a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2668b0d640 1 -- 172.21.15.64:0/3539131786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 msgr2=0x55d4b023da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.451+0000 7f2668b0d640 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b023da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.452+0000 7f2668b0d640 1 -- 172.21.15.64:0/3539131786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b012f3a0 con 0x55d4b023eb40 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.452+0000 7f2663fff640 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b023da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.452+0000 7f26617fa640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f264c0a35d0 0x7f264c0a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.452+0000 7f26617fa640 1 -- 172.21.15.64:0/2531552558 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f26580cf9c0 con 0x7f26641082a0 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.452+0000 7f2668b0d640 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b023d090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f265c010710 tx=0x7f265c010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5281f59c5cf177 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.453+0000 7f2663fff640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f264c0a35d0 0x7f264c0a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.453+0000 7f264a7fc640 1 -- 172.21.15.64:0/3539131786 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c004b20 con 0x55d4b023eb40 2024-09-19T05:06:41.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.453+0000 7f264a7fc640 1 -- 172.21.15.64:0/3539131786 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f265c00ed80 con 0x55d4b023eb40 2024-09-19T05:06:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.453+0000 7f264a7fc640 1 -- 172.21.15.64:0/3539131786 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c005600 con 0x55d4b023eb40 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 -- 172.21.15.64:0/3539131786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 msgr2=0x55d4b023d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f2663fff640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f264c0a35d0 0x7f264c0a5a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2654031440 tx=0x7f2654005a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b023d090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f265c010710 tx=0x7f265c010bd0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 -- 172.21.15.64:0/3539131786 shutdown_connections 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b0248370 0x55d4b024a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b023da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 --2- 172.21.15.64:0/3539131786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b023d090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 -- 172.21.15.64:0/3539131786 >> 172.21.15.64:0/3539131786 conn(0x55d4b0206310 msgr2=0x55d4b0239c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 -- 172.21.15.64:0/3539131786 shutdown_connections 2024-09-19T05:06:41.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.454+0000 7f266c949740 1 -- 172.21.15.64:0/3539131786 wait complete. 2024-09-19T05:06:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.455+0000 7f266c949740 1 -- start start 2024-09-19T05:06:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.456+0000 7f266c949740 1 Processor -- start 2024-09-19T05:06:41.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.456+0000 7f266c949740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 0x55d4b0247de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.456+0000 7f266c949740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0245090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.456+0000 7f266c949740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b02457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.456+0000 7f2663fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 0x55d4b0247de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 0x55d4b0247de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48316/0 (socket says 172.21.15.64:48316) 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 -- 172.21.15.64:0/3042297946 learned_addr learned my addr 172.21.15.64:0/3042297946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4b024b070 con 0x55d4b0248370 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d4b015e060 con 0x55d4b023d5d0 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4b024b1f0 con 0x55d4b023eb40 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2668b0d640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b02457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f266c949740 10 client.0 _ll_register_callbacks cb 0x55d4afeb18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f26637fe640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0245090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 -- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 msgr2=0x55d4b0245090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0245090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 -- 172.21.15.64:0/3042297946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 msgr2=0x55d4b02457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b02457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.457+0000 7f2663fff640 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4aff143c0 con 0x55d4b023d5d0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f26637fe640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0245090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f2668b0d640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b02457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f2663fff640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 0x55d4b0247de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2654032fd0 tx=0x7f265402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673ed31dfe3312d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f26437fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f26437fe640 1 -- 172.21.15.64:0/3042297946 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265404c070 con 0x55d4b023d5d0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f26437fe640 1 -- 172.21.15.64:0/3042297946 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f26540035f0 con 0x55d4b023d5d0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265c00ed80 con 0x55d4b023d5d0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.458+0000 7f26437fe640 1 -- 172.21.15.64:0/3042297946 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2654058050 con 0x55d4b023d5d0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f266c949740 10 client.25139 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4b0256b60 con 0x55d4b023d5d0 2024-09-19T05:06:41.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f26437fe640 1 -- 172.21.15.64:0/3042297946 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f265c00ed80 con 0x55d4b023d5d0 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f26437fe640 1 -- 172.21.15.64:0/3042297946 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f265403c7d0 con 0x55d4b023d5d0 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f266c949740 10 client.25139 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f266c949740 10 client.25139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.459+0000 7f266c949740 20 client.25139 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.460+0000 7f266c949740 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b025a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.460+0000 7f266c949740 4 client.25139 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:41.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.460+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4b01b0860 con 0x55d4b0257f70 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.460+0000 7f26637fe640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b025a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.461+0000 7f26437fe640 10 client.25139 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.461+0000 7f26637fe640 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b025a3c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be4b7d17a30b8c server_cookie=b8344d3b82a68fd6 in_seq=0 out_seq=0 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.461+0000 7f26437fe640 1 -- 172.21.15.64:0/3042297946 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4b01b0860 con 0x55d4b0257f70 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.461+0000 7f26437fe640 10 client.25139 handle_command_reply: tid=0 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.461+0000 7f266c949740 1 client.25139 shutdown 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 msgr2=0x55d4b0247de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 0x55d4b0247de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2654032fd0 tx=0x7f265402a420 comp rx=0 tx=0).stop 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 shutdown_connections 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b025a3c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b02457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f26437fe640 0 client.25139 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:06:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0245090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 --2- 172.21.15.64:0/3042297946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023d5d0 0x55d4b0247de0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 >> 172.21.15.64:0/3042297946 conn(0x55d4b0206310 msgr2=0x55d4b023f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 shutdown_connections 2024-09-19T05:06:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.462+0000 7f266c949740 1 -- 172.21.15.64:0/3042297946 wait complete. 2024-09-19T05:06:41.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.463+0000 7f266c949740 20 client.25139 trim_cache size 0 max 16384 2024-09-19T05:06:41.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 Processor -- start 2024-09-19T05:06:41.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 -- start start 2024-09-19T05:06:41.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:41 smithi064 ceph-mon[30960]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:41.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023d5d0 0x55d4b0254310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b0244180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b0240120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26641b28f0 con 0x55d4b0248370 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26641b01c0 con 0x55d4b023eb40 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.489+0000 7f266c949740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26641ae010 con 0x55d4b023d5d0 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f2663fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023d5d0 0x55d4b0254310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f2663fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023d5d0 0x55d4b0254310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:56164/0 (socket says 172.21.15.64:56164) 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f2663fff640 1 -- 172.21.15.64:0/931582354 learned_addr learned my addr 172.21.15.64:0/931582354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b0244180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 -- 172.21.15.64:0/931582354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023d5d0 msgr2=0x55d4b0254310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023d5d0 0x55d4b0254310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 -- 172.21.15.64:0/931582354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 msgr2=0x55d4b0240120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b0240120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 -- 172.21.15.64:0/931582354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b0238310 con 0x55d4b023eb40 2024-09-19T05:06:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.490+0000 7f26637fe640 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b0244180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26580ccf30 tx=0x7f2658010f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00e83b9319d7670 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.491+0000 7f2649ffb640 1 -- 172.21.15.64:0/931582354 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580b4020 con 0x55d4b023eb40 2024-09-19T05:06:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.491+0000 7f2649ffb640 1 -- 172.21.15.64:0/931582354 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26580a4660 con 0x55d4b023eb40 2024-09-19T05:06:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.491+0000 7f2649ffb640 1 -- 172.21.15.64:0/931582354 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658021040 con 0x55d4b023eb40 2024-09-19T05:06:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 -- 172.21.15.64:0/931582354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 msgr2=0x55d4b0244180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b0244180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26580ccf30 tx=0x7f2658010f70 comp rx=0 tx=0).stop 2024-09-19T05:06:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 -- 172.21.15.64:0/931582354 shutdown_connections 2024-09-19T05:06:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b0248370 0x55d4b0240120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b023eb40 0x55d4b0244180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 --2- 172.21.15.64:0/931582354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023d5d0 0x55d4b0254310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 -- 172.21.15.64:0/931582354 >> 172.21.15.64:0/931582354 conn(0x55d4b0206310 msgr2=0x55d4b0234a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 -- 172.21.15.64:0/931582354 shutdown_connections 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.492+0000 7f266c949740 1 -- 172.21.15.64:0/931582354 wait complete. 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 -- start start 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 Processor -- start 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b02507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0250d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 0x55d4b024bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26641ae680 con 0x55d4b023d5d0 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26641afaf0 con 0x55d4b0248370 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.493+0000 7f266c949740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26641aed50 con 0x55d4b023eb40 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f266c949740 10 client.0 _ll_register_callbacks cb 0x7f2654040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 0x55d4b024bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 0x55d4b024bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48326/0 (socket says 172.21.15.64:48326) 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 -- 172.21.15.64:0/358421451 learned_addr learned my addr 172.21.15.64:0/358421451 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f26637fe640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0250d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2663fff640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b02507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 -- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 msgr2=0x55d4b0250d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0250d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 -- 172.21.15.64:0/358421451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 msgr2=0x55d4b02507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b02507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2668b0d640 1 -- 172.21.15.64:0/358421451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b0239510 con 0x55d4b0248370 2024-09-19T05:06:41.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.494+0000 7f2663fff640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b02507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:41.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f2668b0d640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 0x55d4b024bd70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f265c00f010 tx=0x7f265c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb3f31a016e5873 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f26437fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f26437fe640 1 -- 172.21.15.64:0/358421451 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c005040 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f26437fe640 1 -- 172.21.15.64:0/358421451 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f265c0051e0 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26580a4660 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f26437fe640 1 -- 172.21.15.64:0/358421451 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f265c0053e0 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f266c949740 10 client.25145 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4b02caf20 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f26437fe640 1 -- 172.21.15.64:0/358421451 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f265c012450 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f26437fe640 1 -- 172.21.15.64:0/358421451 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f265c00f040 con 0x55d4b0248370 2024-09-19T05:06:41.506 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f266c949740 10 client.25145 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f266c949740 10 client.25145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.495+0000 7f266c949740 20 client.25145 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.496+0000 7f266c949740 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b02cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.496+0000 7f266c949740 4 client.25145 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.496+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f26580a4660 con 0x55d4b0257f70 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.496+0000 7f2663fff640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b02cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.496+0000 7f26437fe640 10 client.25145 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.496+0000 7f2663fff640 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b02cd370 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=403ce3fb3a44a00b server_cookie=9bd0324413ea00ac in_seq=0 out_seq=0 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f26437fe640 1 -- 172.21.15.64:0/358421451 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f26580a4660 con 0x55d4b0257f70 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f26437fe640 10 client.25145 handle_command_reply: tid=0 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 client.25145 shutdown 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 msgr2=0x55d4b024bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 0x55d4b024bd70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f265c00f010 tx=0x7f265c005df0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 shutdown_connections 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4b0248370 0x55d4b024bd70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4b023eb40 0x55d4b0250d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55d4b0257f70 0x55d4b02cd370 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 --2- 172.21.15.64:0/358421451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4b023d5d0 0x55d4b02507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f26437fe640 0 client.25145 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 >> 172.21.15.64:0/358421451 conn(0x55d4b0206310 msgr2=0x55d4b0244180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 shutdown_connections 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 1 -- 172.21.15.64:0/358421451 wait complete. 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.497+0000 7f266c949740 20 client.25145 trim_cache size 0 max 16384 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.498+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f264c0a35d0 msgr2=0x7f264c0a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.498+0000 7f266a570640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f264c0a35d0 0x7f264c0a5a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2654031440 tx=0x7f2654005a40 comp rx=0 tx=0).stop 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.498+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 msgr2=0x7f2664100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.498+0000 7f266a570640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 0x7f2664100ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f265800d770 tx=0x7f265800dc30 comp rx=0 tx=0).stop 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 shutdown_connections 2024-09-19T05:06:41.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f264c0a35d0 0x7f264c0a5a90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2664101410 0x7f26641a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26641082a0 0x7f2664100ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 --2- 172.21.15.64:0/2531552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26641036e0 0x7f2664100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 >> 172.21.15.64:0/2531552558 conn(0x7f26640fc7d0 msgr2=0x7f2664105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 shutdown_connections 2024-09-19T05:06:41.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:41.499+0000 7f266a570640 1 -- 172.21.15.64:0/2531552558 wait complete. 2024-09-19T05:06:41.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:41 smithi103 ceph-mon[34446]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:41 smithi174 ceph-mon[34488]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:42.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:06:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:42.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 -- 172.21.15.64:0/1508767810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 msgr2=0x7f9d3010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 --2- 172.21.15.64:0/1508767810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 0x7f9d3010da40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009ca0 tx=0x7f9d2802a950 comp rx=0 tx=0).stop 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 -- 172.21.15.64:0/1508767810 shutdown_connections 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 --2- 172.21.15.64:0/1508767810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 0x7f9d3010da40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 --2- 172.21.15.64:0/1508767810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 --2- 172.21.15.64:0/1508767810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d30100e20 0x7f9d30101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 -- 172.21.15.64:0/1508767810 >> 172.21.15.64:0/1508767810 conn(0x7f9d300fc7d0 msgr2=0x7f9d300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 -- 172.21.15.64:0/1508767810 shutdown_connections 2024-09-19T05:06:42.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.942+0000 7f9d374a1640 1 -- 172.21.15.64:0/1508767810 wait complete. 2024-09-19T05:06:42.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 Processor -- start 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 -- start start 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d30100e20 0x7f9d301983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 0x7f9d3019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d30110e30 con 0x7f9d30102ef0 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d30110cb0 con 0x7f9d30100e20 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.943+0000 7f9d374a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d30110fb0 con 0x7f9d30102570 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:56182/0 (socket says 172.21.15.64:56182) 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 -- 172.21.15.64:0/3514338637 learned_addr learned my addr 172.21.15.64:0/3514338637 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:42.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d3523d640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d30100e20 0x7f9d301983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 0x7f9d3019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 -- 172.21.15.64:0/3514338637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d30100e20 msgr2=0x7f9d301983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d30100e20 0x7f9d301983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 -- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 msgr2=0x7f9d3019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 0x7f9d3019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.944+0000 7f9d34a3c640 1 -- 172.21.15.64:0/3514338637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3019d360 con 0x7f9d30102570 2024-09-19T05:06:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.945+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301988f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000d770 tx=0x7f9d2000db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.945+0000 7f9d267fc640 1 -- 172.21.15.64:0/3514338637 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20017070 con 0x7f9d30102570 2024-09-19T05:06:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.945+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3019d650 con 0x7f9d30102570 2024-09-19T05:06:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.945+0000 7f9d267fc640 1 -- 172.21.15.64:0/3514338637 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d20009820 con 0x7f9d30102570 2024-09-19T05:06:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.945+0000 7f9d267fc640 1 -- 172.21.15.64:0/3514338637 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20004c60 con 0x7f9d30102570 2024-09-19T05:06:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.945+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d300fcbb0 con 0x7f9d30102570 2024-09-19T05:06:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 Processor -- start 2024-09-19T05:06:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 -- start start 2024-09-19T05:06:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db135d0 0x55601db13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db26b80 0x55601db28f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55601da792f0 con 0x55601db135d0 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55601da20cf0 con 0x55601db14b40 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.948+0000 7f9d3987a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55601da752b0 con 0x55601db26b80 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db13090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48342/0 (socket says 172.21.15.64:48342) 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 -- 172.21.15.64:0/45401823 learned_addr learned my addr 172.21.15.64:0/45401823 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d3523d640 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db135d0 0x55601db13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 -- 172.21.15.64:0/45401823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db26b80 msgr2=0x55601db28f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:06:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db26b80 0x55601db28f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 -- 172.21.15.64:0/45401823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db135d0 msgr2=0x55601db13a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db135d0 0x55601db13a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d35a3e640 1 -- 172.21.15.64:0/45401823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601da053a0 con 0x55601db14b40 2024-09-19T05:06:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.949+0000 7f9d3523d640 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db135d0 0x55601db13a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.950+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db13090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d28035000 tx=0x7f9d2802fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f065c46b9117030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.950+0000 7f9d267fc640 1 -- 172.21.15.64:0/3514338637 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f9d200099d0 con 0x7f9d30102570 2024-09-19T05:06:42.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.950+0000 7f9d1b7fe640 1 -- 172.21.15.64:0/45401823 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28048070 con 0x55601db14b40 2024-09-19T05:06:42.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.950+0000 7f9d1b7fe640 1 -- 172.21.15.64:0/45401823 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d2802fa00 con 0x55601db14b40 2024-09-19T05:06:42.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.950+0000 7f9d1b7fe640 1 -- 172.21.15.64:0/45401823 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28034040 con 0x55601db14b40 2024-09-19T05:06:42.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d267fc640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d1c0a3550 0x7f9d1c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 -- 172.21.15.64:0/45401823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 msgr2=0x55601db13090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db13090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d28035000 tx=0x7f9d2802fe40 comp rx=0 tx=0).stop 2024-09-19T05:06:42.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 -- 172.21.15.64:0/45401823 shutdown_connections 2024-09-19T05:06:42.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db26b80 0x55601db28f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db135d0 0x55601db13a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 --2- 172.21.15.64:0/45401823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db13090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 -- 172.21.15.64:0/45401823 >> 172.21.15.64:0/45401823 conn(0x55601dadc310 msgr2=0x55601db0fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3523d640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d1c0a3550 0x7f9d1c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 -- 172.21.15.64:0/45401823 shutdown_connections 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d3987a740 1 -- 172.21.15.64:0/45401823 wait complete. 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.951+0000 7f9d267fc640 1 -- 172.21.15.64:0/3514338637 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f9d200cf9a0 con 0x7f9d30102570 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 -- start start 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3523d640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d1c0a3550 0x7f9d1c0a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55601db10ac0 tx=0x7f9d2c00a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 Processor -- start 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db23870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55601db29820 con 0x55601db26b80 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55601da34060 con 0x55601db14b40 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55601db299a0 con 0x55601db135d0 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.952+0000 7f9d3987a740 10 client.0 _ll_register_callbacks cb 0x55601d7878c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d35a3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d34a3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db23870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d34a3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db23870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48350/0 (socket says 172.21.15.64:48350) 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d34a3c640 1 -- 172.21.15.64:0/1159863766 learned_addr learned my addr 172.21.15.64:0/1159863766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 -- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 msgr2=0x55601db23870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db23870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 -- 172.21.15.64:0/1159863766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 msgr2=0x55601db24000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db24000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 -- 172.21.15.64:0/1159863766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601d7ea3c0 con 0x55601db135d0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db23870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d3523d640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db265c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0128e0 tx=0x7f9d2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bdd95ec51ee6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.953+0000 7f9d18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d18ff9640 1 -- 172.21.15.64:0/1159863766 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01de10 con 0x55601db135d0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d18ff9640 1 -- 172.21.15.64:0/1159863766 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f9d2c005020 con 0x55601db135d0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d2802fa00 con 0x55601db135d0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d18ff9640 1 -- 172.21.15.64:0/1159863766 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d2c005220 con 0x55601db135d0 2024-09-19T05:06:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 10 client.25012 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55601db35370 con 0x55601db135d0 2024-09-19T05:06:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d18ff9640 1 -- 172.21.15.64:0/1159863766 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d2c004c50 con 0x55601db135d0 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d18ff9640 1 -- 172.21.15.64:0/1159863766 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d2c019040 con 0x55601db135d0 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 10 client.25012 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 10 client.25012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 20 client.25012 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.954+0000 7f9d3987a740 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601db36830 0x55601db38c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.955+0000 7f9d3987a740 4 client.25012 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.955+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9d2802fa00 con 0x55601db36830 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.955+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601db36830 0x55601db38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.955+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601db36830 0x55601db38c50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd0f850797956f4 server_cookie=6f0ddaac30e8f3e0 in_seq=0 out_seq=0 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.955+0000 7f9d18ff9640 10 client.25012 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d18ff9640 1 -- 172.21.15.64:0/1159863766 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9d2802fa00 con 0x55601db36830 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d18ff9640 10 client.25012 handle_command_reply: tid=0 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 client.25012 shutdown 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 msgr2=0x55601db265c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db265c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0128e0 tx=0x7f9d2c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 shutdown_connections 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601db36830 0x55601db38c50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db14b40 0x55601db23870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d18ff9640 0 client.25012 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 --2- 172.21.15.64:0/1159863766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db265c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.956+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 >> 172.21.15.64:0/1159863766 conn(0x55601dadc310 msgr2=0x55601db15930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.957+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 shutdown_connections 2024-09-19T05:06:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.957+0000 7f9d3987a740 1 -- 172.21.15.64:0/1159863766 wait complete. 2024-09-19T05:06:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.957+0000 7f9d3987a740 20 client.25012 trim_cache size 0 max 16384 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.970+0000 7f9d3987a740 1 Processor -- start 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.970+0000 7f9d3987a740 1 -- start start 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.970+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 0x55601db139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.970+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db14b40 0x55601db0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.970+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.970+0000 7f9d3987a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301ade50 con 0x55601db26b80 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 0x55601db139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3987a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301ab720 con 0x55601db135d0 2024-09-19T05:06:42.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3987a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301a9570 con 0x55601db14b40 2024-09-19T05:06:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d35a3e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 0x55601db139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48362/0 (socket says 172.21.15.64:48362) 2024-09-19T05:06:42.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 -- 172.21.15.64:0/69363970 learned_addr learned my addr 172.21.15.64:0/69363970 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db14b40 0x55601db0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 -- 172.21.15.64:0/69363970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db14b40 msgr2=0x55601db0f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db14b40 0x55601db0f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 -- 172.21.15.64:0/69363970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 msgr2=0x55601db0fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db0fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 -- 172.21.15.64:0/69363970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601db24370 con 0x55601db135d0 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db0fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.971+0000 7f9d3523d640 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 0x55601db139b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c010ea0 tx=0x7f9d2c01bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf4fcc83c2e1800 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.972+0000 7f9d1affd640 1 -- 172.21.15.64:0/69363970 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01b670 con 0x55601db135d0 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.972+0000 7f9d1affd640 1 -- 172.21.15.64:0/69363970 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d2c004be0 con 0x55601db135d0 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.972+0000 7f9d1affd640 1 -- 172.21.15.64:0/69363970 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c02e830 con 0x55601db135d0 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.972+0000 7f9d3987a740 1 -- 172.21.15.64:0/69363970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 msgr2=0x55601db139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 0x55601db139b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c010ea0 tx=0x7f9d2c01bba0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 -- 172.21.15.64:0/69363970 shutdown_connections 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db26b80 0x55601db0fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db14b40 0x55601db0f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 --2- 172.21.15.64:0/69363970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db135d0 0x55601db139b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 -- 172.21.15.64:0/69363970 >> 172.21.15.64:0/69363970 conn(0x55601dadc310 msgr2=0x55601db0aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 -- 172.21.15.64:0/69363970 shutdown_connections 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 -- 172.21.15.64:0/69363970 wait complete. 2024-09-19T05:06:42.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 -- start start 2024-09-19T05:06:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.973+0000 7f9d3987a740 1 Processor -- start 2024-09-19T05:06:42.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db2a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 0x55601db2a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db26b80 0x55601db994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301a9be0 con 0x55601db14b40 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301ab050 con 0x55601db26b80 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301aa2b0 con 0x55601db135d0 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3987a740 10 client.0 _ll_register_callbacks cb 0x7f9d2c00a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d34a3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 0x55601db2a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d34a3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 0x55601db2a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44002/0 (socket says 172.21.15.64:44002) 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d34a3c640 1 -- 172.21.15.64:0/2983640717 learned_addr learned my addr 172.21.15.64:0/2983640717 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db26b80 0x55601db994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.974+0000 7f9d3523d640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db2a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d35a3e640 1 -- 172.21.15.64:0/2983640717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 msgr2=0x55601db2a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db2a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d35a3e640 1 -- 172.21.15.64:0/2983640717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 msgr2=0x55601db2a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 0x55601db2a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d35a3e640 1 -- 172.21.15.64:0/2983640717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601db99ad0 con 0x55601db26b80 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d34a3c640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 0x55601db2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d3523d640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:42.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d35a3e640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db26b80 0x55601db994e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009ca0 tx=0x7f9d2802f5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=866dbb6d02a349f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d18ff9640 1 -- 172.21.15.64:0/2983640717 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28048040 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d18ff9640 1 -- 172.21.15.64:0/2983640717 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f9d2802f640 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.975+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d2c004be0 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.976+0000 7f9d18ff9640 1 -- 172.21.15.64:0/2983640717 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d2805a050 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.976+0000 7f9d3987a740 10 client.25160 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.976+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55601dba0e30 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.976+0000 7f9d18ff9640 1 -- 172.21.15.64:0/2983640717 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d28051070 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.976+0000 7f9d18ff9640 1 -- 172.21.15.64:0/2983640717 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d2804b070 con 0x55601db26b80 2024-09-19T05:06:42.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3987a740 10 client.25160 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3987a740 10 client.25160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3987a740 20 client.25160 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3987a740 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601dba1180 0x55601dba35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3987a740 4 client.25160 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9d2c004be0 con 0x55601dba1180 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.977+0000 7f9d3523d640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601dba1180 0x55601dba35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d18ff9640 10 client.25160 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3523d640 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601dba1180 0x55601dba35a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f050b428c6d7dd server_cookie=849f46f1575a283e in_seq=0 out_seq=0 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d18ff9640 1 -- 172.21.15.64:0/2983640717 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9d2c004be0 con 0x55601dba1180 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d18ff9640 10 client.25160 handle_command_reply: tid=0 2024-09-19T05:06:42.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 client.25160 shutdown 2024-09-19T05:06:42.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db26b80 msgr2=0x55601db994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db26b80 0x55601db994e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009ca0 tx=0x7f9d2802f5d0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 shutdown_connections 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55601db26b80 0x55601db994e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55601dba1180 0x55601dba35a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601db14b40 0x55601db2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 --2- 172.21.15.64:0/2983640717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55601db135d0 0x55601db2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d18ff9640 0 client.25160 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.978+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 >> 172.21.15.64:0/2983640717 conn(0x55601dadc310 msgr2=0x55601db22050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 shutdown_connections 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d3987a740 1 -- 172.21.15.64:0/2983640717 wait complete. 2024-09-19T05:06:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d3987a740 20 client.25160 trim_cache size 0 max 16384 2024-09-19T05:06:42.985 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:06:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d1c0a3550 msgr2=0x7f9d1c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d1c0a3550 0x7f9d1c0a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55601db10ac0 tx=0x7f9d2c00a2d0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 msgr2=0x7f9d301988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301988f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000d770 tx=0x7f9d2000db70 comp rx=0 tx=0).stop 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 shutdown_connections 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d1c0a3550 0x7f9d1c0a5a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d30102ef0 0x7f9d3019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d30102570 0x7f9d301988f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 --2- 172.21.15.64:0/3514338637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d30100e20 0x7f9d301983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.979+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 >> 172.21.15.64:0/3514338637 conn(0x7f9d300fc7d0 msgr2=0x7f9d3006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.980+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 shutdown_connections 2024-09-19T05:06:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:42.980+0000 7f9d374a1640 1 -- 172.21.15.64:0/3514338637 wait complete. 2024-09-19T05:06:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:43 smithi064 ceph-mon[30960]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:06:43.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:43 smithi103 ceph-mon[34446]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:06:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:43 smithi174 ceph-mon[34488]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:06:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 -- 172.21.15.64:0/1950751193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 msgr2=0x7f0c9810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/1950751193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c9810ba80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009900 tx=0x7f0c8802a480 comp rx=0 tx=0).stop 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 -- 172.21.15.64:0/1950751193 shutdown_connections 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/1950751193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 0x7f0c9810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/1950751193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c9810ba80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/1950751193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 0x7f0c98103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 -- 172.21.15.64:0/1950751193 >> 172.21.15.64:0/1950751193 conn(0x7f0c98077990 msgr2=0x7f0c980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 -- 172.21.15.64:0/1950751193 shutdown_connections 2024-09-19T05:06:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.353+0000 7f0c9f2af640 1 -- 172.21.15.64:0/1950751193 wait complete. 2024-09-19T05:06:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 Processor -- start 2024-09-19T05:06:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 -- start start 2024-09-19T05:06:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 0x7f0c981982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c981987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c981987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c981987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52502/0 (socket says 172.21.15.64:52502) 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 0x7f0c9819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9c84a640 1 -- 172.21.15.64:0/376837245 learned_addr learned my addr 172.21.15.64:0/376837245 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98111d60 con 0x7f0c981035a0 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98111be0 con 0x7f0c9810c0d0 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98111ee0 con 0x7f0c98102c80 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 0x7f0c981982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.354+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 0x7f0c9819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9c84a640 1 -- 172.21.15.64:0/376837245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 msgr2=0x7f0c981982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 0x7f0c981982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9c84a640 1 -- 172.21.15.64:0/376837245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 msgr2=0x7f0c9819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 0x7f0c9819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9c84a640 1 -- 172.21.15.64:0/376837245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9819d2a0 con 0x7f0c981035a0 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 0x7f0c9819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 0x7f0c981982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c981987f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005940 tx=0x7f0c88031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c8e7fc640 1 -- 172.21.15.64:0/376837245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8803f030 con 0x7f0c981035a0 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c8e7fc640 1 -- 172.21.15.64:0/376837245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c880043d0 con 0x7f0c981035a0 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c8e7fc640 1 -- 172.21.15.64:0/376837245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88035070 con 0x7f0c981035a0 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.355+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9819d530 con 0x7f0c981035a0 2024-09-19T05:06:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.356+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c981a4e20 con 0x7f0c981035a0 2024-09-19T05:06:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 Processor -- start 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 -- start start 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa991262f0 con 0x55fa991c2410 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa990ce270 con 0x55fa991cccb0 2024-09-19T05:06:44.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.359+0000 7f0ca203f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa991222b0 con 0x55fa991c1af0 2024-09-19T05:06:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.360+0000 7f0c9d04b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.360+0000 7f0c8e7fc640 1 -- 172.21.15.64:0/376837245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f0c88042030 con 0x7f0c981035a0 2024-09-19T05:06:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.360+0000 7f0c9d04b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52514/0 (socket says 172.21.15.64:52514) 2024-09-19T05:06:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.360+0000 7f0c9d04b640 1 -- 172.21.15.64:0/104708749 learned_addr learned my addr 172.21.15.64:0/104708749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.360+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.360+0000 7f0c9d04b640 1 -- 172.21.15.64:0/104708749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 msgr2=0x55fa991c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c8e7fc640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c840a3600 0x7f0c840a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9d04b640 1 -- 172.21.15.64:0/104708749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 msgr2=0x55fa991cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9d04b640 1 -- 172.21.15.64:0/104708749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa990cd340 con 0x55fa991c2410 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c8e7fc640 1 -- 172.21.15.64:0/376837245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c880f0230 con 0x7f0c981035a0 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c840a3600 0x7f0c840a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991cc770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0c940115c0 tx=0x7f0c94011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05d264dc58a10b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c837fe640 1 -- 172.21.15.64:0/104708749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9400f640 con 0x55fa991c2410 2024-09-19T05:06:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c837fe640 1 -- 172.21.15.64:0/104708749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c9400fc90 con 0x55fa991c2410 2024-09-19T05:06:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.361+0000 7f0c837fe640 1 -- 172.21.15.64:0/104708749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94019d60 con 0x55fa991c2410 2024-09-19T05:06:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 -- 172.21.15.64:0/104708749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 msgr2=0x55fa991cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c840a3600 0x7f0c840a5ac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c90005fd0 tx=0x7f0c90004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991cc770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0c940115c0 tx=0x7f0c94011a80 comp rx=0 tx=0).stop 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 -- 172.21.15.64:0/104708749 shutdown_connections 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991cc770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 --2- 172.21.15.64:0/104708749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.362+0000 7f0ca203f740 1 -- 172.21.15.64:0/104708749 >> 172.21.15.64:0/104708749 conn(0x55fa991be060 msgr2=0x55fa991be440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.363+0000 7f0ca203f740 1 -- 172.21.15.64:0/104708749 shutdown_connections 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.363+0000 7f0ca203f740 1 -- 172.21.15.64:0/104708749 wait complete. 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.363+0000 7f0ca203f740 1 -- start start 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.363+0000 7f0ca203f740 1 Processor -- start 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.363+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.363+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0ca203f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa991d3c90 con 0x55fa991c2410 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0ca203f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa990af290 con 0x55fa991cccb0 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0ca203f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa991d3e10 con 0x55fa991c1af0 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52520/0 (socket says 172.21.15.64:52520) 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 -- 172.21.15.64:0/958442041 learned_addr learned my addr 172.21.15.64:0/958442041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0ca203f740 10 client.0 _ll_register_callbacks cb 0x55fa990e0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 -- 172.21.15.64:0/958442041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 msgr2=0x55fa991c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 -- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 msgr2=0x55fa991ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 -- 172.21.15.64:0/958442041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa98e39210 con 0x55fa991c2410 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991ca120 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c880bc030 tx=0x7f0c880bbf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d09ee52f9fd478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.364+0000 7f0c80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0c80ff9640 1 -- 172.21.15.64:0/958442041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880c4d10 con 0x55fa991c2410 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c9400fc90 con 0x55fa991c2410 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0c80ff9640 1 -- 172.21.15.64:0/958442041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c880eda80 con 0x55fa991c2410 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0c80ff9640 1 -- 172.21.15.64:0/958442041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c880c06e0 con 0x55fa991c2410 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0ca203f740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa98f05620 con 0x55fa991c2410 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.365+0000 7f0c80ff9640 1 -- 172.21.15.64:0/958442041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c9400fc90 con 0x55fa991c2410 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0c80ff9640 1 -- 172.21.15.64:0/958442041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0c880d50f0 con 0x55fa991c2410 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0ca203f740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0ca203f740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0ca203f740 20 client.15456 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0ca203f740 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa991e12a0 0x55fa991e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0ca203f740 4 client.15456 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa99133a20 con 0x55fa991e12a0 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa991e12a0 0x55fa991e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0c80ff9640 10 client.15456 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.366+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa991e12a0 0x55fa991e3690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42837bf082a9661 server_cookie=bbacaf411d9bfb22 in_seq=0 out_seq=0 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.367+0000 7f0c80ff9640 1 -- 172.21.15.64:0/958442041 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa99133a20 con 0x55fa991e12a0 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.367+0000 7f0c80ff9640 10 client.15456 handle_command_reply: tid=0 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.367+0000 7f0ca203f740 1 client.15456 shutdown 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 msgr2=0x55fa991ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991ca120 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c880bc030 tx=0x7f0c880bbf30 comp rx=0 tx=0).stop 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 shutdown_connections 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa991e12a0 0x55fa991e3690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991ca120 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 --2- 172.21.15.64:0/958442041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 >> 172.21.15.64:0/958442041 conn(0x55fa991be060 msgr2=0x55fa991c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0c80ff9640 0 client.15456 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 shutdown_connections 2024-09-19T05:06:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 1 -- 172.21.15.64:0/958442041 wait complete. 2024-09-19T05:06:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.368+0000 7f0ca203f740 20 client.15456 trim_cache size 0 max 16384 2024-09-19T05:06:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 Processor -- start 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 -- start start 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52528/0 (socket says 172.21.15.64:52528) 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0c9c84a640 1 -- 172.21.15.64:0/895594397 learned_addr learned my addr 172.21.15.64:0/895594397 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981addd0 con 0x55fa991c2410 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981ab6a0 con 0x55fa991cccb0 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a94f0 con 0x55fa991c1af0 2024-09-19T05:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.397+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9d84c640 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9c84a640 1 -- 172.21.15.64:0/895594397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 msgr2=0x55fa991cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9c84a640 1 -- 172.21.15.64:0/895594397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 msgr2=0x55fa991e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9c84a640 1 -- 172.21.15.64:0/895594397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa991c1870 con 0x55fa991c2410 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991e8f80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c880b8d00 tx=0x7f0c8802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197bdcafbfbc36ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c82ffd640 1 -- 172.21.15.64:0/895594397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880d4070 con 0x55fa991c2410 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c82ffd640 1 -- 172.21.15.64:0/895594397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c880c4fa0 con 0x55fa991c2410 2024-09-19T05:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.398+0000 7f0c82ffd640 1 -- 172.21.15.64:0/895594397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880bbab0 con 0x55fa991c2410 2024-09-19T05:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 msgr2=0x55fa991e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991e8f80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c880b8d00 tx=0x7f0c8802a480 comp rx=0 tx=0).stop 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 shutdown_connections 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991cccb0 0x55fa991e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991c2410 0x55fa991e8f80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 --2- 172.21.15.64:0/895594397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.399+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 >> 172.21.15.64:0/895594397 conn(0x55fa991be060 msgr2=0x55fa991cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.400+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 shutdown_connections 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.400+0000 7f0ca203f740 1 -- 172.21.15.64:0/895594397 wait complete. 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.400+0000 7f0ca203f740 1 -- start start 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.400+0000 7f0ca203f740 1 Processor -- start 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 0x55fa991c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991cccb0 0x55fa991c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a9b60 con 0x55fa991cccb0 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981aafd0 con 0x55fa991c2410 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981aa230 con 0x55fa991c1af0 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0ca203f740 10 client.0 _ll_register_callbacks cb 0x7f0c88004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9d84c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991cccb0 0x55fa991c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 0x55fa991c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9d84c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991cccb0 0x55fa991c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52538/0 (socket says 172.21.15.64:52538) 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9d84c640 1 -- 172.21.15.64:0/741307669 learned_addr learned my addr 172.21.15.64:0/741307669 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 0x55fa991c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58836/0 (socket says 172.21.15.64:58836) 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 -- 172.21.15.64:0/741307669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 msgr2=0x55fa991d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 -- 172.21.15.64:0/741307669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991cccb0 msgr2=0x55fa991c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991cccb0 0x55fa991c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 -- 172.21.15.64:0/741307669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa991c76a0 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c9c84a640 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 0x55fa991c6930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55fa991cca40 tx=0x7f0c880ba840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e26a2c80c77a5906 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.401+0000 7f0c80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0c80ff9640 1 -- 172.21.15.64:0/741307669 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880d9020 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c880c4fa0 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0c80ff9640 1 -- 172.21.15.64:0/741307669 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c880c0180 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0c80ff9640 1 -- 172.21.15.64:0/741307669 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c880a0020 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0ca203f740 10 client.25175 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa9924bf90 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.402+0000 7f0c80ff9640 1 -- 172.21.15.64:0/741307669 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c880c4fa0 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0c80ff9640 1 -- 172.21.15.64:0/741307669 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0c880b8080 con 0x55fa991c2410 2024-09-19T05:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0ca203f740 10 client.25175 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0ca203f740 10 client.25175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0ca203f740 20 client.25175 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0ca203f740 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa9924c460 0x55fa9924e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0ca203f740 4 client.25175 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa9912dd70 con 0x55fa9924c460 2024-09-19T05:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.403+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa9924c460 0x55fa9924e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.404+0000 7f0c9d04b640 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa9924c460 0x55fa9924e850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8704b1226f4135 server_cookie=3a095af390416f5 in_seq=0 out_seq=0 2024-09-19T05:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.404+0000 7f0c80ff9640 10 client.25175 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.407+0000 7f0c80ff9640 1 -- 172.21.15.64:0/741307669 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa9912dd70 con 0x55fa9924c460 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.407+0000 7f0c80ff9640 10 client.25175 handle_command_reply: tid=0 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 client.25175 shutdown 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 msgr2=0x55fa991c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 0x55fa991c6930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55fa991cca40 tx=0x7f0c880ba840 comp rx=0 tx=0).stop 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 shutdown_connections 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fa9924c460 0x55fa9924e850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa991cccb0 0x55fa991c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa991c2410 0x55fa991c6930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 --2- 172.21.15.64:0/741307669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa991c1af0 0x55fa991d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 >> 172.21.15.64:0/741307669 conn(0x55fa991be060 msgr2=0x55fa991bd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0c80ff9640 0 client.25175 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 shutdown_connections 2024-09-19T05:06:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 1 -- 172.21.15.64:0/741307669 wait complete. 2024-09-19T05:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.408+0000 7f0ca203f740 20 client.25175 trim_cache size 0 max 16384 2024-09-19T05:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c840a3600 msgr2=0x7f0c840a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c840a3600 0x7f0c840a5ac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c90005fd0 tx=0x7f0c90004ab0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 msgr2=0x7f0c981987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c981987f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005940 tx=0x7f0c88031a10 comp rx=0 tx=0).stop 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 shutdown_connections 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c840a3600 0x7f0c840a5ac0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c9810c0d0 0x7f0c9819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c981035a0 0x7f0c981987f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 --2- 172.21.15.64:0/376837245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98102c80 0x7f0c981982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 >> 172.21.15.64:0/376837245 conn(0x7f0c98077990 msgr2=0x7f0c980fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 shutdown_connections 2024-09-19T05:06:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:44.409+0000 7f0c9f2af640 1 -- 172.21.15.64:0/376837245 wait complete. 2024-09-19T05:06:44.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:06:44.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:06:45.045 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 -- 172.21.15.64:0/1885864637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98102570 msgr2=0x7f2b981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 --2- 172.21.15.64:0/1885864637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98102570 0x7f2b981029b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009900 tx=0x7f2b8802a480 comp rx=0 tx=0).stop 2024-09-19T05:06:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 -- 172.21.15.64:0/1885864637 shutdown_connections 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 --2- 172.21.15.64:0/1885864637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102ef0 0x7f2b9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 --2- 172.21.15.64:0/1885864637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98102570 0x7f2b981029b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 --2- 172.21.15.64:0/1885864637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b98100e20 0x7f2b98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 -- 172.21.15.64:0/1885864637 >> 172.21.15.64:0/1885864637 conn(0x7f2b980fc7d0 msgr2=0x7f2b980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 -- 172.21.15.64:0/1885864637 shutdown_connections 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.539+0000 7f2ba027d640 1 -- 172.21.15.64:0/1885864637 wait complete. 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.540+0000 7f2ba027d640 1 Processor -- start 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.540+0000 7f2ba027d640 1 -- start start 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2ba027d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98100e20 0x7f2b981972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2ba027d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 0x7f2b981977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2ba027d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b98102ef0 0x7f2b9819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2ba027d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98110e30 con 0x7f2b98100e20 2024-09-19T05:06:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2ba027d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98110cb0 con 0x7f2b98102570 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9d818640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 0x7f2b981977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e019640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98100e20 0x7f2b981972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9d818640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 0x7f2b981977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58844/0 (socket says 172.21.15.64:58844) 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9d818640 1 -- 172.21.15.64:0/2315266279 learned_addr learned my addr 172.21.15.64:0/2315266279 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98110fb0 con 0x7f2b98102ef0 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b98102ef0 0x7f2b9819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e019640 1 -- 172.21.15.64:0/2315266279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b98102ef0 msgr2=0x7f2b9819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e019640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b98102ef0 0x7f2b9819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e019640 1 -- 172.21.15.64:0/2315266279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 msgr2=0x7f2b981977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e019640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 0x7f2b981977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9e019640 1 -- 172.21.15.64:0/2315266279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9819c260 con 0x7f2b98100e20 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.541+0000 7f2b9d818640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 0x7f2b981977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.542+0000 7f2b9e019640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98100e20 0x7f2b981972b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d6d0 tx=0x7f2b9400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.542+0000 7f2b8f7fe640 1 -- 172.21.15.64:0/2315266279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94017070 con 0x7f2b98100e20 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.542+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9819c550 con 0x7f2b98100e20 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.542+0000 7f2b8f7fe640 1 -- 172.21.15.64:0/2315266279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b94009820 con 0x7f2b98100e20 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.542+0000 7f2b8f7fe640 1 -- 172.21.15.64:0/2315266279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94004c40 con 0x7f2b98100e20 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.542+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b98069f70 con 0x7f2b98100e20 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.545+0000 7f2ba2656740 1 Processor -- start 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.545+0000 7f2ba2656740 1 -- start start 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.545+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.545+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d3480 0x55c95e1dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.545+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ddd40 0x55c95e1e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2ba2656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c95e138280 con 0x55c95e1ddd40 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2ba2656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c95e0dfb10 con 0x55c95e1d2b60 2024-09-19T05:06:45.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2ba2656740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c95e134240 con 0x55c95e1d3480 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2b9e81a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2b9d818640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ddd40 0x55c95e1e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2b9e81a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58852/0 (socket says 172.21.15.64:58852) 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.546+0000 7f2b9e81a640 1 -- 172.21.15.64:0/1758700818 learned_addr learned my addr 172.21.15.64:0/1758700818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b9d818640 1 -- 172.21.15.64:0/1758700818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d3480 msgr2=0x55c95e1dd800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b9d818640 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d3480 0x55c95e1dd800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b9d818640 1 -- 172.21.15.64:0/1758700818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 msgr2=0x55c95e1d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b9d818640 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b9d818640 1 -- 172.21.15.64:0/1758700818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95e0ec1f0 con 0x55c95e1ddd40 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b8f7fe640 1 -- 172.21.15.64:0/2315266279 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f2b940099d0 con 0x7f2b98100e20 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b9d818640 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ddd40 0x55c95e1e0130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009230 tx=0x7f2b88032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a1eccdaafbee9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b8cdea640 1 -- 172.21.15.64:0/1758700818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88040070 con 0x55c95e1ddd40 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b8cdea640 1 -- 172.21.15.64:0/1758700818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b88031870 con 0x55c95e1ddd40 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.547+0000 7f2b8cdea640 1 -- 172.21.15.64:0/1758700818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88035040 con 0x55c95e1ddd40 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.548+0000 7f2b8f7fe640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b840a7960 0x7f2b840a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.548+0000 7f2b8f7fe640 1 -- 172.21.15.64:0/2315266279 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2b940cf820 con 0x7f2b98100e20 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.548+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b840a7960 0x7f2b840a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.548+0000 7f2ba2656740 1 -- 172.21.15.64:0/1758700818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ddd40 msgr2=0x55c95e1e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.548+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ddd40 0x55c95e1e0130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009230 tx=0x7f2b88032f90 comp rx=0 tx=0).stop 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 -- 172.21.15.64:0/1758700818 shutdown_connections 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ddd40 0x55c95e1e0130 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d3480 0x55c95e1dd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1758700818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 -- 172.21.15.64:0/1758700818 >> 172.21.15.64:0/1758700818 conn(0x55c95e1cf0f0 msgr2=0x55c95e1cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b840a7960 0x7f2b840a9e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000f630 tx=0x7f2b90010040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 -- 172.21.15.64:0/1758700818 shutdown_connections 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 -- 172.21.15.64:0/1758700818 wait complete. 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.549+0000 7f2ba2656740 1 -- start start 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 Processor -- start 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 0x55c95e1dacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1db200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1ddd40 0x55c95e1db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c95e1e0910 con 0x55c95e1d3480 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c95e0f2f10 con 0x55c95e1ddd40 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c95e1e0a90 con 0x55c95e1d2b60 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2b9d818640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1db200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2ba2656740 10 client.0 _ll_register_callbacks cb 0x55c95e0bba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2b9e019640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 0x55c95e1dacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2b9e019640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 0x55c95e1dacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:54728/0 (socket says 172.21.15.64:54728) 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2b9e019640 1 -- 172.21.15.64:0/3690123919 learned_addr learned my addr 172.21.15.64:0/3690123919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:45.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.550+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1ddd40 0x55c95e1db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9d818640 1 -- 172.21.15.64:0/3690123919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 msgr2=0x55c95e1dacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9d818640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 0x55c95e1dacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9d818640 1 -- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1ddd40 msgr2=0x55c95e1db740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9d818640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1ddd40 0x55c95e1db740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9d818640 1 -- 172.21.15.64:0/3690123919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c95dea96a0 con 0x55c95e1d3480 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9e019640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 0x55c95e1dacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:45.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b9d818640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1db200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b88031a20 tx=0x7f2b88009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa0bcc0330dd095 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b81ffb640 1 -- 172.21.15.64:0/3690123919 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8804b070 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b88031870 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.551+0000 7f2b81ffb640 1 -- 172.21.15.64:0/3690123919 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2b8803c910 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.552+0000 7f2b81ffb640 1 -- 172.21.15.64:0/3690123919 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b88057050 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.552+0000 7f2ba2656740 10 client.15474 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.552+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c95e1ec5c0 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.552+0000 7f2b81ffb640 1 -- 172.21.15.64:0/3690123919 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b88031870 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2b81ffb640 1 -- 172.21.15.64:0/3690123919 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b88044bb0 con 0x55c95e1d3480 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2ba2656740 10 client.15474 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2ba2656740 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:45.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2ba2656740 20 client.15474 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2ba2656740 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e1ed910 0x55c95e1efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2ba2656740 4 client.15474 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c95e1457f0 con 0x55c95e1ed910 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.553+0000 7f2b9e019640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e1ed910 0x55c95e1efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.554+0000 7f2b9e019640 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e1ed910 0x55c95e1efd00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d9e53fd14c5c91 server_cookie=8ae2f07ec6ac9a12 in_seq=0 out_seq=0 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.554+0000 7f2b81ffb640 10 client.15474 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2b81ffb640 1 -- 172.21.15.64:0/3690123919 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c95e1457f0 con 0x55c95e1ed910 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2b81ffb640 10 client.15474 handle_command_reply: tid=0 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 client.15474 shutdown 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 msgr2=0x55c95e1db200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1db200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b88031a20 tx=0x7f2b88009680 comp rx=0 tx=0).stop 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 shutdown_connections 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1ddd40 0x55c95e1db740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e1ed910 0x55c95e1efd00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1db200 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 --2- 172.21.15.64:0/3690123919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1d2b60 0x55c95e1dacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2b81ffb640 0 client.15474 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 >> 172.21.15.64:0/3690123919 conn(0x55c95e1cf0f0 msgr2=0x55c95e1d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 shutdown_connections 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 1 -- 172.21.15.64:0/3690123919 wait complete. 2024-09-19T05:06:45.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.555+0000 7f2ba2656740 20 client.15474 trim_cache size 0 max 16384 2024-09-19T05:06:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 Processor -- start 2024-09-19T05:06:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 -- start start 2024-09-19T05:06:45.591 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:45 smithi064 ceph-mon[30960]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:45.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981adeb0 con 0x55c95e1d3480 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ab780 con 0x55c95e1d2b60 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.589+0000 7f2ba2656740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981a95d0 con 0x55c95e1ddd40 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e019640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:54744/0 (socket says 172.21.15.64:54744) 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e019640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58858/0 (socket says 172.21.15.64:58858) 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 -- 172.21.15.64:0/1430027713 learned_addr learned my addr 172.21.15.64:0/1430027713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 -- 172.21.15.64:0/1430027713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 msgr2=0x55c95e1d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 -- 172.21.15.64:0/1430027713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 msgr2=0x55c95e1dcb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1dcb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 -- 172.21.15.64:0/1430027713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95e1e04a0 con 0x55c95e1ddd40 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e019640 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f2d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b90003d90 tx=0x7f2b9000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967d249b09ea4bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b83fff640 1 -- 172.21.15.64:0/1430027713 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000e6f0 con 0x55c95e1ddd40 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b83fff640 1 -- 172.21.15.64:0/1430027713 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b9003c070 con 0x55c95e1ddd40 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.590+0000 7f2b83fff640 1 -- 172.21.15.64:0/1430027713 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90037af0 con 0x55c95e1ddd40 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.591+0000 7f2ba2656740 1 -- 172.21.15.64:0/1430027713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 msgr2=0x55c95e1f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.591+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f2d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b90003d90 tx=0x7f2b9000ff90 comp rx=0 tx=0).stop 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.591+0000 7f2ba2656740 1 -- 172.21.15.64:0/1430027713 shutdown_connections 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.591+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f2d20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.591+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1d3480 0x55c95e1dcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.591+0000 7f2ba2656740 1 --2- 172.21.15.64:0/1430027713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.592+0000 7f2ba2656740 1 -- 172.21.15.64:0/1430027713 >> 172.21.15.64:0/1430027713 conn(0x55c95e1cf0f0 msgr2=0x55c95e1dc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:45.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.604+0000 7f2ba2656740 1 -- 172.21.15.64:0/1430027713 shutdown_connections 2024-09-19T05:06:45.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.604+0000 7f2ba2656740 1 -- 172.21.15.64:0/1430027713 wait complete. 2024-09-19T05:06:45.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 -- start start 2024-09-19T05:06:45.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 Processor -- start 2024-09-19T05:06:45.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ed910 0x55c95e1edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981a9c40 con 0x55c95e1ed910 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ab0b0 con 0x55c95e1d2b60 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981aa310 con 0x55c95e1ddd40 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.605+0000 7f2ba2656740 10 client.0 _ll_register_callbacks cb 0x7f2b90009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:54756/0 (socket says 172.21.15.64:54756) 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 -- 172.21.15.64:0/2685894297 learned_addr learned my addr 172.21.15.64:0/2685894297 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9e019640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 -- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 msgr2=0x55c95e1f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:45.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ed910 0x55c95e1edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 -- 172.21.15.64:0/2685894297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ed910 msgr2=0x55c95e1edcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ed910 0x55c95e1edcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 -- 172.21.15.64:0/2685894297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c95e1ee430 con 0x55c95e1ddd40 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9e81a640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ed910 0x55c95e1edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b9d818640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f3460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005ab0 tx=0x7f2b8802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51af8b31f0fa2cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.606+0000 7f2b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2b81ffb640 1 -- 172.21.15.64:0/2685894297 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88054020 con 0x55c95e1ddd40 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9003c070 con 0x55c95e1ddd40 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2b81ffb640 1 -- 172.21.15.64:0/2685894297 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2b88004440 con 0x55c95e1ddd40 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2b81ffb640 1 -- 172.21.15.64:0/2685894297 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b8804b070 con 0x55c95e1ddd40 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2ba2656740 10 client.25051 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:45.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c95e27ac30 con 0x55c95e1ddd40 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2b81ffb640 1 -- 172.21.15.64:0/2685894297 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b9003c070 con 0x55c95e1ddd40 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2b81ffb640 1 -- 172.21.15.64:0/2685894297 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b8803c8a0 con 0x55c95e1ddd40 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2ba2656740 10 client.25051 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.607+0000 7f2ba2656740 10 client.25051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2ba2656740 20 client.25051 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2ba2656740 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e27b300 0x55c95e27d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2ba2656740 4 client.25051 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c95e13fd00 con 0x55c95e27b300 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2b9e019640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e27b300 0x55c95e27d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2b9e019640 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e27b300 0x55c95e27d720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1bfdd134f9a49c7 server_cookie=c9d4b68e30eace90 in_seq=0 out_seq=0 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.608+0000 7f2b81ffb640 10 client.25051 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.609+0000 7f2b81ffb640 1 -- 172.21.15.64:0/2685894297 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c95e13fd00 con 0x55c95e27b300 2024-09-19T05:06:45.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.609+0000 7f2b81ffb640 10 client.25051 handle_command_reply: tid=0 2024-09-19T05:06:45.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.609+0000 7f2ba2656740 1 client.25051 shutdown 2024-09-19T05:06:45.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.610+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 msgr2=0x55c95e1f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.610+0000 7f2ba2656740 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f3460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005ab0 tx=0x7f2b8802a480 comp rx=0 tx=0).stop 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 shutdown_connections 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c95e27b300 0x55c95e27d720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c95e1ed910 0x55c95e1edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c95e1ddd40 0x55c95e1f3460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 --2- 172.21.15.64:0/2685894297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95e1d2b60 0x55c95e1f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:06:45.616 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "2703839a-07ef-42e6-9c4c-45e48c6adf1f", 2024-09-19T05:06:45.617 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:06:45.617 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:06:45.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 >> 172.21.15.64:0/2685894297 conn(0x55c95e1cf0f0 msgr2=0x55c95e1e9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:45.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2b81ffb640 0 client.25051 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 shutdown_connections 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 1 -- 172.21.15.64:0/2685894297 wait complete. 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.611+0000 7f2ba2656740 20 client.25051 trim_cache size 0 max 16384 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b840a7960 msgr2=0x7f2b840a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b840a7960 0x7f2b840a9e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000f630 tx=0x7f2b90010040 comp rx=0 tx=0).stop 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98100e20 msgr2=0x7f2b981972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98100e20 0x7f2b981972b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d6d0 tx=0x7f2b9400db90 comp rx=0 tx=0).stop 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 shutdown_connections 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b840a7960 0x7f2b840a9e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b98102ef0 0x7f2b9819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b98102570 0x7f2b981977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 --2- 172.21.15.64:0/2315266279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b98100e20 0x7f2b981972b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:45.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 >> 172.21.15.64:0/2315266279 conn(0x7f2b980fc7d0 msgr2=0x7f2b980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:45.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 shutdown_connections 2024-09-19T05:06:45.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:45.612+0000 7f2ba027d640 1 -- 172.21.15.64:0/2315266279 wait complete. 2024-09-19T05:06:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:45 smithi103 ceph-mon[34446]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:45 smithi174 ceph-mon[34488]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:46.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:06:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:47.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.208+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/1356789921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c38102d30 msgr2=0x7f4c38103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.208+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/1356789921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c38102d30 0x7f4c38103110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009930 tx=0x7f4c2802a420 comp rx=0 tx=0).stop 2024-09-19T05:06:47.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.209+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/1356789921 shutdown_connections 2024-09-19T05:06:47.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.209+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/1356789921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c381082a0 0x7f4c3810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.209+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/1356789921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c381036e0 0x7f4c38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.209+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/1356789921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c38102d30 0x7f4c38103110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.209+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/1356789921 >> 172.21.15.64:0/1356789921 conn(0x7f4c380fc7d0 msgr2=0x7f4c380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:47.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.211+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/1356789921 shutdown_connections 2024-09-19T05:06:47.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.211+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/1356789921 wait complete. 2024-09-19T05:06:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.212+0000 7f4c3c8b6640 1 Processor -- start 2024-09-19T05:06:47.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.212+0000 7f4c3c8b6640 1 -- start start 2024-09-19T05:06:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3c8b6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c38102d30 0x7f4c38100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3c8b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c381036e0 0x7f4c38100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3c8b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 0x7f4c38101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3c8b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38111f50 con 0x7f4c381082a0 2024-09-19T05:06:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3c8b6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38111dd0 con 0x7f4c381036e0 2024-09-19T05:06:47.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3c8b6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381120d0 con 0x7f4c38102d30 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 0x7f4c38101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 0x7f4c38101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52602/0 (socket says 172.21.15.64:52602) 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 -- 172.21.15.64:0/3548874266 learned_addr learned my addr 172.21.15.64:0/3548874266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c381036e0 0x7f4c38100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c3659c640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c38102d30 0x7f4c38100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 -- 172.21.15.64:0/3548874266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c38102d30 msgr2=0x7f4c38100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c38102d30 0x7f4c38100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 -- 172.21.15.64:0/3548874266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c381036e0 msgr2=0x7f4c38100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.213+0000 7f4c36d9d640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c381036e0 0x7f4c38100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c36d9d640 1 -- 172.21.15.64:0/3548874266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381a6930 con 0x7f4c381082a0 2024-09-19T05:06:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c36d9d640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 0x7f4c38101410 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000cce0 tx=0x7f4c3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:47.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c277fe640 1 -- 172.21.15.64:0/3548874266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000f450 con 0x7f4c381082a0 2024-09-19T05:06:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c277fe640 1 -- 172.21.15.64:0/3548874266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c3000faa0 con 0x7f4c381082a0 2024-09-19T05:06:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381a6c20 con 0x7f4c381082a0 2024-09-19T05:06:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c277fe640 1 -- 172.21.15.64:0/3548874266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30018b70 con 0x7f4c381082a0 2024-09-19T05:06:47.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.214+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381a7110 con 0x7f4c381082a0 2024-09-19T05:06:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.218+0000 7f4c3ec8f740 1 Processor -- start 2024-09-19T05:06:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.218+0000 7f4c277fe640 1 -- 172.21.15.64:0/3548874266 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f4c30018d10 con 0x7f4c381082a0 2024-09-19T05:06:47.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.218+0000 7f4c3ec8f740 1 -- start start 2024-09-19T05:06:47.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2971880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 0x563aa297c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa297ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563aa28d72e0 con 0x563aa29714a0 2024-09-19T05:06:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563aa287ece0 con 0x563aa2971dc0 2024-09-19T05:06:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563aa28d32a0 con 0x563aa297c680 2024-09-19T05:06:47.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 0x563aa297c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 0x563aa297c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58898/0 (socket says 172.21.15.64:58898) 2024-09-19T05:06:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.219+0000 7f4c3659c640 1 -- 172.21.15.64:0/507115113 learned_addr learned my addr 172.21.15.64:0/507115113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2971880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c277fe640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4c200a34d0 0x7f4c200a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c277fe640 1 -- 172.21.15.64:0/3548874266 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4c300d02f0 con 0x7f4c381082a0 2024-09-19T05:06:47.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c36d9d640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4c200a34d0 0x7f4c200a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c3659c640 1 -- 172.21.15.64:0/507115113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 msgr2=0x563aa297ea70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:06:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c3659c640 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa297ea70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c3659c640 1 -- 172.21.15.64:0/507115113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 msgr2=0x563aa2971880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c3659c640 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2971880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.220+0000 7f4c3659c640 1 -- 172.21.15.64:0/507115113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa2863440 con 0x563aa2971dc0 2024-09-19T05:06:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.221+0000 7f4c36d9d640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4c200a34d0 0x7f4c200a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c300cd920 tx=0x7f4c300d05f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.221+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2971880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.221+0000 7f4c3659c640 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 0x563aa297c140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009930 tx=0x7f4c28005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2007670d0be11b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.222+0000 7f4c24ff9640 1 -- 172.21.15.64:0/507115113 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28040070 con 0x563aa2971dc0 2024-09-19T05:06:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.222+0000 7f4c24ff9640 1 -- 172.21.15.64:0/507115113 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c280316a0 con 0x563aa2971dc0 2024-09-19T05:06:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.222+0000 7f4c24ff9640 1 -- 172.21.15.64:0/507115113 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28035040 con 0x563aa2971dc0 2024-09-19T05:06:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.222+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/507115113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 msgr2=0x563aa297c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 0x563aa297c140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009930 tx=0x7f4c28005fd0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/507115113 shutdown_connections 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa297ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2971dc0 0x563aa297c140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/507115113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2971880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/507115113 >> 172.21.15.64:0/507115113 conn(0x563aa296da30 msgr2=0x563aa296de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/507115113 shutdown_connections 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.223+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/507115113 wait complete. 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.224+0000 7f4c3ec8f740 1 -- start start 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.224+0000 7f4c3ec8f740 1 Processor -- start 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 0x563aa2979490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa297c680 0x563aa29799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563aa297f250 con 0x563aa297c680 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563aa2892050 con 0x563aa2979f10 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563aa297f3d0 con 0x563aa29714a0 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c35d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 0x563aa2979490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c35d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 0x563aa2979490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:54778/0 (socket says 172.21.15.64:54778) 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1797712342 learned_addr learned my addr 172.21.15.64:0/1797712342 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3ec8f740 10 client.0 _ll_register_callbacks cb 0x563aa28d2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa297c680 0x563aa29799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 msgr2=0x563aa29886f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.225+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29886f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1797712342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa297c680 msgr2=0x563aa29799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa297c680 0x563aa29799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1797712342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa26483c0 con 0x563aa29714a0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa297c680 0x563aa29799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 0x563aa2979490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00be40 tx=0x7f4c2c0040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d764744aaaa5fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1797712342 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c005220 con 0x563aa29714a0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1797712342 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4c2c0053c0 con 0x563aa29714a0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c280316a0 con 0x563aa29714a0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1797712342 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c2c0055c0 con 0x563aa29714a0 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c3ec8f740 10 client.25057 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.226+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563aa298b220 con 0x563aa29714a0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1797712342 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c2c01b040 con 0x563aa29714a0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1797712342 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4c2c01e020 con 0x563aa29714a0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3ec8f740 10 client.25057 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3ec8f740 10 client.25057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3ec8f740 20 client.25057 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa298c6e0 0x563aa298eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3ec8f740 4 client.25057 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c280316a0 con 0x563aa298c6e0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.227+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa298c6e0 0x563aa298eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.228+0000 7f4c1dffb640 10 client.25057 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.228+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa298c6e0 0x563aa298eb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4289f8abd76234 server_cookie=bb8fae2dbdacb0e1 in_seq=0 out_seq=0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.228+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1797712342 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4c280316a0 con 0x563aa298c6e0 2024-09-19T05:06:47.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.228+0000 7f4c1dffb640 10 client.25057 handle_command_reply: tid=0 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 client.25057 shutdown 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 msgr2=0x563aa2979490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 0x563aa2979490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00be40 tx=0x7f4c2c0040f0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 shutdown_connections 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29886f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa298c6e0 0x563aa298eb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c1dffb640 0 client.25057 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa297c680 0x563aa29799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1797712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa29714a0 0x563aa2979490 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 >> 172.21.15.64:0/1797712342 conn(0x563aa296da30 msgr2=0x563aa2973760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 shutdown_connections 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1797712342 wait complete. 2024-09-19T05:06:47.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.229+0000 7f4c3ec8f740 20 client.25057 trim_cache size 0 max 16384 2024-09-19T05:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.263+0000 7f4c3ec8f740 1 Processor -- start 2024-09-19T05:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.263+0000 7f4c3ec8f740 1 -- start start 2024-09-19T05:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2974b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa2988a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b22b0 con 0x563aa29714a0 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381afb80 con 0x563aa2979f10 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ad9d0 con 0x563aa297c680 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2974b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2974b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52628/0 (socket says 172.21.15.64:52628) 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 -- 172.21.15.64:0/2681479376 learned_addr learned my addr 172.21.15.64:0/2681479376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:47.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c3659c640 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 -- 172.21.15.64:0/2681479376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 msgr2=0x563aa2988a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa2988a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 -- 172.21.15.64:0/2681479376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 msgr2=0x563aa29884e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29884e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 -- 172.21.15.64:0/2681479376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa296e6c0 con 0x563aa29714a0 2024-09-19T05:06:47.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.264+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2974b60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c011be0 tx=0x7f4c2c010510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bbe1af02ce1d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c1ffff640 1 -- 172.21.15.64:0/2681479376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c02f020 con 0x563aa29714a0 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c1ffff640 1 -- 172.21.15.64:0/2681479376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c2c005020 con 0x563aa29714a0 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c1ffff640 1 -- 172.21.15.64:0/2681479376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c004cf0 con 0x563aa29714a0 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/2681479376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 msgr2=0x563aa2974b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2974b60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c011be0 tx=0x7f4c2c010510 comp rx=0 tx=0).stop 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/2681479376 shutdown_connections 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.265+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa2988a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa29884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/2681479376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2974b60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/2681479376 >> 172.21.15.64:0/2681479376 conn(0x563aa296da30 msgr2=0x563aa2979570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/2681479376 shutdown_connections 2024-09-19T05:06:47.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/2681479376 wait complete. 2024-09-19T05:06:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- start start 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 Processor -- start 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2980790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa2980cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa29f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ae040 con 0x563aa29714a0 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381af4b0 con 0x563aa2979f10 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ae710 con 0x563aa297c680 2024-09-19T05:06:47.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.266+0000 7f4c3ec8f740 10 client.0 _ll_register_callbacks cb 0x7f4c2c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2980790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2980790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52640/0 (socket says 172.21.15.64:52640) 2024-09-19T05:06:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1569650926 learned_addr learned my addr 172.21.15.64:0/1569650926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa2980cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1569650926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 msgr2=0x563aa29f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa29f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 msgr2=0x563aa2980cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa2980cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 -- 172.21.15.64:0/1569650926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa2988f60 con 0x563aa29714a0 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa2980cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c35d9b640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2980790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00bb60 tx=0x7f4c2c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e39897c2dfee155 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1569650926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c038020 con 0x563aa29714a0 2024-09-19T05:06:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2c005020 con 0x563aa29714a0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.267+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1569650926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4c2c004a30 con 0x563aa29714a0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1569650926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c2c02f030 con 0x563aa29714a0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563aa2a00030 con 0x563aa29714a0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1569650926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c2c005020 con 0x563aa29714a0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1569650926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4c2c01e070 con 0x563aa29714a0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 20 client.15492 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa2a006f0 0x563aa2a02ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 4 client.15492 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.268+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563aa28e4850 con 0x563aa2a006f0 2024-09-19T05:06:47.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.269+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa2a006f0 0x563aa2a02ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.269+0000 7f4c1dffb640 10 client.15492 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.269+0000 7f4c3659c640 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa2a006f0 0x563aa2a02ae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07cdbbe888d5680 server_cookie=77fb61d251bac1aa in_seq=0 out_seq=0 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.269+0000 7f4c1dffb640 1 -- 172.21.15.64:0/1569650926 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563aa28e4850 con 0x563aa2a006f0 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.269+0000 7f4c1dffb640 10 client.15492 handle_command_reply: tid=0 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.269+0000 7f4c3ec8f740 1 client.15492 shutdown 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 msgr2=0x563aa2980790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2980790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00bb60 tx=0x7f4c2c0040f0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 shutdown_connections 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563aa297c680 0x563aa29f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aa2979f10 0x563aa2980cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563aa2a006f0 0x563aa2a02ae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c1dffb640 0 client.15492 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:47.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 --2- 172.21.15.64:0/1569650926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563aa29714a0 0x563aa2980790 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 >> 172.21.15.64:0/1569650926 conn(0x563aa296da30 msgr2=0x563aa297acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:47.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 shutdown_connections 2024-09-19T05:06:47.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 1 -- 172.21.15.64:0/1569650926 wait complete. 2024-09-19T05:06:47.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3ec8f740 20 client.15492 trim_cache size 0 max 16384 2024-09-19T05:06:47.279 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:06:47.279 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:06:47.279 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:06:47.279 INFO:teuthology.orchestra.run.smithi064.stdout: "2703839a-07ef-42e6-9c4c-45e48c6adf1f": { 2024-09-19T05:06:47.279 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:06:47.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "2703839a-07ef-42e6-9c4c-45e48c6adf1f", 2024-09-19T05:06:47.280 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:06:47.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:06:47.280 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:06:47.280 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:06:47.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4c200a34d0 msgr2=0x7f4c200a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4c200a34d0 0x7f4c200a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c300cd920 tx=0x7f4c300d05f0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 msgr2=0x7f4c38101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:47.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.270+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 0x7f4c38101410 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000cce0 tx=0x7f4c3000a620 comp rx=0 tx=0).stop 2024-09-19T05:06:47.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 shutdown_connections 2024-09-19T05:06:47.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4c200a34d0 0x7f4c200a5990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c381082a0 0x7f4c38101410 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c381036e0 0x7f4c38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 --2- 172.21.15.64:0/3548874266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c38102d30 0x7f4c38100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 >> 172.21.15.64:0/3548874266 conn(0x7f4c380fc7d0 msgr2=0x7f4c38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 shutdown_connections 2024-09-19T05:06:47.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:06:47.271+0000 7f4c3c8b6640 1 -- 172.21.15.64:0/3548874266 wait complete. 2024-09-19T05:06:47.643 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:47 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:06:47.643 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:47 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:47.643 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:47 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:06:47.643 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:47 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:47.643 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:47 smithi064 ceph-mon[30960]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T05:06:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:47 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:06:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:47 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:47 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:06:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:47 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:47 smithi103 ceph-mon[34446]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T05:06:47.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:47 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:06:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:47 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:47 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:06:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:47 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:47 smithi174 ceph-mon[34488]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T05:06:47.983 INFO:tasks.cephfs.filesystem:scrub status for tag:2703839a-07ef-42e6-9c4c-45e48c6adf1f - {'path': '/', 'tag': '2703839a-07ef-42e6-9c4c-45e48c6adf1f', 'options': 'recursive,force'} 2024-09-19T05:06:49.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:49 smithi064 ceph-mon[30960]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:49.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:49 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:06:49.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:49 smithi103 ceph-mon[34446]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:49.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:49 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:06:49.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:49 smithi174 ceph-mon[34488]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:49.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:49 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:06:51.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:51 smithi064 ceph-mon[30960]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:51 smithi103 ceph-mon[34446]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:51.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:51 smithi174 ceph-mon[34488]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:06:53.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:53 smithi064 ceph-mon[30960]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:06:53.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:53 smithi103 ceph-mon[34446]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:06:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:53 smithi174 ceph-mon[34488]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:06:55.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:55 smithi064 ceph-mon[30960]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:06:55.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:55 smithi103 ceph-mon[34446]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:06:55.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:55 smithi174 ceph-mon[34488]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:06:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.501+0000 7f0c46b9f640 1 -- 172.21.15.64:0/1831958646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c401009b0 msgr2=0x7f0c40100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.501+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/1831958646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c401009b0 0x7f0c40100e10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009930 tx=0x7f0c2c02a420 comp rx=0 tx=0).stop 2024-09-19T05:06:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 -- 172.21.15.64:0/1831958646 shutdown_connections 2024-09-19T05:06:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/1831958646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c40101350 0x7f0c4010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/1831958646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c401009b0 0x7f0c40100e10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/1831958646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401069c0 0x7f0c40106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 -- 172.21.15.64:0/1831958646 >> 172.21.15.64:0/1831958646 conn(0x7f0c40077990 msgr2=0x7f0c400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 -- 172.21.15.64:0/1831958646 shutdown_connections 2024-09-19T05:06:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 -- 172.21.15.64:0/1831958646 wait complete. 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 Processor -- start 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.502+0000 7f0c46b9f640 1 -- start start 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c46b9f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401009b0 0x7f0c401982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c46b9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 0x7f0c40198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c46b9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c401069c0 0x7f0c4019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c46b9f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111da0 con 0x7f0c40101350 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c46b9f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111c20 con 0x7f0c401009b0 2024-09-19T05:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c46b9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111f20 con 0x7f0c401069c0 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 0x7f0c40198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 0x7f0c40198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58108/0 (socket says 172.21.15.64:58108) 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 -- 172.21.15.64:0/3585373346 learned_addr learned my addr 172.21.15.64:0/3585373346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c4513c640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c401069c0 0x7f0c4019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c4493b640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401009b0 0x7f0c401982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 -- 172.21.15.64:0/3585373346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c401069c0 msgr2=0x7f0c4019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c401069c0 0x7f0c4019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 -- 172.21.15.64:0/3585373346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401009b0 msgr2=0x7f0c401982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401009b0 0x7f0c401982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c3ffff640 1 -- 172.21.15.64:0/3585373346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4019d2c0 con 0x7f0c40101350 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.503+0000 7f0c4493b640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401009b0 0x7f0c401982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.504+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 0x7f0c40198810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004770 tx=0x7f0c2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.504+0000 7f0c3dffb640 1 -- 172.21.15.64:0/3585373346 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c03f070 con 0x7f0c40101350 2024-09-19T05:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.504+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4019d550 con 0x7f0c40101350 2024-09-19T05:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.504+0000 7f0c3dffb640 1 -- 172.21.15.64:0/3585373346 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c2c03b460 con 0x7f0c40101350 2024-09-19T05:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.504+0000 7f0c3dffb640 1 -- 172.21.15.64:0/3585373346 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c035070 con 0x7f0c40101350 2024-09-19T05:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.504+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a4e40 con 0x7f0c40101350 2024-09-19T05:06:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.507+0000 7f0c3dffb640 1 -- 172.21.15.64:0/3585373346 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 138582+0+0 (secure 0 0 0) 0x7f0c2c03b610 con 0x7f0c40101350 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 Processor -- start 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 -- start start 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 0x55939cb7ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb88770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb8b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55939cae22f0 con 0x55939cb88cb0 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55939ca8a270 con 0x55939cb7e410 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c49a16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55939cade2b0 con 0x55939cb7daf0 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c3ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb8b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c4513c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 0x55939cb7ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c4513c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 0x55939cb7ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:59238/0 (socket says 172.21.15.64:59238) 2024-09-19T05:06:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c4513c640 1 -- 172.21.15.64:0/1768803860 learned_addr learned my addr 172.21.15.64:0/1768803860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.508+0000 7f0c4493b640 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb88770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4513c640 1 -- 172.21.15.64:0/1768803860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 msgr2=0x55939cb88770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4513c640 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb88770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4513c640 1 -- 172.21.15.64:0/1768803860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 msgr2=0x55939cb8b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4513c640 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb8b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4513c640 1 -- 172.21.15.64:0/1768803860 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939ca89340 con 0x55939cb7daf0 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c3dffb640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c300a3680 0x7f0c300a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4493b640 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb88770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4513c640 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 0x55939cb7ded0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c38010370 tx=0x7f0c38010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d2c923a3de01f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c4493b640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c300a3680 0x7f0c300a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c3dffb640 1 -- 172.21.15.64:0/3585373346 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c2c0f0250 con 0x7f0c40101350 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c2affd640 1 -- 172.21.15.64:0/1768803860 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38016070 con 0x55939cb7daf0 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c2affd640 1 -- 172.21.15.64:0/1768803860 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c38004d20 con 0x55939cb7daf0 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c2affd640 1 -- 172.21.15.64:0/1768803860 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c380053e0 con 0x55939cb7daf0 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.509+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb8b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 -- 172.21.15.64:0/1768803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 msgr2=0x55939cb7ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c4493b640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c300a3680 0x7f0c300a5b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005e00 tx=0x7f0c34005d90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 0x55939cb7ded0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c38010370 tx=0x7f0c38010830 comp rx=0 tx=0).stop 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 -- 172.21.15.64:0/1768803860 shutdown_connections 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb8b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb88770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 --2- 172.21.15.64:0/1768803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7daf0 0x55939cb7ded0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 -- 172.21.15.64:0/1768803860 >> 172.21.15.64:0/1768803860 conn(0x55939cb7a060 msgr2=0x55939cb7a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 -- 172.21.15.64:0/1768803860 shutdown_connections 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.510+0000 7f0c49a16740 1 -- 172.21.15.64:0/1768803860 wait complete. 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 -- start start 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 Processor -- start 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 0x55939cb85c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb88cb0 0x55939cb86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55939cb8fc90 con 0x55939cb7daf0 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55939ca6b290 con 0x55939cb7e410 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c49a16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55939cb8fe10 con 0x55939cb88cb0 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c4493b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 0x55939cb85c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c4493b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 0x55939cb85c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58130/0 (socket says 172.21.15.64:58130) 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.511+0000 7f0c4493b640 1 -- 172.21.15.64:0/2054377212 learned_addr learned my addr 172.21.15.64:0/2054377212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c49a16740 10 client.0 _ll_register_callbacks cb 0x55939ca9c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4513c640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb88cb0 0x55939cb86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4493b640 1 -- 172.21.15.64:0/2054377212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb88cb0 msgr2=0x55939cb86690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4493b640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb88cb0 0x55939cb86690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4493b640 1 -- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 msgr2=0x55939cb86150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4493b640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb86150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4493b640 1 -- 172.21.15.64:0/2054377212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939c7f5210 con 0x55939cb7daf0 2024-09-19T05:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c4493b640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 0x55939cb85c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0c3401c620 tx=0x7f0c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2199199f0e85655 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c1ffff640 1 -- 172.21.15.64:0/2054377212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34028680 con 0x55939cb7daf0 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c1ffff640 1 -- 172.21.15.64:0/2054377212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c3401d410 con 0x55939cb7daf0 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.512+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c38004d20 con 0x55939cb7daf0 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c1ffff640 1 -- 172.21.15.64:0/2054377212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c34028820 con 0x55939cb7daf0 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 10 client.15501 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55939c8c1620 con 0x55939cb7daf0 2024-09-19T05:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c1ffff640 1 -- 172.21.15.64:0/2054377212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c3401d610 con 0x55939cb7daf0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c1ffff640 1 -- 172.21.15.64:0/2054377212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0c34031480 con 0x55939cb7daf0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 10 client.15501 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 20 client.15501 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cb9f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 4 client.15501 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.513+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0c38004d20 con 0x55939cb9cfe0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.514+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cb9f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.514+0000 7f0c1ffff640 10 client.15501 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.514+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cb9f3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680e2c9bd4ee2a6d server_cookie=43bcbe5629f652f0 in_seq=0 out_seq=0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c1ffff640 1 -- 172.21.15.64:0/2054377212 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0c38004d20 con 0x55939cb9cfe0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c1ffff640 10 client.15501 handle_command_reply: tid=0 2024-09-19T05:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 client.15501 shutdown 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 msgr2=0x55939cb85c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 0x55939cb85c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0c3401c620 tx=0x7f0c340046c0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 shutdown_connections 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb88cb0 0x55939cb86690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7e410 0x55939cb86150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cb9f3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c1ffff640 0 client.15501 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 --2- 172.21.15.64:0/2054377212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb7daf0 0x55939cb85c10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 >> 172.21.15.64:0/2054377212 conn(0x55939cb7a060 msgr2=0x55939cb7fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 shutdown_connections 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 1 -- 172.21.15.64:0/2054377212 wait complete. 2024-09-19T05:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.515+0000 7f0c49a16740 20 client.15501 trim_cache size 0 max 16384 2024-09-19T05:06:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 Processor -- start 2024-09-19T05:06:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 -- start start 2024-09-19T05:06:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cba4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cba5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401ade00 con 0x55939cb88cb0 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401ab6d0 con 0x55939cb7daf0 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c49a16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401a9520 con 0x55939cb7e410 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c4513c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cba5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c4513c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cba5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58146/0 (socket says 172.21.15.64:58146) 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c4513c640 1 -- 172.21.15.64:0/4067670813 learned_addr learned my addr 172.21.15.64:0/4067670813 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cba4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.530+0000 7f0c4493b640 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c4513c640 1 -- 172.21.15.64:0/4067670813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 msgr2=0x55939cba4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c4513c640 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cba4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c4513c640 1 -- 172.21.15.64:0/4067670813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 msgr2=0x55939cb80560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c4513c640 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb80560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c4513c640 1 -- 172.21.15.64:0/4067670813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939cb9f740 con 0x55939cb88cb0 2024-09-19T05:06:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c4513c640 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cba5220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0c38011010 tx=0x7f0c38003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08476e24f23073a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c2a7fc640 1 -- 172.21.15.64:0/4067670813 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38016030 con 0x55939cb88cb0 2024-09-19T05:06:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c2a7fc640 1 -- 172.21.15.64:0/4067670813 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c380129c0 con 0x55939cb88cb0 2024-09-19T05:06:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.531+0000 7f0c2a7fc640 1 -- 172.21.15.64:0/4067670813 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38005620 con 0x55939cb88cb0 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 -- 172.21.15.64:0/4067670813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 msgr2=0x55939cba5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cba5220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0c38011010 tx=0x7f0c38003bb0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 -- 172.21.15.64:0/4067670813 shutdown_connections 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cba5220 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cba4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 --2- 172.21.15.64:0/4067670813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb80560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 -- 172.21.15.64:0/4067670813 >> 172.21.15.64:0/4067670813 conn(0x55939cb7a060 msgr2=0x55939cb87c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 -- 172.21.15.64:0/4067670813 shutdown_connections 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 -- 172.21.15.64:0/4067670813 wait complete. 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 -- start start 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.532+0000 7f0c49a16740 1 Processor -- start 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cb955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb92070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401a9b90 con 0x55939cb88cb0 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401ab000 con 0x55939cb7daf0 2024-09-19T05:06:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401aa260 con 0x55939cb7e410 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c49a16740 10 client.0 _ll_register_callbacks cb 0x7f0c3400df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c4513c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb92070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c4513c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb92070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58150/0 (socket says 172.21.15.64:58150) 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c4513c640 1 -- 172.21.15.64:0/467671399 learned_addr learned my addr 172.21.15.64:0/467671399 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c4493b640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.533+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cb955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4493b640 1 -- 172.21.15.64:0/467671399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 msgr2=0x55939cb955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4493b640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cb955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4493b640 1 -- 172.21.15.64:0/467671399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 msgr2=0x55939cb92070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4493b640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb92070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4493b640 1 -- 172.21.15.64:0/467671399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939cb925b0 con 0x55939cb7daf0 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4513c640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb92070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c4493b640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb950a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400dfd0 tx=0x7f0c340316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e0aacf0c596c823 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c1ffff640 1 -- 172.21.15.64:0/467671399 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400da70 con 0x55939cb7daf0 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c1ffff640 1 -- 172.21.15.64:0/467671399 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c34028bd0 con 0x55939cb7daf0 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c380129c0 con 0x55939cb7daf0 2024-09-19T05:06:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c1ffff640 1 -- 172.21.15.64:0/467671399 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c34049050 con 0x55939cb7daf0 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c49a16740 10 client.25226 fetch_fsmap learned FSMap version 25 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.534+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55939cc083e0 con 0x55939cb7daf0 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c1ffff640 1 -- 172.21.15.64:0/467671399 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c34011cf0 con 0x55939cb7daf0 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c1ffff640 1 -- 172.21.15.64:0/467671399 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0c34044030 con 0x55939cb7daf0 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c49a16740 10 client.25226 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c49a16740 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c49a16740 20 client.25226 populate_metadata read hostname 'smithi064' 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c49a16740 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cc0ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c49a16740 4 client.25226 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:06:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0c380129c0 con 0x55939cb9cfe0 2024-09-19T05:06:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.535+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cc0ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.536+0000 7f0c1ffff640 10 client.25226 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.536+0000 7f0c3ffff640 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cc0ab90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78e13b0e328a159 server_cookie=fef73aa3367c756e in_seq=0 out_seq=0 2024-09-19T05:06:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c1ffff640 1 -- 172.21.15.64:0/467671399 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0c380129c0 con 0x55939cb9cfe0 2024-09-19T05:06:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c1ffff640 10 client.25226 handle_command_reply: tid=0 2024-09-19T05:06:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 client.25226 shutdown 2024-09-19T05:06:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 msgr2=0x55939cb950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb950a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400dfd0 tx=0x7f0c340316b0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 shutdown_connections 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55939cb9cfe0 0x55939cc0ab90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55939cb88cb0 0x55939cb92070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c1ffff640 0 client.25226 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55939cb7e410 0x55939cb955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 --2- 172.21.15.64:0/467671399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55939cb7daf0 0x55939cb950a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.540+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 >> 172.21.15.64:0/467671399 conn(0x55939cb7a060 msgr2=0x55939cb85bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 shutdown_connections 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c49a16740 1 -- 172.21.15.64:0/467671399 wait complete. 2024-09-19T05:06:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c49a16740 20 client.25226 trim_cache size 0 max 16384 2024-09-19T05:06:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:06:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:06:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:06:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:06:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c300a3680 msgr2=0x7f0c300a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c300a3680 0x7f0c300a5b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005e00 tx=0x7f0c34005d90 comp rx=0 tx=0).stop 2024-09-19T05:06:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 msgr2=0x7f0c40198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 0x7f0c40198810 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004770 tx=0x7f0c2c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 shutdown_connections 2024-09-19T05:06:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c300a3680 0x7f0c300a5b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c401069c0 0x7f0c4019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c40101350 0x7f0c40198810 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.541+0000 7f0c46b9f640 1 --2- 172.21.15.64:0/3585373346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c401009b0 0x7f0c401982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.542+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 >> 172.21.15.64:0/3585373346 conn(0x7f0c40077990 msgr2=0x7f0c40103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.542+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 shutdown_connections 2024-09-19T05:06:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:06:57.542+0000 7f0c46b9f640 1 -- 172.21.15.64:0/3585373346 wait complete. 2024-09-19T05:06:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:57 smithi064 ceph-mon[30960]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:06:57.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:57 smithi103 ceph-mon[34446]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:06:57.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:57 smithi174 ceph-mon[34488]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:06:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:06:59 smithi064 ceph-mon[30960]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-19T05:06:59.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:06:59 smithi103 ceph-mon[34446]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-19T05:06:59.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:06:59 smithi174 ceph-mon[34488]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-19T05:07:00.400 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:07:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:07:00.086+0000 7fdbe07c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:07:00.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:07:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:07:00.086+0000 7fdbe07c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:07:00.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:07:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:07:00.086+0000 7fdbe07c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:07:00.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:07:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:07:00.086+0000 7fdbe07c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:07:00.401 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:07:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:07:00.086+0000 7fdbe07c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:07:01.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:01 smithi103 ceph-mon[34446]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-19T05:07:01.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:01 smithi174 ceph-mon[34488]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-19T05:07:01.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:01 smithi064 ceph-mon[30960]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-19T05:07:03.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:03 smithi103 ceph-mon[34446]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:07:03.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:03 smithi103 ceph-mon[34446]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-09-19T05:07:03.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:03 smithi174 ceph-mon[34488]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:07:03.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:03 smithi174 ceph-mon[34488]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-09-19T05:07:03.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:03 smithi064 ceph-mon[30960]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:07:03.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:03 smithi064 ceph-mon[30960]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-09-19T05:07:04.406 INFO:tasks.workunit.client.0.smithi064.stderr:Updating files: 94% (12190/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T05:07:04.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:07:04.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:07:04.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:07:04.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:07:04.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:07:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:07:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:07:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:07:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:07:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:07:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:07:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:07:05.136 INFO:tasks.workunit.client.0.smithi064.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T05:07:05.136 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.136 INFO:tasks.workunit.client.0.smithi064.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: git switch -c 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr:Or undo this operation with: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: git switch - 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T05:07:05.137 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-19T05:07:05.138 INFO:tasks.workunit.client.0.smithi064.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T05:07:05.142 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T05:07:05.164 INFO:tasks.workunit.client.0.smithi064.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T05:07:05.167 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:07:05.167 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T05:07:05.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:05 smithi103 ceph-mon[34446]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:07:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:05 smithi174 ceph-mon[34488]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:07:05.871 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T05:07:05.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:05 smithi064 ceph-mon[30960]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:07:05.947 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T05:07:05.990 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:07:05.992 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:07:05.992 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T05:07:06.033 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:07:06.036 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T05:07:06.037 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T05:07:06.094 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-09-19T05:07:06.095 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-09-19T05:07:06.095 DEBUG:teuthology.orchestra.run.smithi064:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-09-19T05:07:06.160 INFO:tasks.workunit.client.0.smithi064.stdout:test large (16MB) dio write 2024-09-19T05:07:06.160 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo 'test large (16MB) dio write' 2024-09-19T05:07:06.160 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-09-19T05:07:06.417 INFO:tasks.workunit.client.0.smithi064.stderr:1+0 records in 2024-09-19T05:07:06.417 INFO:tasks.workunit.client.0.smithi064.stderr:1+0 records out 2024-09-19T05:07:06.417 INFO:tasks.workunit.client.0.smithi064.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.25445 s, 65.9 MB/s 2024-09-19T05:07:06.417 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo OK 2024-09-19T05:07:06.418 INFO:tasks.workunit.client.0.smithi064.stdout:OK 2024-09-19T05:07:06.418 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:07:06.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:07:06.497 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-09-19T05:07:06.497 DEBUG:teuthology.orchestra.run.smithi064:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-09-19T05:07:07.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:07 smithi103 ceph-mon[34446]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T05:07:07.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:07 smithi174 ceph-mon[34488]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T05:07:07.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:07 smithi064 ceph-mon[30960]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T05:07:09.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:09 smithi103 ceph-mon[34446]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:07:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:09 smithi174 ceph-mon[34488]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:07:09.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:09 smithi064 ceph-mon[30960]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.635+0000 7f7e1bc65640 1 -- 172.21.15.64:0/1927268164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14107b70 msgr2=0x7f7e14107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.635+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/1927268164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14107b70 0x7f7e14107f50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009930 tx=0x7f7e1002a440 comp rx=0 tx=0).stop 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 -- 172.21.15.64:0/1927268164 shutdown_connections 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/1927268164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e14102470 0x7f7e1410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/1927268164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14101ad0 0x7f7e14101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/1927268164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14107b70 0x7f7e14107f50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 -- 172.21.15.64:0/1927268164 >> 172.21.15.64:0/1927268164 conn(0x7f7e140fd7d0 msgr2=0x7f7e140ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 -- 172.21.15.64:0/1927268164 shutdown_connections 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 -- 172.21.15.64:0/1927268164 wait complete. 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 Processor -- start 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 -- start start 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.636+0000 7f7e1bc65640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e14101ad0 0x7f7e14198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1bc65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 0x7f7e14198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1bc65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14107b70 0x7f7e1419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1bc65640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14111dd0 con 0x7f7e14102470 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1bc65640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14111c50 con 0x7f7e14107b70 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1bc65640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14111f50 con 0x7f7e14101ad0 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 0x7f7e14198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 0x7f7e14198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33152/0 (socket says 172.21.15.64:33152) 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19a01640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e14101ad0 0x7f7e14198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 -- 172.21.15.64:0/3492375078 learned_addr learned my addr 172.21.15.64:0/3492375078 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1a202640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14107b70 0x7f7e1419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 -- 172.21.15.64:0/3492375078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e14101ad0 msgr2=0x7f7e14198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e14101ad0 0x7f7e14198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 -- 172.21.15.64:0/3492375078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14107b70 msgr2=0x7f7e1419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14107b70 0x7f7e1419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 -- 172.21.15.64:0/3492375078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1419d2f0 con 0x7f7e14102470 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e1a202640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14107b70 0x7f7e1419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e19200640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 0x7f7e14198840 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d950 tx=0x7f7e0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.637+0000 7f7e0affd640 1 -- 172.21.15.64:0/3492375078 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04017070 con 0x7f7e14102470 2024-09-19T05:07:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.638+0000 7f7e0affd640 1 -- 172.21.15.64:0/3492375078 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e04009c30 con 0x7f7e14102470 2024-09-19T05:07:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.638+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1419d5e0 con 0x7f7e14102470 2024-09-19T05:07:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.638+0000 7f7e0affd640 1 -- 172.21.15.64:0/3492375078 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0400f620 con 0x7f7e14102470 2024-09-19T05:07:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.638+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e141a4e70 con 0x7f7e14102470 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.640+0000 7f7e0affd640 1 -- 172.21.15.64:0/3492375078 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7f7e040109a0 con 0x7f7e14102470 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.640+0000 7f7e1ea04740 1 Processor -- start 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 -- start start 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec810ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dec80722f0 con 0x55dec810e310 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dec801a270 con 0x55dec811d0c0 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dec806e2b0 con 0x55dec810d9f0 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e0affd640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7e000a3610 0x7f7e000a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e19a01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1a202640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec810ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e19a01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33162/0 (socket says 172.21.15.64:33162) 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e19a01640 1 -- 172.21.15.64:0/244191681 learned_addr learned my addr 172.21.15.64:0/244191681 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e0affd640 1 -- 172.21.15.64:0/3492375078 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7e040cfc20 con 0x7f7e14102470 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e1a202640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7e000a3610 0x7f7e000a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.641+0000 7f7e19200640 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e19a01640 1 -- 172.21.15.64:0/244191681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 msgr2=0x55dec810ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e19a01640 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec810ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e19a01640 1 -- 172.21.15.64:0/244191681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 msgr2=0x55dec811f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e19a01640 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e19a01640 1 -- 172.21.15.64:0/244191681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec8019340 con 0x55dec810e310 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e19a01640 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811cb80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7e1002a960 tx=0x7f7e10032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d7e2621542efb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7e1a202640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7e000a3610 0x7f7e000a5ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009fd0 tx=0x7f7e0c0093d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7dfffff640 1 -- 172.21.15.64:0/244191681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10040070 con 0x55dec810e310 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7dfffff640 1 -- 172.21.15.64:0/244191681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e100316a0 con 0x55dec810e310 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.642+0000 7f7dfffff640 1 -- 172.21.15.64:0/244191681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10035040 con 0x55dec810e310 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 -- 172.21.15.64:0/244191681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 msgr2=0x55dec811cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811cb80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7e1002a960 tx=0x7f7e10032dd0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 -- 172.21.15.64:0/244191681 shutdown_connections 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811cb80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/244191681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 -- 172.21.15.64:0/244191681 >> 172.21.15.64:0/244191681 conn(0x55dec810a060 msgr2=0x55dec810a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 -- 172.21.15.64:0/244191681 shutdown_connections 2024-09-19T05:07:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 -- 172.21.15.64:0/244191681 wait complete. 2024-09-19T05:07:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 -- start start 2024-09-19T05:07:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.643+0000 7f7e1ea04740 1 Processor -- start 2024-09-19T05:07:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec811a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dec811fcc0 con 0x55dec810e310 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dec7ffb290 con 0x55dec811d0c0 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dec811fe40 con 0x55dec810d9f0 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1ea04740 10 client.0 _ll_register_callbacks cb 0x55dec802c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33178/0 (socket says 172.21.15.64:33178) 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 -- 172.21.15.64:0/2766262018 learned_addr learned my addr 172.21.15.64:0/2766262018 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19a01640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec811a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e1a202640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 -- 172.21.15.64:0/2766262018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 msgr2=0x55dec811a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec811a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 -- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 msgr2=0x55dec811aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.644+0000 7f7e19200640 1 -- 172.21.15.64:0/2766262018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec7d85210 con 0x55dec810e310 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e19200640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811a550 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e0409b9c0 tx=0x7f7e040a1200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e51453cf9edf409 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7dfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2766262018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04014020 con 0x55dec810e310 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2766262018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7e0409c650 con 0x55dec810e310 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e100316a0 con 0x55dec810e310 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2766262018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e04020040 con 0x55dec810e310 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 10 client.15525 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dec7e51620 con 0x55dec810e310 2024-09-19T05:07:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2766262018 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e100316a0 con 0x55dec810e310 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2766262018 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7e04097a10 con 0x55dec810e310 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 10 client.15525 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 20 client.15525 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec812d0b0 0x55dec812f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 4 client.15525 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.645+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec807fa20 con 0x55dec812d0b0 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.646+0000 7f7e19a01640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec812d0b0 0x55dec812f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.646+0000 7f7dfd7fa640 10 client.15525 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.646+0000 7f7e19a01640 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec812d0b0 0x55dec812f4a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c04cd666ea5da1 server_cookie=f416816612bc65fe in_seq=0 out_seq=0 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2766262018 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dec807fa20 con 0x55dec812d0b0 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7dfd7fa640 10 client.15525 handle_command_reply: tid=0 2024-09-19T05:07:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 client.15525 shutdown 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 msgr2=0x55dec811a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811a550 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e0409b9c0 tx=0x7f7e040a1200 comp rx=0 tx=0).stop 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 shutdown_connections 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec811d0c0 0x55dec811aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec812d0b0 0x55dec812f4a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec811a550 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2766262018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec810d9f0 0x55dec811a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7dfd7fa640 0 client.15525 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 >> 172.21.15.64:0/2766262018 conn(0x55dec810a060 msgr2=0x55dec810fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 shutdown_connections 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2766262018 wait complete. 2024-09-19T05:07:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.647+0000 7f7e1ea04740 20 client.15525 trim_cache size 0 max 16384 2024-09-19T05:07:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 Processor -- start 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 -- start start 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810d9f0 0x55dec8134d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec81352d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8117250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141add50 con 0x55dec810e310 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141ab620 con 0x55dec810d9f0 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141a9470 con 0x55dec811d0c0 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e19200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec81352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e19200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec81352d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33188/0 (socket says 172.21.15.64:33188) 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e19200640 1 -- 172.21.15.64:0/55693767 learned_addr learned my addr 172.21.15.64:0/55693767 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.665+0000 7f7e1a202640 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8117250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19a01640 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810d9f0 0x55dec8134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19200640 1 -- 172.21.15.64:0/55693767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 msgr2=0x55dec8117250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19200640 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8117250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19200640 1 -- 172.21.15.64:0/55693767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810d9f0 msgr2=0x55dec8134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19200640 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810d9f0 0x55dec8134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19200640 1 -- 172.21.15.64:0/55693767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec8118fa0 con 0x55dec810e310 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7e19200640 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec81352d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e0409b790 tx=0x7f7e04009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c37109c5572f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7dff7fe640 1 -- 172.21.15.64:0/55693767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04014020 con 0x55dec810e310 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7dff7fe640 1 -- 172.21.15.64:0/55693767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e040a0070 con 0x55dec810e310 2024-09-19T05:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.666+0000 7f7dff7fe640 1 -- 172.21.15.64:0/55693767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04098330 con 0x55dec810e310 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 -- 172.21.15.64:0/55693767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 msgr2=0x55dec81352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec81352d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e0409b790 tx=0x7f7e04009740 comp rx=0 tx=0).stop 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 -- 172.21.15.64:0/55693767 shutdown_connections 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810e310 0x55dec81352d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/55693767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810d9f0 0x55dec8134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 -- 172.21.15.64:0/55693767 >> 172.21.15.64:0/55693767 conn(0x55dec810a060 msgr2=0x55dec811c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 -- 172.21.15.64:0/55693767 shutdown_connections 2024-09-19T05:07:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 -- 172.21.15.64:0/55693767 wait complete. 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 -- start start 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 Processor -- start 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 0x55dec81215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.667+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810e310 0x55dec8121b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1ea04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141a9ae0 con 0x55dec810d9f0 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141aaf50 con 0x55dec810e310 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1ea04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141aa1b0 con 0x55dec811d0c0 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1ea04740 10 client.0 _ll_register_callbacks cb 0x7f7e0409ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 0x55dec81215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 0x55dec81215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33198/0 (socket says 172.21.15.64:33198) 2024-09-19T05:07:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 -- 172.21.15.64:0/2581784066 learned_addr learned my addr 172.21.15.64:0/2581784066 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1a202640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19200640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810e310 0x55dec8121b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 -- 172.21.15.64:0/2581784066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 msgr2=0x55dec8196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 -- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810e310 msgr2=0x55dec8121b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810e310 0x55dec8121b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 -- 172.21.15.64:0/2581784066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec812f810 con 0x55dec810d9f0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19200640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810e310 0x55dec8121b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e19a01640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 0x55dec81215f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009930 tx=0x7f7e10031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5bb65baad7a09d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7dfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2581784066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10035040 con 0x55dec810d9f0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2581784066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7e10031cf0 con 0x55dec810d9f0 2024-09-19T05:07:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e040a0070 con 0x55dec810d9f0 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.668+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2581784066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e1005c050 con 0x55dec810d9f0 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 10 client.15531 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dec8198380 con 0x55dec810d9f0 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2581784066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e040a0070 con 0x55dec810d9f0 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2581784066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7e100049c0 con 0x55dec810d9f0 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 10 client.15531 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 20 client.15531 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec81986d0 0x55dec819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 4 client.15531 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:07:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dec8079d70 con 0x55dec81986d0 2024-09-19T05:07:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.669+0000 7f7e19200640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec81986d0 0x55dec819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.670+0000 7f7e19200640 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec81986d0 0x55dec819aac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0d5a18dcf83932 server_cookie=d24f90f402302abf in_seq=0 out_seq=0 2024-09-19T05:07:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.670+0000 7f7dfd7fa640 10 client.15531 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.681+0000 7f7dfd7fa640 1 -- 172.21.15.64:0/2581784066 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dec8079d70 con 0x55dec81986d0 2024-09-19T05:07:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.681+0000 7f7dfd7fa640 10 client.15531 handle_command_reply: tid=0 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 client.15531 shutdown 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 msgr2=0x55dec81215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 0x55dec81215f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009930 tx=0x7f7e10031970 comp rx=0 tx=0).stop 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 shutdown_connections 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dec811d0c0 0x55dec8196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dec810e310 0x55dec8121b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55dec81986d0 0x55dec819aac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 --2- 172.21.15.64:0/2581784066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dec810d9f0 0x55dec81215f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7dfd7fa640 0 client.15531 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 >> 172.21.15.64:0/2581784066 conn(0x55dec810a060 msgr2=0x55dec810f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 shutdown_connections 2024-09-19T05:07:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 1 -- 172.21.15.64:0/2581784066 wait complete. 2024-09-19T05:07:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.682+0000 7f7e1ea04740 20 client.15531 trim_cache size 0 max 16384 2024-09-19T05:07:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:07:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:07:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:07:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7e000a3610 msgr2=0x7f7e000a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7e000a3610 0x7f7e000a5ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009fd0 tx=0x7f7e0c0093d0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 msgr2=0x7f7e14198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 0x7f7e14198840 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d950 tx=0x7f7e0400de10 comp rx=0 tx=0).stop 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 shutdown_connections 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7e000a3610 0x7f7e000a5ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14107b70 0x7f7e1419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7e14102470 0x7f7e14198840 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 --2- 172.21.15.64:0/3492375078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e14101ad0 0x7f7e14198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 >> 172.21.15.64:0/3492375078 conn(0x7f7e140fd7d0 msgr2=0x7f7e140ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 shutdown_connections 2024-09-19T05:07:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:10.683+0000 7f7e1bc65640 1 -- 172.21.15.64:0/3492375078 wait complete. 2024-09-19T05:07:11.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:11 smithi103 ceph-mon[34446]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:07:11.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:11 smithi174 ceph-mon[34488]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:07:11.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:11 smithi064 ceph-mon[30960]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:07:13.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:13 smithi103 ceph-mon[34446]: pgmap v42: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-09-19T05:07:13.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:13 smithi174 ceph-mon[34488]: pgmap v42: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-09-19T05:07:13.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:13 smithi064 ceph-mon[30960]: pgmap v42: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-09-19T05:07:15.702 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:15 smithi064 ceph-mon[30960]: pgmap v43: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 414 op/s 2024-09-19T05:07:15.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:15 smithi103 ceph-mon[34446]: pgmap v43: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 414 op/s 2024-09-19T05:07:15.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:15 smithi174 ceph-mon[34488]: pgmap v43: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 414 op/s 2024-09-19T05:07:17.001 INFO:tasks.workunit.client.0.smithi064.stdout:.......... 2024-09-19T05:07:17.001 INFO:tasks.workunit.client.0.smithi064.stdout:do_o_direct_reads: SUCCESS 2024-09-19T05:07:17.005 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:07:17.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:07:17.081 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-09-19T05:07:17.081 DEBUG:teuthology.orchestra.run.smithi064:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-09-19T05:07:17.148 INFO:tasks.workunit.client.0.smithi064.stdout:test read from hole 2024-09-19T05:07:17.149 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo 'test read from hole' 2024-09-19T05:07:17.149 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-09-19T05:07:17.151 INFO:tasks.workunit.client.0.smithi064.stderr:0+0 records in 2024-09-19T05:07:17.151 INFO:tasks.workunit.client.0.smithi064.stderr:0+0 records out 2024-09-19T05:07:17.151 INFO:tasks.workunit.client.0.smithi064.stderr:0 bytes copied, 8.0002e-05 s, 0.0 kB/s 2024-09-19T05:07:17.151 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-09-19T05:07:17.154 INFO:tasks.workunit.client.0.smithi064.stderr:2+0 records in 2024-09-19T05:07:17.154 INFO:tasks.workunit.client.0.smithi064.stderr:2+0 records out 2024-09-19T05:07:17.154 INFO:tasks.workunit.client.0.smithi064.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00178773 s, 573 kB/s 2024-09-19T05:07:17.154 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-09-19T05:07:17.155 INFO:tasks.workunit.client.0.smithi064.stderr:2+0 records in 2024-09-19T05:07:17.155 INFO:tasks.workunit.client.0.smithi064.stderr:2+0 records out 2024-09-19T05:07:17.155 INFO:tasks.workunit.client.0.smithi064.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.227e-05 s, 24.2 MB/s 2024-09-19T05:07:17.155 INFO:tasks.workunit.client.0.smithi064.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-09-19T05:07:17.156 INFO:tasks.workunit.client.0.smithi064.stdout:other thing 2024-09-19T05:07:17.156 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo 'other thing' 2024-09-19T05:07:17.156 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-09-19T05:07:17.156 INFO:tasks.workunit.client.0.smithi064.stderr:1+0 records in 2024-09-19T05:07:17.157 INFO:tasks.workunit.client.0.smithi064.stderr:1+0 records out 2024-09-19T05:07:17.157 INFO:tasks.workunit.client.0.smithi064.stderr:500 bytes copied, 4.0563e-05 s, 12.3 MB/s 2024-09-19T05:07:17.157 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-09-19T05:07:17.164 INFO:tasks.workunit.client.0.smithi064.stderr:0+1 records in 2024-09-19T05:07:17.164 INFO:tasks.workunit.client.0.smithi064.stderr:0+1 records out 2024-09-19T05:07:17.164 INFO:tasks.workunit.client.0.smithi064.stderr:500 bytes copied, 0.00522451 s, 95.7 kB/s 2024-09-19T05:07:17.165 INFO:tasks.workunit.client.0.smithi064.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-09-19T05:07:17.170 INFO:tasks.workunit.client.0.smithi064.stderr:0+1 records in 2024-09-19T05:07:17.170 INFO:tasks.workunit.client.0.smithi064.stderr:0+1 records out 2024-09-19T05:07:17.170 INFO:tasks.workunit.client.0.smithi064.stderr:500 bytes copied, 0.0047518 s, 105 kB/s 2024-09-19T05:07:17.170 INFO:tasks.workunit.client.0.smithi064.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-09-19T05:07:17.171 INFO:tasks.workunit.client.0.smithi064.stdout:OK 2024-09-19T05:07:17.172 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo OK 2024-09-19T05:07:17.172 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:07:17.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:07:17.254 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-09-19T05:07:17.254 DEBUG:teuthology.orchestra.run.smithi064:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-09-19T05:07:17.327 INFO:tasks.workunit.client.0.smithi064.stdout:writing first 3 bytes of 10k file 2024-09-19T05:07:17.327 INFO:tasks.workunit.client.0.smithi064.stdout:reading O_DIRECT 2024-09-19T05:07:17.327 INFO:tasks.workunit.client.0.smithi064.stdout:got 10000 2024-09-19T05:07:17.328 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:07:17.328 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:07:17.404 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-09-19T05:07:17.405 DEBUG:teuthology.orchestra.run.smithi064:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-09-19T05:07:17.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:17 smithi064 ceph-mon[30960]: pgmap v44: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.43k op/s 2024-09-19T05:07:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:17 smithi103 ceph-mon[34446]: pgmap v44: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.43k op/s 2024-09-19T05:07:17.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:17 smithi174 ceph-mon[34488]: pgmap v44: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.43k op/s 2024-09-19T05:07:17.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:07:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:18.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 -- 172.21.15.64:0/466462534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 msgr2=0x7f1e741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 --2- 172.21.15.64:0/466462534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e741029b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e6802a480 comp rx=0 tx=0).stop 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 -- 172.21.15.64:0/466462534 shutdown_connections 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 --2- 172.21.15.64:0/466462534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e74102ef0 0x7f1e7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 --2- 172.21.15.64:0/466462534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e741029b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 --2- 172.21.15.64:0/466462534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e74100e20 0x7f1e74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 -- 172.21.15.64:0/466462534 >> 172.21.15.64:0/466462534 conn(0x7f1e740fc7d0 msgr2=0x7f1e740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 -- 172.21.15.64:0/466462534 shutdown_connections 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.793+0000 7f1e78c06640 1 -- 172.21.15.64:0/466462534 wait complete. 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.794+0000 7f1e78c06640 1 Processor -- start 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.794+0000 7f1e78c06640 1 -- start start 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.794+0000 7f1e78c06640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e74100e20 0x7f1e741983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.794+0000 7f1e78c06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e74198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e78c06640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e74102ef0 0x7f1e7419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e78c06640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74110e30 con 0x7f1e74102570 2024-09-19T05:07:18.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e78c06640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74110cb0 con 0x7f1e74102ef0 2024-09-19T05:07:18.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e78c06640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e74110fb0 con 0x7f1e74100e20 2024-09-19T05:07:18.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e74198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.848 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:07:18.848 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:07:18.848 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:07:18.848 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e74198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60754/0 (socket says 172.21.15.64:60754) 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 -- 172.21.15.64:0/1062003133 learned_addr learned my addr 172.21.15.64:0/1062003133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e7259c640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e74100e20 0x7f1e741983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e74102ef0 0x7f1e7419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 -- 172.21.15.64:0/1062003133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e74100e20 msgr2=0x7f1e741983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e74100e20 0x7f1e741983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 -- 172.21.15.64:0/1062003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e74102ef0 msgr2=0x7f1e7419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e74102ef0 0x7f1e7419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 -- 172.21.15.64:0/1062003133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7419d390 con 0x7f1e74102570 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.795+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e74198920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006010 tx=0x7f1e680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:18.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.796+0000 7f1e637fe640 1 -- 172.21.15.64:0/1062003133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6803f030 con 0x7f1e74102570 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.796+0000 7f1e637fe640 1 -- 172.21.15.64:0/1062003133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e68003cb0 con 0x7f1e74102570 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.796+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7419d620 con 0x7f1e74102570 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.796+0000 7f1e637fe640 1 -- 172.21.15.64:0/1062003133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68035070 con 0x7f1e74102570 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.796+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e741a4f60 con 0x7f1e74102570 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.799+0000 7f1e637fe640 1 -- 172.21.15.64:0/1062003133 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7f1e68042030 con 0x7f1e74102570 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.799+0000 7f1e7afdf740 1 Processor -- start 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.799+0000 7f1e7afdf740 1 -- start start 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb7a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb85680 0x55a0fbb87a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a0fbae02e0 con 0x55a0fbb7adc0 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0fba87ce0 con 0x55a0fbb85680 2024-09-19T05:07:18.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0fbadc2a0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb85140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60766/0 (socket says 172.21.15.64:60766) 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 -- 172.21.15.64:0/3437245135 learned_addr learned my addr 172.21.15.64:0/3437245135 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb85680 0x55a0fbb87a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb7a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e637fe640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e5c0a35e0 0x7f1e5c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 -- 172.21.15.64:0/3437245135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 msgr2=0x55a0fbb7a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb7a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 -- 172.21.15.64:0/3437245135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb85680 msgr2=0x55a0fbb87a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb85680 0x55a0fbb87a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 -- 172.21.15.64:0/3437245135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0fba6c440 con 0x55a0fbb7adc0 2024-09-19T05:07:18.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb85680 0x55a0fbb87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:18.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e5c0a35e0 0x7f1e5c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e637fe640 1 -- 172.21.15.64:0/1062003133 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1e680f0800 con 0x7f1e74102570 2024-09-19T05:07:18.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.800+0000 7f1e7259c640 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb85140 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ad00 tx=0x7f1e6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a71c0cd3ab22ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:18.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.801+0000 7f1e60ff9640 1 -- 172.21.15.64:0/3437245135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6401d070 con 0x55a0fbb7adc0 2024-09-19T05:07:18.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.801+0000 7f1e60ff9640 1 -- 172.21.15.64:0/3437245135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e64018af0 con 0x55a0fbb7adc0 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.801+0000 7f1e60ff9640 1 -- 172.21.15.64:0/3437245135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64010b60 con 0x55a0fbb7adc0 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3437245135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 msgr2=0x55a0fbb85140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e5c0a35e0 0x7f1e5c0a5aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005fd0 tx=0x7f1e6c004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb85140 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ad00 tx=0x7f1e6400f790 comp rx=0 tx=0).stop 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3437245135 shutdown_connections 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb85680 0x55a0fbb87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb85140 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3437245135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb7a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3437245135 >> 172.21.15.64:0/3437245135 conn(0x55a0fbb76a30 msgr2=0x55a0fbb76e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3437245135 shutdown_connections 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3437245135 wait complete. 2024-09-19T05:07:18.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 -- start start 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.802+0000 7f1e7afdf740 1 Processor -- start 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb84f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 0x55a0fbb82750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a0fbb88250 con 0x55a0fbb85680 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0fba9b050 con 0x55a0fbb7adc0 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0fbb883d0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7afdf740 10 client.0 _ll_register_callbacks cb 0x55a0fbadb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:18.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 0x55a0fbb82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 0x55a0fbb82750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60782/0 (socket says 172.21.15.64:60782) 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 -- 172.21.15.64:0/3447356215 learned_addr learned my addr 172.21.15.64:0/3447356215 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e7259c640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb84f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 -- 172.21.15.64:0/3447356215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 msgr2=0x55a0fbb84f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb84f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 -- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7adc0 msgr2=0x55a0fbb82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 -- 172.21.15.64:0/3447356215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fb8513c0 con 0x55a0fbb85680 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 0x55a0fbb82750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c012880 tx=0x7f1e6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d1677cee7f111 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.803+0000 7f1e59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:18.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e59ffb640 1 -- 172.21.15.64:0/3447356215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c01db70 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e59ffb640 1 -- 172.21.15.64:0/3447356215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1e6c01dd10 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e64018af0 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e59ffb640 1 -- 172.21.15.64:0/3447356215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e6c027470 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 10 client.15543 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0fbb93c70 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e59ffb640 1 -- 172.21.15.64:0/3447356215 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e64018af0 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e59ffb640 1 -- 172.21.15.64:0/3447356215 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e6c00f040 con 0x55a0fbb85680 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 10 client.15543 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 20 client.15543 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb95080 0x55a0fbb974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 4 client.15543 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.804+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0fbaed850 con 0x55a0fbb95080 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.805+0000 7f1e7259c640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb95080 0x55a0fbb974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.805+0000 7f1e59ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.805+0000 7f1e7259c640 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb95080 0x55a0fbb974d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bfa6a029aa44759 server_cookie=9bc1a5889db15fea in_seq=0 out_seq=0 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e59ffb640 1 -- 172.21.15.64:0/3447356215 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0fbaed850 con 0x55a0fbb95080 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e59ffb640 10 client.15543 handle_command_reply: tid=0 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 client.15543 shutdown 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 msgr2=0x55a0fbb82750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 0x55a0fbb82750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c012880 tx=0x7f1e6c004ab0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 shutdown_connections 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb95080 0x55a0fbb974d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e59ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb85680 0x55a0fbb82750 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/3447356215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb84f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 >> 172.21.15.64:0/3447356215 conn(0x55a0fbb76a30 msgr2=0x55a0fbb7c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 shutdown_connections 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 1 -- 172.21.15.64:0/3447356215 wait complete. 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.806+0000 7f1e7afdf740 20 client.15543 trim_cache size 0 max 16384 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.831+0000 7f1e7afdf740 1 Processor -- start 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 -- start start 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb76a30 0x55a0fbb914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb9d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741affd0 con 0x55a0fbb7adc0 2024-09-19T05:07:18.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741ad8a0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741ab6f0 con 0x55a0fbb76a30 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb9d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb9d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60786/0 (socket says 172.21.15.64:60786) 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 -- 172.21.15.64:0/1001672455 learned_addr learned my addr 172.21.15.64:0/1001672455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7259c640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb76a30 0x55a0fbb914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 -- 172.21.15.64:0/1001672455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb76a30 msgr2=0x55a0fbb914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb76a30 0x55a0fbb914b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 -- 172.21.15.64:0/1001672455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 msgr2=0x55a0fbb9cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb9cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e72d9d640 1 -- 172.21.15.64:0/1001672455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0fbb7abf0 con 0x55a0fbb7adc0 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:18.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.832+0000 7f1e7259c640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb76a30 0x55a0fbb914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.833+0000 7f1e72d9d640 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb9d2e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011c80 tx=0x7f1e6c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e2a54c041c103b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.833+0000 7f1e5bfff640 1 -- 172.21.15.64:0/1001672455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c01d7b0 con 0x55a0fbb7adc0 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.833+0000 7f1e5bfff640 1 -- 172.21.15.64:0/1001672455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e6c022070 con 0x55a0fbb7adc0 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.833+0000 7f1e5bfff640 1 -- 172.21.15.64:0/1001672455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c027970 con 0x55a0fbb7adc0 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.833+0000 7f1e7afdf740 1 -- 172.21.15.64:0/1001672455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 msgr2=0x55a0fbb9d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.833+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb9d2e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011c80 tx=0x7f1e6c005cb0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- 172.21.15.64:0/1001672455 shutdown_connections 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb9d2e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/1001672455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb76a30 0x55a0fbb914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- 172.21.15.64:0/1001672455 >> 172.21.15.64:0/1001672455 conn(0x55a0fbb828b0 msgr2=0x55a0fbb80ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- 172.21.15.64:0/1001672455 shutdown_connections 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- 172.21.15.64:0/1001672455 wait complete. 2024-09-19T05:07:18.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- start start 2024-09-19T05:07:18.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 Processor -- start 2024-09-19T05:07:18.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb76a30 0x55a0fbbfb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb8d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb8dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741abd60 con 0x55a0fbb76a30 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741ad1d0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e741ac430 con 0x55a0fbb7adc0 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.834+0000 7f1e7afdf740 10 client.0 _ll_register_callbacks cb 0x7f1e6c011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb8d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb8d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58778/0 (socket says 172.21.15.64:58778) 2024-09-19T05:07:18.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 -- 172.21.15.64:0/2826385510 learned_addr learned my addr 172.21.15.64:0/2826385510 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e7259c640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb76a30 0x55a0fbbfb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 -- 172.21.15.64:0/2826385510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7adc0 msgr2=0x55a0fbb8dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb8dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 -- 172.21.15.64:0/2826385510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb76a30 msgr2=0x55a0fbbfb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb76a30 0x55a0fbbfb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 -- 172.21.15.64:0/2826385510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fbbf8ba0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e7259c640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb76a30 0x55a0fbbfb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e71d9b640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb8d620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e680bc5a0 tx=0x7f1e680bc4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45ecef2e7519592 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:18.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:07:18.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e59ffb640 1 -- 172.21.15.64:0/2826385510 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68049040 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e6c022070 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.835+0000 7f1e59ffb640 1 -- 172.21.15.64:0/2826385510 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1e680bcb00 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e59ffb640 1 -- 172.21.15.64:0/2826385510 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e680044e0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 10 client.25262 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:18.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0fbc04460 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e59ffb640 1 -- 172.21.15.64:0/2826385510 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e6c022070 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e59ffb640 1 -- 172.21.15.64:0/2826385510 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e680bdbd0 con 0x55a0fbb7a4a0 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 10 client.25262 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 10 client.25262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 20 client.25262 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb85680 0x55a0fbc06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:18.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 4 client.25262 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0fbae7d60 con 0x55a0fbb85680 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.836+0000 7f1e7259c640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb85680 0x55a0fbc06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7259c640 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb85680 0x55a0fbc06b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb636df0ac05890 server_cookie=1f7ff770969cbba1 in_seq=0 out_seq=0 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e59ffb640 10 client.25262 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e59ffb640 1 -- 172.21.15.64:0/2826385510 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0fbae7d60 con 0x55a0fbb85680 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e59ffb640 10 client.25262 handle_command_reply: tid=0 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 client.25262 shutdown 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 msgr2=0x55a0fbb8d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb8d620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e680bc5a0 tx=0x7f1e680bc4a0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 shutdown_connections 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0fbb7adc0 0x55a0fbb8dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0fbb7a4a0 0x55a0fbb8d620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55a0fbb85680 0x55a0fbc06b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 --2- 172.21.15.64:0/2826385510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0fbb76a30 0x55a0fbbfb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e59ffb640 0 client.25262 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 >> 172.21.15.64:0/2826385510 conn(0x55a0fbb828b0 msgr2=0x55a0fbb83a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 shutdown_connections 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.837+0000 7f1e7afdf740 1 -- 172.21.15.64:0/2826385510 wait complete. 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e7afdf740 20 client.25262 trim_cache size 0 max 16384 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e5c0a35e0 msgr2=0x7f1e5c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e5c0a35e0 0x7f1e5c0a5aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005fd0 tx=0x7f1e6c004a40 comp rx=0 tx=0).stop 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 msgr2=0x7f1e74198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e74198920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006010 tx=0x7f1e680042f0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 shutdown_connections 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e5c0a35e0 0x7f1e5c0a5aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e74102ef0 0x7f1e7419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e74102570 0x7f1e74198920 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 --2- 172.21.15.64:0/1062003133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e74100e20 0x7f1e741983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 >> 172.21.15.64:0/1062003133 conn(0x7f1e740fc7d0 msgr2=0x7f1e7406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 shutdown_connections 2024-09-19T05:07:18.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:18.838+0000 7f1e78c06640 1 -- 172.21.15.64:0/1062003133 wait complete. 2024-09-19T05:07:19.696 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:19 smithi064 ceph-mon[30960]: pgmap v45: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-19T05:07:19.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:07:19.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:07:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:19 smithi103 ceph-mon[34446]: pgmap v45: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-19T05:07:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:19 smithi174 ceph-mon[34488]: pgmap v45: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-19T05:07:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 -- 172.21.15.64:0/4241709326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c102d30 msgr2=0x7f5d9c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 --2- 172.21.15.64:0/4241709326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c102d30 0x7f5d9c103110 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009930 tx=0x7f5d9802a440 comp rx=0 tx=0).stop 2024-09-19T05:07:20.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 -- 172.21.15.64:0/4241709326 shutdown_connections 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 --2- 172.21.15.64:0/4241709326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 --2- 172.21.15.64:0/4241709326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 --2- 172.21.15.64:0/4241709326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c102d30 0x7f5d9c103110 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 -- 172.21.15.64:0/4241709326 >> 172.21.15.64:0/4241709326 conn(0x7f5d9c0fc7d0 msgr2=0x7f5d9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 -- 172.21.15.64:0/4241709326 shutdown_connections 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 -- 172.21.15.64:0/4241709326 wait complete. 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.472+0000 7f5da3f54640 1 Processor -- start 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 -- start start 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9c102d30 0x7f5d9c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:20.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c111f90 con 0x7f5d9c1082a0 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c111e10 con 0x7f5d9c1036e0 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da3f54640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c112110 con 0x7f5d9c102d30 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60800/0 (socket says 172.21.15.64:60800) 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 -- 172.21.15.64:0/3523311956 learned_addr learned my addr 172.21.15.64:0/3523311956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da1cf0640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9c102d30 0x7f5d9c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da14ef640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 -- 172.21.15.64:0/3523311956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9c102d30 msgr2=0x7f5d9c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9c102d30 0x7f5d9c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 -- 172.21.15.64:0/3523311956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1036e0 msgr2=0x7f5d9c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da24f1640 1 -- 172.21.15.64:0/3523311956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c19d480 con 0x7f5d9c1082a0 2024-09-19T05:07:20.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.473+0000 7f5da14ef640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.474+0000 7f5da24f1640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c19cda0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400cce0 tx=0x7f5d9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.474+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400f450 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.474+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9400faa0 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.474+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c19d770 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.474+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94018b70 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.474+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a5050 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.475+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c06a850 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.476+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7f5d94018d10 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.477+0000 7f5d92ffd640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5d880a35e0 0x7f5d880a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.477+0000 7f5da1cf0640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5d880a35e0 0x7f5d880a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.477+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5d940d0540 con 0x7f5d9c1082a0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.478+0000 7f5da1cf0640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5d880a35e0 0x7f5d880a5aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d98031400 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:20.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.482+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d94014030 con 0x7f5d9c1082a0 2024-09-19T05:07:20.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.739+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d9c103110 con 0x7f5d9c1082a0 2024-09-19T05:07:20.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.740+0000 7f5d92ffd640 1 -- 172.21.15.64:0/3523311956 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f5d9c103110 con 0x7f5d9c1082a0 2024-09-19T05:07:20.741 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:07:20.741 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:07:20.742 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:07:20.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5d880a35e0 msgr2=0x7f5d880a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:20.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5d880a35e0 0x7f5d880a5aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d98031400 comp rx=0 tx=0).stop 2024-09-19T05:07:20.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 msgr2=0x7f5d9c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c19cda0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400cce0 tx=0x7f5d9400a620 comp rx=0 tx=0).stop 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 shutdown_connections 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5d880a35e0 0x7f5d880a5aa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d9c1082a0 0x7f5d9c19cda0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 --2- 172.21.15.64:0/3523311956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9c102d30 0x7f5d9c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:20.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 >> 172.21.15.64:0/3523311956 conn(0x7f5d9c0fc7d0 msgr2=0x7f5d9c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:20.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.742+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 shutdown_connections 2024-09-19T05:07:20.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:20.743+0000 7f5da3f54640 1 -- 172.21.15.64:0/3523311956 wait complete. 2024-09-19T05:07:21.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:07:21.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:21 smithi103 ceph-mon[34446]: pgmap v46: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-19T05:07:21.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:21 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3523311956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:21 smithi174 ceph-mon[34488]: pgmap v46: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-19T05:07:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:21 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3523311956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:21.885 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:21.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:21 smithi064 ceph-mon[30960]: pgmap v46: 97 pgs: 97 active+clean; 1.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 1.42k op/s 2024-09-19T05:07:21.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:21 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3523311956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.382+0000 7ff989dcd640 1 -- 172.21.15.64:0/2930894933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984100e20 msgr2=0x7ff984101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.382+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2930894933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984100e20 0x7ff984101200 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff974009900 tx=0x7ff97402a480 comp rx=0 tx=0).stop 2024-09-19T05:07:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 -- 172.21.15.64:0/2930894933 shutdown_connections 2024-09-19T05:07:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2930894933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984102ef0 0x7ff98410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2930894933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff984102570 0x7ff9841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2930894933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984100e20 0x7ff984101200 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 -- 172.21.15.64:0/2930894933 >> 172.21.15.64:0/2930894933 conn(0x7ff9840fc7d0 msgr2=0x7ff9840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 -- 172.21.15.64:0/2930894933 shutdown_connections 2024-09-19T05:07:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 -- 172.21.15.64:0/2930894933 wait complete. 2024-09-19T05:07:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.383+0000 7ff989dcd640 1 Processor -- start 2024-09-19T05:07:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 -- start start 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984100e20 0x7ff984198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 0x7ff984198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff984102ef0 0x7ff98419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff984110e50 con 0x7ff984102570 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff984110cd0 con 0x7ff984102ef0 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff989dcd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff984110fd0 con 0x7ff984100e20 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 0x7ff984198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 0x7ff984198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60814/0 (socket says 172.21.15.64:60814) 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 -- 172.21.15.64:0/2590999049 learned_addr learned my addr 172.21.15.64:0/2590999049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff983fff640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff984102ef0 0x7ff98419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff9837fe640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984100e20 0x7ff984198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 -- 172.21.15.64:0/2590999049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984100e20 msgr2=0x7ff984198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984100e20 0x7ff984198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 -- 172.21.15.64:0/2590999049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff984102ef0 msgr2=0x7ff98419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff984102ef0 0x7ff98419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.384+0000 7ff982ffd640 1 -- 172.21.15.64:0/2590999049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98419d3f0 con 0x7ff984102570 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff9837fe640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984100e20 0x7ff984198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff982ffd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 0x7ff984198980 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d950 tx=0x7ff97800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff980ff9640 1 -- 172.21.15.64:0/2590999049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978017070 con 0x7ff984102570 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff980ff9640 1 -- 172.21.15.64:0/2590999049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff978009c30 con 0x7ff984102570 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98419d6e0 con 0x7ff984102570 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff980ff9640 1 -- 172.21.15.64:0/2590999049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97800f620 con 0x7ff984102570 2024-09-19T05:07:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.385+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9841a4fc0 con 0x7ff984102570 2024-09-19T05:07:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.387+0000 7ff980ff9640 1 -- 172.21.15.64:0/2590999049 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7ff9780109a0 con 0x7ff984102570 2024-09-19T05:07:22.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.388+0000 7ff98c1a6740 1 Processor -- start 2024-09-19T05:07:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.388+0000 7ff980ff9640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff96c0a35e0 0x7ff96c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.388+0000 7ff98c1a6740 1 -- start start 2024-09-19T05:07:22.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.388+0000 7ff980ff9640 1 -- 172.21.15.64:0/2590999049 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff9780cfbd0 con 0x7ff984102570 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.388+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b45b40 0x562d59b44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff983fff640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff96c0a35e0 0x7ff96c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b53780 0x562d59b55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d59aaa2f0 con 0x562d59b445d0 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d59a51cf0 con 0x562d59b53780 2024-09-19T05:07:22.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d59aa62b0 con 0x562d59b45b40 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b44a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60828/0 (socket says 172.21.15.64:60828) 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 -- 172.21.15.64:0/3606883493 learned_addr learned my addr 172.21.15.64:0/3606883493 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b45b40 0x562d59b44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 -- 172.21.15.64:0/3606883493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b45b40 msgr2=0x562d59b44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b45b40 0x562d59b44090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 -- 172.21.15.64:0/3606883493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b53780 msgr2=0x562d59b55b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b53780 0x562d59b55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 -- 172.21.15.64:0/3606883493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d59a363a0 con 0x562d59b445d0 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff983fff640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff96c0a35e0 0x7ff96c0a5aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009f70 tx=0x7ff97c009650 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:22.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b44a10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9780980d0 tx=0x7ff97809bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ce69938db3b667 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.389+0000 7ff969ffb640 1 -- 172.21.15.64:0/3606883493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978020040 con 0x562d59b445d0 2024-09-19T05:07:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.390+0000 7ff969ffb640 1 -- 172.21.15.64:0/3606883493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9780a0020 con 0x562d59b445d0 2024-09-19T05:07:22.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.390+0000 7ff969ffb640 1 -- 172.21.15.64:0/3606883493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9780979c0 con 0x562d59b445d0 2024-09-19T05:07:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.390+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3606883493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 msgr2=0x562d59b44a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.390+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b44a10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9780980d0 tx=0x7ff97809bb70 comp rx=0 tx=0).stop 2024-09-19T05:07:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.390+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3606883493 shutdown_connections 2024-09-19T05:07:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.390+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b53780 0x562d59b55b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b44a10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3606883493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b45b40 0x562d59b44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3606883493 >> 172.21.15.64:0/3606883493 conn(0x562d59b0d310 msgr2=0x562d59b40c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3606883493 shutdown_connections 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3606883493 wait complete. 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 -- start start 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.391+0000 7ff98c1a6740 1 Processor -- start 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b45b40 0x562d59b50460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d59b563c0 con 0x562d59b445d0 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d59a65060 con 0x562d59b45b40 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d59b56540 con 0x562d59b53780 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60840/0 (socket says 172.21.15.64:60840) 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 -- 172.21.15.64:0/3299723154 learned_addr learned my addr 172.21.15.64:0/3299723154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff98c1a6740 10 client.0 _ll_register_callbacks cb 0x562d597b88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff983fff640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b50bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b45b40 0x562d59b50460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 -- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 msgr2=0x562d59b50bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b50bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 -- 172.21.15.64:0/3299723154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b45b40 msgr2=0x562d59b50460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b45b40 0x562d59b50460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 -- 172.21.15.64:0/3299723154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5981b3c0 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b45b40 0x562d59b50460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff983fff640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b531b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff974009900 tx=0x7ff9740316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142d111164c339e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.392+0000 7ff962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff962ffd640 1 -- 172.21.15.64:0/3299723154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97403f070 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff962ffd640 1 -- 172.21.15.64:0/3299723154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff974004450 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9780a0020 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff962ffd640 1 -- 172.21.15.64:0/3299723154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff974035070 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff98c1a6740 10 client.15567 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d59b61e80 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.393+0000 7ff962ffd640 1 -- 172.21.15.64:0/3299723154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9740054e0 con 0x562d59b445d0 2024-09-19T05:07:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff962ffd640 1 -- 172.21.15.64:0/3299723154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9740035f0 con 0x562d59b445d0 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff98c1a6740 10 client.15567 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff98c1a6740 10 client.15567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff98c1a6740 20 client.15567 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59b63340 0x562d59b65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff98c1a6740 4 client.15567 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff9780a0020 con 0x562d59b63340 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.394+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59b63340 0x562d59b65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.395+0000 7ff962ffd640 10 client.15567 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.395+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59b63340 0x562d59b65770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c37e879941f9d63e server_cookie=fdc47ec5e7afcdac in_seq=0 out_seq=0 2024-09-19T05:07:22.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.395+0000 7ff962ffd640 1 -- 172.21.15.64:0/3299723154 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff9780a0020 con 0x562d59b63340 2024-09-19T05:07:22.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.395+0000 7ff962ffd640 10 client.15567 handle_command_reply: tid=0 2024-09-19T05:07:22.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.395+0000 7ff98c1a6740 1 client.15567 shutdown 2024-09-19T05:07:22.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 msgr2=0x562d59b531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b531b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff974009900 tx=0x7ff9740316c0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 shutdown_connections 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59b63340 0x562d59b65770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b45b40 0x562d59b50460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3299723154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b445d0 0x562d59b531b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 >> 172.21.15.64:0/3299723154 conn(0x562d59b0d310 msgr2=0x562d59b46990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff962ffd640 0 client.15567 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 shutdown_connections 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3299723154 wait complete. 2024-09-19T05:07:22.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.396+0000 7ff98c1a6740 20 client.15567 trim_cache size 0 max 16384 2024-09-19T05:07:22.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.411+0000 7ff98c1a6740 1 Processor -- start 2024-09-19T05:07:22.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.411+0000 7ff98c1a6740 1 -- start start 2024-09-19T05:07:22.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b6b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b6b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841ade90 con 0x562d59b45b40 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841ab760 con 0x562d59b445d0 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841a95b0 con 0x562d59b53780 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b6b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b6b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60854/0 (socket says 172.21.15.64:60854) 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 -- 172.21.15.64:0/856737122 learned_addr learned my addr 172.21.15.64:0/856737122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff9837fe640 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff983fff640 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b6b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 -- 172.21.15.64:0/856737122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 msgr2=0x562d59b6b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b6b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 -- 172.21.15.64:0/856737122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 msgr2=0x562d59b40e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b40e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 -- 172.21.15.64:0/856737122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d59b513f0 con 0x562d59b45b40 2024-09-19T05:07:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.412+0000 7ff982ffd640 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b6b040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff97809bc00 tx=0x7ff97800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad8a52b1f2664dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff9697fa640 1 -- 172.21.15.64:0/856737122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978020050 con 0x562d59b45b40 2024-09-19T05:07:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff9697fa640 1 -- 172.21.15.64:0/856737122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9780982e0 con 0x562d59b45b40 2024-09-19T05:07:22.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff9697fa640 1 -- 172.21.15.64:0/856737122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978097c50 con 0x562d59b45b40 2024-09-19T05:07:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 -- 172.21.15.64:0/856737122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 msgr2=0x562d59b6b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b6b040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff97809bc00 tx=0x7ff97800bf60 comp rx=0 tx=0).stop 2024-09-19T05:07:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 -- 172.21.15.64:0/856737122 shutdown_connections 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b6b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b6b040 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/856737122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b40e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.413+0000 7ff98c1a6740 1 -- 172.21.15.64:0/856737122 >> 172.21.15.64:0/856737122 conn(0x562d59b0d310 msgr2=0x562d59b40700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 -- 172.21.15.64:0/856737122 shutdown_connections 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 -- 172.21.15.64:0/856737122 wait complete. 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 -- start start 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 Processor -- start 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b5bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b57010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b57550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841a9c20 con 0x562d59b45b40 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841ab090 con 0x562d59b445d0 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.414+0000 7ff98c1a6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841aa2f0 con 0x562d59b53780 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff98c1a6740 10 client.0 _ll_register_callbacks cb 0x7ff974009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b57010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b57010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60870/0 (socket says 172.21.15.64:60870) 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 -- 172.21.15.64:0/3459291454 learned_addr learned my addr 172.21.15.64:0/3459291454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b5bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff983fff640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b57550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 -- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 msgr2=0x562d59b57550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b57550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 -- 172.21.15.64:0/3459291454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 msgr2=0x562d59b5bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b5bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 -- 172.21.15.64:0/3459291454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d59b57b10 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff982ffd640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b57010 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff97809c520 tx=0x7ff978098580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2df0c17b7ac398c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.415+0000 7ff962ffd640 1 -- 172.21.15.64:0/3459291454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978097c30 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff962ffd640 1 -- 172.21.15.64:0/3459291454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff97809b0c0 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9780982e0 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff962ffd640 1 -- 172.21.15.64:0/3459291454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff978020030 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 10 client.15573 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d59bcdde0 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff962ffd640 1 -- 172.21.15.64:0/3459291454 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9780982e0 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff962ffd640 1 -- 172.21.15.64:0/3459291454 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff978010460 con 0x562d59b45b40 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 10 client.15573 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 10 client.15573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 20 client.15573 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59bce060 0x562d59bd0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 4 client.15573 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.416+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d59ab7860 con 0x562d59bce060 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.417+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59bce060 0x562d59bd0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.417+0000 7ff962ffd640 10 client.15573 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.417+0000 7ff9837fe640 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59bce060 0x562d59bd0450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46959a62754056f9 server_cookie=13f839344b06bcd0 in_seq=0 out_seq=0 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff962ffd640 1 -- 172.21.15.64:0/3459291454 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d59ab7860 con 0x562d59bce060 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff962ffd640 10 client.15573 handle_command_reply: tid=0 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 client.15573 shutdown 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 msgr2=0x562d59b57010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b57010 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff97809c520 tx=0x7ff978098580 comp rx=0 tx=0).stop 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 shutdown_connections 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562d59bce060 0x562d59bd0450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d59b53780 0x562d59b57550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d59b45b40 0x562d59b57010 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 --2- 172.21.15.64:0/3459291454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d59b445d0 0x562d59b5bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 >> 172.21.15.64:0/3459291454 conn(0x562d59b0d310 msgr2=0x562d59b46110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff962ffd640 0 client.15573 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 shutdown_connections 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 1 -- 172.21.15.64:0/3459291454 wait complete. 2024-09-19T05:07:22.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.418+0000 7ff98c1a6740 20 client.15573 trim_cache size 0 max 16384 2024-09-19T05:07:22.425 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:07:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff96c0a35e0 msgr2=0x7ff96c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff96c0a35e0 0x7ff96c0a5aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009f70 tx=0x7ff97c009650 comp rx=0 tx=0).stop 2024-09-19T05:07:22.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 msgr2=0x7ff984198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 0x7ff984198980 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d950 tx=0x7ff97800de10 comp rx=0 tx=0).stop 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 shutdown_connections 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff96c0a35e0 0x7ff96c0a5aa0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff984102ef0 0x7ff98419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984102570 0x7ff984198980 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 --2- 172.21.15.64:0/2590999049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff984100e20 0x7ff984198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 >> 172.21.15.64:0/2590999049 conn(0x7ff9840fc7d0 msgr2=0x7ff984069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 shutdown_connections 2024-09-19T05:07:22.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:22.419+0000 7ff989dcd640 1 -- 172.21.15.64:0/2590999049 wait complete. 2024-09-19T05:07:23.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:07:23.502 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:23.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:23 smithi103 ceph-mon[34446]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 1.88k op/s 2024-09-19T05:07:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:23 smithi174 ceph-mon[34488]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 1.88k op/s 2024-09-19T05:07:23.778 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:23 smithi064 ceph-mon[30960]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 1.88k op/s 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 -- 172.21.15.64:0/2705595726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88107ad0 msgr2=0x7f7c88107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/2705595726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88107ad0 0x7f7c88107eb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009930 tx=0x7f7c7c02a420 comp rx=0 tx=0).stop 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 -- 172.21.15.64:0/2705595726 shutdown_connections 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/2705595726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c88102460 0x7f7c8810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/2705595726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c88101ac0 0x7f7c88101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/2705595726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88107ad0 0x7f7c88107eb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.789+0000 7f7c8ef18640 1 -- 172.21.15.64:0/2705595726 >> 172.21.15.64:0/2705595726 conn(0x7f7c880fd820 msgr2=0x7f7c880ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 -- 172.21.15.64:0/2705595726 shutdown_connections 2024-09-19T05:07:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 -- 172.21.15.64:0/2705595726 wait complete. 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 Processor -- start 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 -- start start 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c88101ac0 0x7f7c88198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 0x7f7c881987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c88107ad0 0x7f7c8819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111dd0 con 0x7f7c88102460 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111c50 con 0x7f7c88107ad0 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.790+0000 7f7c8ef18640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111f50 con 0x7f7c88101ac0 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 0x7f7c881987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 0x7f7c881987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60886/0 (socket says 172.21.15.64:60886) 2024-09-19T05:07:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 -- 172.21.15.64:0/3688693192 learned_addr learned my addr 172.21.15.64:0/3688693192 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c88107ad0 0x7f7c8819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c88101ac0 0x7f7c88198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 -- 172.21.15.64:0/3688693192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c88101ac0 msgr2=0x7f7c88198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c88101ac0 0x7f7c88198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 -- 172.21.15.64:0/3688693192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c88107ad0 msgr2=0x7f7c8819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c88107ad0 0x7f7c8819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 -- 172.21.15.64:0/3688693192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8819d280 con 0x7f7c88102460 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c87fff640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 0x7f7c881987d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400e9f0 tx=0x7f7c7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c85ffb640 1 -- 172.21.15.64:0/3688693192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400cd80 con 0x7f7c88102460 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c85ffb640 1 -- 172.21.15.64:0/3688693192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c740108e0 con 0x7f7c88102460 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8819d570 con 0x7f7c88102460 2024-09-19T05:07:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.791+0000 7f7c85ffb640 1 -- 172.21.15.64:0/3688693192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74018920 con 0x7f7c88102460 2024-09-19T05:07:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.792+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c881a4e00 con 0x7f7c88102460 2024-09-19T05:07:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.795+0000 7f7c85ffb640 1 -- 172.21.15.64:0/3688693192 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7f7c74010a90 con 0x7f7c88102460 2024-09-19T05:07:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 Processor -- start 2024-09-19T05:07:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 -- start start 2024-09-19T05:07:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c85ffb640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7c780a3690 0x7f7c780a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c8d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c8d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256d6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60900/0 (socket says 172.21.15.64:60900) 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/330153313 learned_addr learned my addr 172.21.15.64:0/330153313 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256e1cb0 0x55d0256e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d02563b2f0 con 0x55d0256d6af0 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0255e3270 con 0x55d0256d7410 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0256372b0 con 0x55d0256e1cb0 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c85ffb640 1 -- 172.21.15.64:0/3688693192 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7c740d01e0 con 0x7f7c88102460 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.796+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/330153313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256e1cb0 msgr2=0x55d0256e40a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:07:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256e1cb0 0x55d0256e40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/330153313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 msgr2=0x55d0256e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/330153313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0255e2340 con 0x55d0256d6af0 2024-09-19T05:07:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c87fff640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7c780a3690 0x7f7c780a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256d6ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009f50 tx=0x7f7c800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534ed8396d946c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c72ffd640 1 -- 172.21.15.64:0/330153313 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004b20 con 0x55d0256d6af0 2024-09-19T05:07:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c72ffd640 1 -- 172.21.15.64:0/330153313 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c80004cc0 con 0x55d0256d6af0 2024-09-19T05:07:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.797+0000 7f7c72ffd640 1 -- 172.21.15.64:0/330153313 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8000e6d0 con 0x55d0256d6af0 2024-09-19T05:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 msgr2=0x55d0256d6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256d6ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009f50 tx=0x7f7c800118d0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c87fff640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7c780a3690 0x7f7c780a5b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c740cd790 tx=0x7f7c7400cf90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 shutdown_connections 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256e1cb0 0x55d0256e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/330153313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256d6ed0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 >> 172.21.15.64:0/330153313 conn(0x55d0256d3060 msgr2=0x55d0256d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 shutdown_connections 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.798+0000 7f7c91cb1740 1 -- 172.21.15.64:0/330153313 wait complete. 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.799+0000 7f7c91cb1740 1 -- start start 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.799+0000 7f7c91cb1740 1 Processor -- start 2024-09-19T05:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 0x55d0256df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d0256e8c90 con 0x55d0256e1cb0 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0255c4290 con 0x55d0256d7410 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0256e8e10 con 0x55d0256d6af0 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c91cb1740 10 client.0 _ll_register_callbacks cb 0x55d0255f5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 0x55d0256df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 0x55d0256df600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60912/0 (socket says 172.21.15.64:60912) 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/1331984184 learned_addr learned my addr 172.21.15.64:0/1331984184 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/1331984184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 msgr2=0x55d0256deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 msgr2=0x55d0256df0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.800+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256df0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/1331984184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02534e210 con 0x55d0256e1cb0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 0x55d0256df600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c8001b040 tx=0x7f7c80000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795f33d5cd0d429a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c67fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c67fff640 1 -- 172.21.15.64:0/1331984184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80003970 con 0x55d0256e1cb0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c67fff640 1 -- 172.21.15.64:0/1331984184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7c80003b10 con 0x55d0256e1cb0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c80004cc0 con 0x55d0256e1cb0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c67fff640 1 -- 172.21.15.64:0/1331984184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c80003d10 con 0x55d0256e1cb0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c91cb1740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.801+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d02541a620 con 0x55d0256e1cb0 2024-09-19T05:07:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c67fff640 1 -- 172.21.15.64:0/1331984184 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c80004cc0 con 0x55d0256e1cb0 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c67fff640 1 -- 172.21.15.64:0/1331984184 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c80004220 con 0x55d0256e1cb0 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c91cb1740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c91cb1740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c91cb1740 20 client.15585 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d0256f5f40 0x55d0256f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c91cb1740 4 client.15585 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d025648a20 con 0x55d0256f5f40 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.802+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d0256f5f40 0x55d0256f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.803+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d0256f5f40 0x55d0256f8330 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d77be1c2aed8ab server_cookie=b7a8ae4267f1304d in_seq=0 out_seq=0 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.803+0000 7f7c67fff640 10 client.15585 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.803+0000 7f7c67fff640 1 -- 172.21.15.64:0/1331984184 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d025648a20 con 0x55d0256f5f40 2024-09-19T05:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.803+0000 7f7c67fff640 10 client.15585 handle_command_reply: tid=0 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.803+0000 7f7c91cb1740 1 client.15585 shutdown 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 msgr2=0x55d0256df600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 0x55d0256df600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c8001b040 tx=0x7f7c80000f40 comp rx=0 tx=0).stop 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 shutdown_connections 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d0256f5f40 0x55d0256f8330 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256e1cb0 0x55d0256df600 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256d7410 0x55d0256df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/1331984184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 >> 172.21.15.64:0/1331984184 conn(0x55d0256d3060 msgr2=0x55d0256d8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c67fff640 0 client.15585 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 shutdown_connections 2024-09-19T05:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 1 -- 172.21.15.64:0/1331984184 wait complete. 2024-09-19T05:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.804+0000 7f7c91cb1740 20 client.15585 trim_cache size 0 max 16384 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.818+0000 7f7c91cb1740 1 Processor -- start 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.818+0000 7f7c91cb1740 1 -- start start 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256e0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d7410 0x55d0256fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881adf70 con 0x55d0256d6af0 2024-09-19T05:07:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881ab840 con 0x55d0256e1cb0 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881a9690 con 0x55d0256d7410 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256e0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256e0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60918/0 (socket says 172.21.15.64:60918) 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/2000453847 learned_addr learned my addr 172.21.15.64:0/2000453847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/2000453847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d7410 msgr2=0x55d0256fd9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d7410 0x55d0256fd9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/2000453847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 msgr2=0x55d0256fdf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256fdf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.819+0000 7f7c8d4b5640 1 -- 172.21.15.64:0/2000453847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0256d2e90 con 0x55d0256d6af0 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.820+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256e0af0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c80011910 tx=0x7f7c8000e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42d316095575cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.820+0000 7f7c727fc640 1 -- 172.21.15.64:0/2000453847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8000ee60 con 0x55d0256d6af0 2024-09-19T05:07:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.820+0000 7f7c727fc640 1 -- 172.21.15.64:0/2000453847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c80004360 con 0x55d0256d6af0 2024-09-19T05:07:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.820+0000 7f7c727fc640 1 -- 172.21.15.64:0/2000453847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80027730 con 0x55d0256d6af0 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2000453847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 msgr2=0x55d0256e0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256e0af0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c80011910 tx=0x7f7c8000e800 comp rx=0 tx=0).stop 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2000453847 shutdown_connections 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256fdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d7410 0x55d0256fd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2000453847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d6af0 0x55d0256e0af0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2000453847 >> 172.21.15.64:0/2000453847 conn(0x55d0256d3060 msgr2=0x55d0256e03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2000453847 shutdown_connections 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2000453847 wait complete. 2024-09-19T05:07:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.821+0000 7f7c91cb1740 1 -- start start 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 Processor -- start 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 0x55d0256e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881a9d00 con 0x55d0256d7410 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881ab170 con 0x55d0256e1cb0 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c91cb1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c881aa3d0 con 0x55d0256d6af0 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.822+0000 7f7c8ccb4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 0x55d0256e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 0x55d0256e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60922/0 (socket says 172.21.15.64:60922) 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 -- 172.21.15.64:0/2287277908 learned_addr learned my addr 172.21.15.64:0/2287277908 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c91cb1740 10 client.0 _ll_register_callbacks cb 0x7f7c80009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 -- 172.21.15.64:0/2287277908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 msgr2=0x55d0256ee5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256ee5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 -- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 msgr2=0x55d0256e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 -- 172.21.15.64:0/2287277908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0256d4110 con 0x55d0256d7410 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c8ccb4640 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 0x55d0256e9910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02a460 tx=0x7f7c7c004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40c37786bedca45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c67fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c67fff640 1 -- 172.21.15.64:0/2287277908 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c035040 con 0x55d0256d7410 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c67fff640 1 -- 172.21.15.64:0/2287277908 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7c7c005450 con 0x55d0256d7410 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.823+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c80004360 con 0x55d0256d7410 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c67fff640 1 -- 172.21.15.64:0/2287277908 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c7c053050 con 0x55d0256d7410 2024-09-19T05:07:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 10 client.15591 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0257605d0 con 0x55d0256d7410 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c67fff640 1 -- 172.21.15.64:0/2287277908 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c80004360 con 0x55d0256d7410 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c67fff640 1 -- 172.21.15.64:0/2287277908 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7c7c0455e0 con 0x55d0256d7410 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 10 client.15591 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 20 client.15591 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d025760d80 0x55d0257631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 4 client.15591 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.824+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d025642d70 con 0x55d025760d80 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.825+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d025760d80 0x55d0257631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.825+0000 7f7c67fff640 10 client.15591 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.825+0000 7f7c8d4b5640 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d025760d80 0x55d0257631a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcee2d0bbde99c94 server_cookie=123ea5a2d4de9e7c in_seq=0 out_seq=0 2024-09-19T05:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c67fff640 1 -- 172.21.15.64:0/2287277908 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d025642d70 con 0x55d025760d80 2024-09-19T05:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c67fff640 10 client.15591 handle_command_reply: tid=0 2024-09-19T05:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 client.15591 shutdown 2024-09-19T05:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 msgr2=0x55d0256e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 0x55d0256e9910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c02a460 tx=0x7f7c7c004420 comp rx=0 tx=0).stop 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 shutdown_connections 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0256e1cb0 0x55d0256e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d025760d80 0x55d0257631a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0256d7410 0x55d0256e9910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c67fff640 0 client.15591 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 --2- 172.21.15.64:0/2287277908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0256d6af0 0x55d0256ee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.840+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 >> 172.21.15.64:0/2287277908 conn(0x55d0256d3060 msgr2=0x55d0256d8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 shutdown_connections 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c91cb1740 1 -- 172.21.15.64:0/2287277908 wait complete. 2024-09-19T05:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c91cb1740 20 client.15591 trim_cache size 0 max 16384 2024-09-19T05:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7c780a3690 msgr2=0x7f7c780a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7c780a3690 0x7f7c780a5b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c740cd790 tx=0x7f7c7400cf90 comp rx=0 tx=0).stop 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 msgr2=0x7f7c881987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 0x7f7c881987d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400e9f0 tx=0x7f7c7400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 shutdown_connections 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7c780a3690 0x7f7c780a5b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c88107ad0 0x7f7c8819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c88102460 0x7f7c881987d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 --2- 172.21.15.64:0/3688693192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c88101ac0 0x7f7c88198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.841+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 >> 172.21.15.64:0/3688693192 conn(0x7f7c880fd820 msgr2=0x7f7c880fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.842+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 shutdown_connections 2024-09-19T05:07:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:23.842+0000 7f7c8ef18640 1 -- 172.21.15.64:0/3688693192 wait complete. 2024-09-19T05:07:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.012+0000 7f4d083f7640 1 -- 172.21.15.64:0/3512801117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00102d90 msgr2=0x7f4d00103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 --2- 172.21.15.64:0/3512801117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00102d90 0x7f4d00103170 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009930 tx=0x7f4cfc02a420 comp rx=0 tx=0).stop 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 -- 172.21.15.64:0/3512801117 shutdown_connections 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 --2- 172.21.15.64:0/3512801117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d00108290 0x7f4d0010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 --2- 172.21.15.64:0/3512801117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d00103740 0x7f4d00107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 --2- 172.21.15.64:0/3512801117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00102d90 0x7f4d00103170 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 -- 172.21.15.64:0/3512801117 >> 172.21.15.64:0/3512801117 conn(0x7f4d000fc7d0 msgr2=0x7f4d000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 -- 172.21.15.64:0/3512801117 shutdown_connections 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 -- 172.21.15.64:0/3512801117 wait complete. 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.013+0000 7f4d083f7640 1 Processor -- start 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 -- start start 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d00102d90 0x7f4d001009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 0x7f4d00100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d00108290 0x7f4d00101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00111f40 con 0x7f4d00103740 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00111dc0 con 0x7f4d00108290 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d083f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001120c0 con 0x7f4d00102d90 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 0x7f4d00100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 0x7f4d00100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60950/0 (socket says 172.21.15.64:60950) 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 -- 172.21.15.64:0/1611887800 learned_addr learned my addr 172.21.15.64:0/1611887800 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d06994640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d00108290 0x7f4d00101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 -- 172.21.15.64:0/1611887800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d00102d90 msgr2=0x7f4d001009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d00102d90 0x7f4d001009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 -- 172.21.15.64:0/1611887800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d00108290 msgr2=0x7f4d00101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d00108290 0x7f4d00101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.014+0000 7f4d05992640 1 -- 172.21.15.64:0/1611887800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d001a69c0 con 0x7f4d00103740 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.015+0000 7f4d05992640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 0x7f4d00100f30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000d740 tx=0x7f4cf000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.015+0000 7f4cf77fe640 1 -- 172.21.15.64:0/1611887800 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0017070 con 0x7f4d00103740 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.015+0000 7f4cf77fe640 1 -- 172.21.15.64:0/1611887800 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf000de50 con 0x7f4d00103740 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.015+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d001a6cb0 con 0x7f4d00103740 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.015+0000 7f4cf77fe640 1 -- 172.21.15.64:0/1611887800 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0004d60 con 0x7f4d00103740 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.015+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001a71a0 con 0x7f4d00103740 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.017+0000 7f4cf77fe640 1 -- 172.21.15.64:0/1611887800 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7f4cf0004f00 con 0x7f4d00103740 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 Processor -- start 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 -- start start 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5654413362f0 con 0x5654413d05d0 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654412ddcf0 con 0x5654413db370 2024-09-19T05:07:24.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5654413322b0 con 0x5654413d1b40 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4cf77fe640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4cec0a3560 0x7f4cec0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60964/0 (socket says 172.21.15.64:60964) 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d06193640 1 -- 172.21.15.64:0/2691250944 learned_addr learned my addr 172.21.15.64:0/2691250944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d05992640 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.018+0000 7f4d06994640 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06994640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4cec0a3560 0x7f4cec0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4cf77fe640 1 -- 172.21.15.64:0/1611887800 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4cf00cfa50 con 0x7f4d00103740 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06193640 1 -- 172.21.15.64:0/2691250944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 msgr2=0x5654413d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06193640 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06193640 1 -- 172.21.15.64:0/2691250944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 msgr2=0x5654413dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06193640 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06193640 1 -- 172.21.15.64:0/2691250944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654412c23a0 con 0x5654413d05d0 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06193640 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413d0a10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005de0 tx=0x7f4cfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a89c3fe2b1f4689 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4d06994640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4cec0a3560 0x7f4cec0a5a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8005c40 tx=0x7f4cf8009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4cf4dea640 1 -- 172.21.15.64:0/2691250944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc03e030 con 0x5654413d05d0 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4cf4dea640 1 -- 172.21.15.64:0/2691250944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cfc032c00 con 0x5654413d05d0 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.019+0000 7f4cf4dea640 1 -- 172.21.15.64:0/2691250944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc035040 con 0x5654413d05d0 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2691250944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 msgr2=0x5654413d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413d0a10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005de0 tx=0x7f4cfc000bc0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2691250944 shutdown_connections 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413d0a10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2691250944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.020+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2691250944 >> 172.21.15.64:0/2691250944 conn(0x565441399310 msgr2=0x5654413ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2691250944 shutdown_connections 2024-09-19T05:07:24.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2691250944 wait complete. 2024-09-19T05:07:24.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 -- start start 2024-09-19T05:07:24.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 Processor -- start 2024-09-19T05:07:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5654413de010 con 0x5654413d05d0 2024-09-19T05:07:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654412f1060 con 0x5654413db370 2024-09-19T05:07:24.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5654413de190 con 0x5654413d1b40 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d0a7d0740 10 client.0 _ll_register_callbacks cb 0x5654410448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.021+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413dac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60980/0 (socket says 172.21.15.64:60980) 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 -- 172.21.15.64:0/1907828491 learned_addr learned my addr 172.21.15.64:0/1907828491 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d05992640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06994640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 -- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 msgr2=0x5654413d7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 -- 172.21.15.64:0/1907828491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 msgr2=0x5654413d8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413d8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 -- 172.21.15.64:0/1907828491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654410a73c0 con 0x5654413d05d0 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d05992640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:24.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d06193640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413dac80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006010 tx=0x7f4cfc0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f89793af6f02a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4ce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/1907828491 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc03e040 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/1907828491 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4cfc005250 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.022+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cfc032c00 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/1907828491 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cfc035040 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 10 client.15603 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654413e99a0 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/1907828491 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cfc032c00 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/1907828491 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4cfc043410 con 0x5654413d05d0 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 10 client.15603 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 10 client.15603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 20 client.15603 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5654413eaf40 0x5654413ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 4 client.15603 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565441343860 con 0x5654413eaf40 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.023+0000 7f4d05992640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5654413eaf40 0x5654413ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.024+0000 7f4ce9ffb640 10 client.15603 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.024+0000 7f4d05992640 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5654413eaf40 0x5654413ed390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ea88d8e1d2e58d server_cookie=536741aef276e5bc in_seq=0 out_seq=0 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/1907828491 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565441343860 con 0x5654413eaf40 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4ce9ffb640 10 client.15603 handle_command_reply: tid=0 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 client.15603 shutdown 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 msgr2=0x5654413dac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413dac80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006010 tx=0x7f4cfc0316d0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 shutdown_connections 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413db370 0x5654413d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413d1b40 0x5654413d7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5654413eaf40 0x5654413ed390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/1907828491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413dac80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4ce9ffb640 0 client.15603 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 >> 172.21.15.64:0/1907828491 conn(0x565441399310 msgr2=0x5654413d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 shutdown_connections 2024-09-19T05:07:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/1907828491 wait complete. 2024-09-19T05:07:24.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.025+0000 7f4d0a7d0740 20 client.15603 trim_cache size 0 max 16384 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.039+0000 7f4d0a7d0740 1 Processor -- start 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.039+0000 7f4d0a7d0740 1 -- start start 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.039+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.039+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.039+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001b23a0 con 0x5654413d05d0 2024-09-19T05:07:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001afc70 con 0x5654413d1b40 2024-09-19T05:07:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001adac0 con 0x5654413db370 2024-09-19T05:07:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60994/0 (socket says 172.21.15.64:60994) 2024-09-19T05:07:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 -- 172.21.15.64:0/17170943 learned_addr learned my addr 172.21.15.64:0/17170943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d05992640 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 -- 172.21.15.64:0/17170943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 msgr2=0x5654413d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06994640 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 -- 172.21.15.64:0/17170943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 msgr2=0x5654413e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d06193640 1 -- 172.21.15.64:0/17170943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654413d64a0 con 0x5654413d05d0 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.040+0000 7f4d05992640 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.041+0000 7f4d06193640 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413cce10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0437e0 tx=0x7f4cfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f31b58e746dc2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.041+0000 7f4cebfff640 1 -- 172.21.15.64:0/17170943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc05a020 con 0x5654413d05d0 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.041+0000 7f4cebfff640 1 -- 172.21.15.64:0/17170943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cfc043c50 con 0x5654413d05d0 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.041+0000 7f4cebfff640 1 -- 172.21.15.64:0/17170943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc03e070 con 0x5654413d05d0 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/17170943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 msgr2=0x5654413cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413cce10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0437e0 tx=0x7f4cfc000bc0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/17170943 shutdown_connections 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/17170943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413cce10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/17170943 >> 172.21.15.64:0/17170943 conn(0x565441399310 msgr2=0x5654413cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/17170943 shutdown_connections 2024-09-19T05:07:24.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/17170943 wait complete. 2024-09-19T05:07:24.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.042+0000 7f4d0a7d0740 1 -- start start 2024-09-19T05:07:24.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 Processor -- start 2024-09-19T05:07:24.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001ae130 con 0x5654413d05d0 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001af5a0 con 0x5654413d1b40 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001ae800 con 0x5654413db370 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d0a7d0740 10 client.0 _ll_register_callbacks cb 0x7f4cfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d06193640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413e3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60996/0 (socket says 172.21.15.64:60996) 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d06193640 1 -- 172.21.15.64:0/2876410556 learned_addr learned my addr 172.21.15.64:0/2876410556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d05992640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.043+0000 7f4d06994640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d06193640 1 -- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 msgr2=0x5654413d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d06193640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d06193640 1 -- 172.21.15.64:0/2876410556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 msgr2=0x5654413e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d06193640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d06193640 1 -- 172.21.15.64:0/2876410556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654413d4ae0 con 0x5654413d05d0 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d05992640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4d06193640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413e3480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005ec0 tx=0x7f4cfc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b133f4e54f58fdcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4ce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/2876410556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc05a030 con 0x5654413d05d0 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.044+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/2876410556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4cfc004240 con 0x5654413d05d0 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cfc043c50 con 0x5654413d05d0 2024-09-19T05:07:24.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/2876410556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cfc03e040 con 0x5654413d05d0 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 10 client.15609 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654414553c0 con 0x5654413d05d0 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/2876410556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cfc043c50 con 0x5654413d05d0 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/2876410556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4cfc067030 con 0x5654413d05d0 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 10 client.15609 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 10 client.15609 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 20 client.15609 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x565441455960 0x565441457d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 4 client.15609 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.045+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56544133dd70 con 0x565441455960 2024-09-19T05:07:24.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.046+0000 7f4d05992640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x565441455960 0x565441457d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4ce9ffb640 10 client.15609 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d05992640 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x565441455960 0x565441457d80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4c94f74c5dc9d74 server_cookie=e735ba8753febf52 in_seq=0 out_seq=0 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4ce9ffb640 1 -- 172.21.15.64:0/2876410556 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56544133dd70 con 0x565441455960 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4ce9ffb640 10 client.15609 handle_command_reply: tid=0 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 client.15609 shutdown 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 msgr2=0x5654413e3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413e3480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005ec0 tx=0x7f4cfc0056d0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 shutdown_connections 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654413db370 0x5654413d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654413d1b40 0x5654413e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x565441455960 0x565441457d80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4ce9ffb640 0 client.15609 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 --2- 172.21.15.64:0/2876410556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654413d05d0 0x5654413e3480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 >> 172.21.15.64:0/2876410556 conn(0x565441399310 msgr2=0x5654413cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 shutdown_connections 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.047+0000 7f4d0a7d0740 1 -- 172.21.15.64:0/2876410556 wait complete. 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d0a7d0740 20 client.15609 trim_cache size 0 max 16384 2024-09-19T05:07:24.053 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:07:24.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4cec0a3560 msgr2=0x7f4cec0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4cec0a3560 0x7f4cec0a5a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8005c40 tx=0x7f4cf8009500 comp rx=0 tx=0).stop 2024-09-19T05:07:24.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 msgr2=0x7f4d00100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 0x7f4d00100f30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000d740 tx=0x7f4cf000dc00 comp rx=0 tx=0).stop 2024-09-19T05:07:24.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 shutdown_connections 2024-09-19T05:07:24.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4cec0a3560 0x7f4cec0a5a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d00108290 0x7f4d00101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d00103740 0x7f4d00100f30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 --2- 172.21.15.64:0/1611887800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d00102d90 0x7f4d001009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.048+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 >> 172.21.15.64:0/1611887800 conn(0x7f4d000fc7d0 msgr2=0x7f4d00105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.049+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 shutdown_connections 2024-09-19T05:07:24.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:24.049+0000 7f4d083f7640 1 -- 172.21.15.64:0/1611887800 wait complete. 2024-09-19T05:07:24.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:07:25.194 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:25.597 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:25 smithi064 ceph-mon[30960]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.5 KiB/s wr, 1.48k op/s 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 -- 172.21.15.64:0/2905283304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc102570 msgr2=0x7f78cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 --2- 172.21.15.64:0/2905283304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc102570 0x7f78cc1029b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009900 tx=0x7f78c002a480 comp rx=0 tx=0).stop 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 -- 172.21.15.64:0/2905283304 shutdown_connections 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 --2- 172.21.15.64:0/2905283304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc102ef0 0x7f78cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 --2- 172.21.15.64:0/2905283304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc102570 0x7f78cc1029b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 --2- 172.21.15.64:0/2905283304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc100e20 0x7f78cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 -- 172.21.15.64:0/2905283304 >> 172.21.15.64:0/2905283304 conn(0x7f78cc0fc7d0 msgr2=0x7f78cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 -- 172.21.15.64:0/2905283304 shutdown_connections 2024-09-19T05:07:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.705+0000 7f78d14a2640 1 -- 172.21.15.64:0/2905283304 wait complete. 2024-09-19T05:07:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.706+0000 7f78d14a2640 1 Processor -- start 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.706+0000 7f78d14a2640 1 -- start start 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.706+0000 7f78d14a2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 0x7f78cc10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.706+0000 7f78d14a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc102570 0x7f78cc10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.706+0000 7f78d14a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc102ef0 0x7f78cc10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.706+0000 7f78d14a2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc111f80 con 0x7f78cc100e20 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78d14a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc111e00 con 0x7f78cc102570 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78d14a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc112100 con 0x7f78cc102ef0 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 0x7f78cc10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 0x7f78cc10e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39064/0 (socket says 172.21.15.64:39064) 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc102570 0x7f78cc10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 -- 172.21.15.64:0/876809331 learned_addr learned my addr 172.21.15.64:0/876809331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc102ef0 0x7f78cc10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 -- 172.21.15.64:0/876809331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc102ef0 msgr2=0x7f78cc10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc102ef0 0x7f78cc10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 -- 172.21.15.64:0/876809331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc102570 msgr2=0x7f78cc10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc102570 0x7f78cc10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78caffd640 1 -- 172.21.15.64:0/876809331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1a6930 con 0x7f78cc100e20 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.707+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc102570 0x7f78cc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.708+0000 7f78caffd640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 0x7f78cc10e4d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d950 tx=0x7f78bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.708+0000 7f78b3fff640 1 -- 172.21.15.64:0/876809331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc017070 con 0x7f78cc100e20 2024-09-19T05:07:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.708+0000 7f78b3fff640 1 -- 172.21.15.64:0/876809331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78bc009c30 con 0x7f78cc100e20 2024-09-19T05:07:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.708+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a6c20 con 0x7f78cc100e20 2024-09-19T05:07:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.708+0000 7f78b3fff640 1 -- 172.21.15.64:0/876809331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00f620 con 0x7f78cc100e20 2024-09-19T05:07:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.708+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a7130 con 0x7f78cc100e20 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.711+0000 7f78d387b740 1 Processor -- start 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.711+0000 7f78d387b740 1 -- start start 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.711+0000 7f78b3fff640 1 -- 172.21.15.64:0/876809331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138639+0+0 (secure 0 0 0) 0x7f78bc0109a0 con 0x7f78cc100e20 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.711+0000 7f78d387b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39bb40 0x55f33c39a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.711+0000 7f78d387b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 0x55f33c39aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.711+0000 7f78d387b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78d387b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f33c3002f0 con 0x55f33c39a5d0 2024-09-19T05:07:25.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78d387b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f33c2a7cf0 con 0x55f33c3a5370 2024-09-19T05:07:25.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78d387b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f33c2fc2b0 con 0x55f33c39bb40 2024-09-19T05:07:25.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 0x55f33c39aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 0x55f33c39aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39080/0 (socket says 172.21.15.64:39080) 2024-09-19T05:07:25.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 -- 172.21.15.64:0/3052316239 learned_addr learned my addr 172.21.15.64:0/3052316239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:25.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 -- 172.21.15.64:0/3052316239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39bb40 msgr2=0x55f33c39a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39bb40 0x55f33c39a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 -- 172.21.15.64:0/3052316239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 msgr2=0x55f33c3a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78caffd640 1 -- 172.21.15.64:0/3052316239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33c28c3a0 con 0x55f33c39a5d0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78b3fff640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78b40a3560 0x7f78b40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.712+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78b40a3560 0x7f78b40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.713+0000 7f78b3fff640 1 -- 172.21.15.64:0/876809331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f78bc0cfb90 con 0x7f78cc100e20 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.713+0000 7f78caffd640 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 0x55f33c39aa10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0cd040 tx=0x7f78bc0d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2bbefd2561b1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.713+0000 7f78b17fa640 1 -- 172.21.15.64:0/3052316239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc020030 con 0x55f33c39a5d0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.713+0000 7f78b17fa640 1 -- 172.21.15.64:0/3052316239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78bc009de0 con 0x55f33c39a5d0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.713+0000 7f78b17fa640 1 -- 172.21.15.64:0/3052316239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc09b820 con 0x55f33c39a5d0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.713+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78b40a3560 0x7f78b40a5a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005fd0 tx=0x7f78c4009a60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 -- 172.21.15.64:0/3052316239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 msgr2=0x55f33c39aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 0x55f33c39aa10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0cd040 tx=0x7f78bc0d8670 comp rx=0 tx=0).stop 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 -- 172.21.15.64:0/3052316239 shutdown_connections 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39a5d0 0x55f33c39aa10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 --2- 172.21.15.64:0/3052316239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39bb40 0x55f33c39a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 -- 172.21.15.64:0/3052316239 >> 172.21.15.64:0/3052316239 conn(0x55f33c363310 msgr2=0x55f33c396c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:25.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 -- 172.21.15.64:0/3052316239 shutdown_connections 2024-09-19T05:07:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 -- 172.21.15.64:0/3052316239 wait complete. 2024-09-19T05:07:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.714+0000 7f78d387b740 1 -- start start 2024-09-19T05:07:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 Processor -- start 2024-09-19T05:07:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c3a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f33c3a8070 con 0x55f33c39bb40 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f33c2bb060 con 0x55f33c3a5370 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f33c3a81f0 con 0x55f33c39a5d0 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78d387b740 10 client.0 _ll_register_callbacks cb 0x55f33c00e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39096/0 (socket says 172.21.15.64:39096) 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 -- 172.21.15.64:0/404383545 learned_addr learned my addr 172.21.15.64:0/404383545 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78caffd640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c3a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 -- 172.21.15.64:0/404383545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 msgr2=0x55f33c3a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c3a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 -- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 msgr2=0x55f33c3a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.715+0000 7f78ca7fc640 1 -- 172.21.15.64:0/404383545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33c0713c0 con 0x55f33c39bb40 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a2070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009900 tx=0x7f78c00053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc451097c37868d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78aeff5640 1 -- 172.21.15.64:0/404383545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0035040 con 0x55f33c39bb40 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78aeff5640 1 -- 172.21.15.64:0/404383545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f78c0004960 con 0x55f33c39bb40 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc009de0 con 0x55f33c39bb40 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78aeff5640 1 -- 172.21.15.64:0/404383545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78c0004b60 con 0x55f33c39bb40 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78d387b740 10 client.15621 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.716+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f33c3b3b80 con 0x55f33c39bb40 2024-09-19T05:07:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78aeff5640 1 -- 172.21.15.64:0/404383545 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78bc009de0 con 0x55f33c39bb40 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78aeff5640 1 -- 172.21.15.64:0/404383545 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78c0044020 con 0x55f33c39bb40 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78d387b740 10 client.15621 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78d387b740 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78d387b740 20 client.15621 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78d387b740 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c3b5040 0x55f33c3b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78d387b740 4 client.15621 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f33c30d860 con 0x55f33c3b5040 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.717+0000 7f78caffd640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c3b5040 0x55f33c3b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.718+0000 7f78aeff5640 10 client.15621 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.718+0000 7f78caffd640 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c3b5040 0x55f33c3b7460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=408f6b900d811072 server_cookie=540dec036bcd6054 in_seq=0 out_seq=0 2024-09-19T05:07:25.723 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:25 smithi103 ceph-mon[34446]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.5 KiB/s wr, 1.48k op/s 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78aeff5640 1 -- 172.21.15.64:0/404383545 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f33c30d860 con 0x55f33c3b5040 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78aeff5640 10 client.15621 handle_command_reply: tid=0 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 client.15621 shutdown 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 msgr2=0x55f33c3a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a2070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009900 tx=0x7f78c00053b0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 shutdown_connections 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c3a5370 0x55f33c3a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c3b5040 0x55f33c3b7460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a2070 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 --2- 172.21.15.64:0/404383545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c3a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78aeff5640 0 client.15621 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 >> 172.21.15.64:0/404383545 conn(0x55f33c363310 msgr2=0x55f33c39c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 shutdown_connections 2024-09-19T05:07:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 1 -- 172.21.15.64:0/404383545 wait complete. 2024-09-19T05:07:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.719+0000 7f78d387b740 20 client.15621 trim_cache size 0 max 16384 2024-09-19T05:07:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 Processor -- start 2024-09-19T05:07:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 -- start start 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c396d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39bb40 0x55f33c3a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 0x55f33c3bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1b23f0 con 0x55f33c3a5370 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1afcc0 con 0x55f33c39bb40 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78d387b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1adb10 con 0x55f33c39a5d0 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 0x55f33c3bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 0x55f33c3bcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39106/0 (socket says 172.21.15.64:39106) 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78cb7fe640 1 -- 172.21.15.64:0/2644884269 learned_addr learned my addr 172.21.15.64:0/2644884269 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.733+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39bb40 0x55f33c3a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78cb7fe640 1 -- 172.21.15.64:0/2644884269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 msgr2=0x55f33c396d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78caffd640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c396d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c396d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78cb7fe640 1 -- 172.21.15.64:0/2644884269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39bb40 msgr2=0x55f33c3a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39bb40 0x55f33c3a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78cb7fe640 1 -- 172.21.15.64:0/2644884269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33c39d120 con 0x55f33c3a5370 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78caffd640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c396d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39bb40 0x55f33c3a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 0x55f33c3bcd30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009a30 tx=0x7f78c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4858e8ae68c5dec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78b0ff9640 1 -- 172.21.15.64:0/2644884269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c401f6e0 con 0x55f33c3a5370 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78b0ff9640 1 -- 172.21.15.64:0/2644884269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78c401fd30 con 0x55f33c3a5370 2024-09-19T05:07:25.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.734+0000 7f78b0ff9640 1 -- 172.21.15.64:0/2644884269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4027e50 con 0x55f33c3a5370 2024-09-19T05:07:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 -- 172.21.15.64:0/2644884269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 msgr2=0x55f33c3bcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 0x55f33c3bcd30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009a30 tx=0x7f78c4013000 comp rx=0 tx=0).stop 2024-09-19T05:07:25.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 -- 172.21.15.64:0/2644884269 shutdown_connections 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c3a5370 0x55f33c3bcd30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39bb40 0x55f33c3a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 --2- 172.21.15.64:0/2644884269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c39a5d0 0x55f33c396d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 -- 172.21.15.64:0/2644884269 >> 172.21.15.64:0/2644884269 conn(0x55f33c363310 msgr2=0x55f33c3a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 -- 172.21.15.64:0/2644884269 shutdown_connections 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 -- 172.21.15.64:0/2644884269 wait complete. 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.735+0000 7f78d387b740 1 -- start start 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 Processor -- start 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39a5d0 0x55f33c3adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c3a5370 0x55f33c3a9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ae180 con 0x55f33c39bb40 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1af5f0 con 0x55f33c39a5d0 2024-09-19T05:07:25.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ae850 con 0x55f33c3a5370 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78d387b740 10 client.0 _ll_register_callbacks cb 0x7f78c0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39110/0 (socket says 172.21.15.64:39110) 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 -- 172.21.15.64:0/4106504350 learned_addr learned my addr 172.21.15.64:0/4106504350 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78caffd640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39a5d0 0x55f33c3adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78cb7fe640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c3a5370 0x55f33c3a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 -- 172.21.15.64:0/4106504350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c3a5370 msgr2=0x55f33c3a9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c3a5370 0x55f33c3a9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 -- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39a5d0 msgr2=0x55f33c3adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39a5d0 0x55f33c3adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78ca7fc640 1 -- 172.21.15.64:0/4106504350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33c3a9a00 con 0x55f33c39bb40 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.736+0000 7f78caffd640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39a5d0 0x55f33c3adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78ca7fc640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a8dd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009850 tx=0x7f78c0004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c8499c8b9310e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78aeff5640 1 -- 172.21.15.64:0/4106504350 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0035030 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78aeff5640 1 -- 172.21.15.64:0/4106504350 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f78c0004d00 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78c401fd30 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78aeff5640 1 -- 172.21.15.64:0/4106504350 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78c005c050 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78d387b740 10 client.15627 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.737+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f33c41fb60 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78aeff5640 1 -- 172.21.15.64:0/4106504350 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78c401fd30 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78aeff5640 1 -- 172.21.15.64:0/4106504350 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f78c005f020 con 0x55f33c39bb40 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78d387b740 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78d387b740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78d387b740 20 client.15627 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78d387b740 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c420360 0x55f33c422780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78d387b740 4 client.15627 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f33c307d70 con 0x55f33c420360 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.738+0000 7f78caffd640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c420360 0x55f33c422780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78aeff5640 10 client.15627 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78caffd640 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c420360 0x55f33c422780 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3532a72a4dc120 server_cookie=694b53f87bc02395 in_seq=0 out_seq=0 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78aeff5640 1 -- 172.21.15.64:0/4106504350 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f33c307d70 con 0x55f33c420360 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78aeff5640 10 client.15627 handle_command_reply: tid=0 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 client.15627 shutdown 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 msgr2=0x55f33c3a8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a8dd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009850 tx=0x7f78c0004990 comp rx=0 tx=0).stop 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 shutdown_connections 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f33c3a5370 0x55f33c3a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f33c420360 0x55f33c422780 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f33c39bb40 0x55f33c3a8dd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 --2- 172.21.15.64:0/4106504350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f33c39a5d0 0x55f33c3adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 >> 172.21.15.64:0/4106504350 conn(0x55f33c363310 msgr2=0x55f33c3a1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78aeff5640 0 client.15627 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 shutdown_connections 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.739+0000 7f78d387b740 1 -- 172.21.15.64:0/4106504350 wait complete. 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d387b740 20 client.15627 trim_cache size 0 max 16384 2024-09-19T05:07:25.746 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78b40a3560 msgr2=0x7f78b40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78b40a3560 0x7f78b40a5a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005fd0 tx=0x7f78c4009a60 comp rx=0 tx=0).stop 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 msgr2=0x7f78cc10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 0x7f78cc10e4d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d950 tx=0x7f78bc00de10 comp rx=0 tx=0).stop 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 shutdown_connections 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78b40a3560 0x7f78b40a5a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc102ef0 0x7f78cc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78cc102570 0x7f78cc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 --2- 172.21.15.64:0/876809331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78cc100e20 0x7f78cc10e4d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 >> 172.21.15.64:0/876809331 conn(0x7f78cc0fc7d0 msgr2=0x7f78cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 shutdown_connections 2024-09-19T05:07:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:25.740+0000 7f78d14a2640 1 -- 172.21.15.64:0/876809331 wait complete. 2024-09-19T05:07:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:25 smithi174 ceph-mon[34488]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.5 KiB/s wr, 1.48k op/s 2024-09-19T05:07:27.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:07:27.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:07:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:27 smithi103 ceph-mon[34446]: pgmap v49: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 KiB/s wr, 1.49k op/s 2024-09-19T05:07:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:27 smithi103 ceph-mon[34446]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-19T05:07:27.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:27 smithi174 ceph-mon[34488]: pgmap v49: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 KiB/s wr, 1.49k op/s 2024-09-19T05:07:27.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:27 smithi174 ceph-mon[34488]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-19T05:07:27.845 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:27.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:27 smithi064 ceph-mon[30960]: pgmap v49: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 14 KiB/s wr, 1.49k op/s 2024-09-19T05:07:27.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:27 smithi064 ceph-mon[30960]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.343+0000 7fb6cee4c640 1 -- 172.21.15.64:0/2251746430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8101410 msgr2=0x7fb6c810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.343+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/2251746430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8101410 0x7fb6c810d940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009cd0 tx=0x7fb6c002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 -- 172.21.15.64:0/2251746430 shutdown_connections 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/2251746430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8101410 0x7fb6c810d940 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/2251746430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c8100a70 0x7fb6c8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/2251746430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 0x7fb6c8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 -- 172.21.15.64:0/2251746430 >> 172.21.15.64:0/2251746430 conn(0x7fb6c80fc790 msgr2=0x7fb6c80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 -- 172.21.15.64:0/2251746430 shutdown_connections 2024-09-19T05:07:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 -- 172.21.15.64:0/2251746430 wait complete. 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 Processor -- start 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.344+0000 7fb6cee4c640 1 -- start start 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cee4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 0x7fb6c8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cee4c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c8101410 0x7fb6c81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cee4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 0x7fb6c819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cee4c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8110d90 con 0x7fb6c8100a70 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cee4c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8110c10 con 0x7fb6c8101410 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cee4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8110f10 con 0x7fb6c8106a70 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 0x7fb6c8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 0x7fb6c8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39126/0 (socket says 172.21.15.64:39126) 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/971582973 learned_addr learned my addr 172.21.15.64:0/971582973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c8101410 0x7fb6c81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 0x7fb6c819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/971582973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 msgr2=0x7fb6c819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 0x7fb6c819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/971582973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c8101410 msgr2=0x7fb6c81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c8101410 0x7fb6c81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/971582973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c819d320 con 0x7fb6c8100a70 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 0x7fb6c819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 0x7fb6c8198370 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00b4a0 tx=0x7fb6bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.345+0000 7fb6c5ffb640 1 -- 172.21.15.64:0/971582973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc014070 con 0x7fb6c8100a70 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.346+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c819d610 con 0x7fb6c8100a70 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.346+0000 7fb6c5ffb640 1 -- 172.21.15.64:0/971582973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc010ac0 con 0x7fb6c8100a70 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.346+0000 7fb6c5ffb640 1 -- 172.21.15.64:0/971582973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc017c30 con 0x7fb6c8100a70 2024-09-19T05:07:28.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.346+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81a4ef0 con 0x7fb6c8100a70 2024-09-19T05:07:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.348+0000 7fb6c5ffb640 1 -- 172.21.15.64:0/971582973 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb6bc017070 con 0x7fb6c8100a70 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 Processor -- start 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 -- start start 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c48b60 0x557dc9c48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c49480 0x557dc9c53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557dc9bae280 con 0x557dc9c53d40 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dc9b55b10 con 0x557dc9c48b60 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6d1225740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557dc9baa240 con 0x557dc9c49480 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6c7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6c7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c56130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39134/0 (socket says 172.21.15.64:39134) 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6c7fff640 1 -- 172.21.15.64:0/4121122679 learned_addr learned my addr 172.21.15.64:0/4121122679 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6c5ffb640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb6b80a3430 0x7fb6b80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.349+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c48b60 0x557dc9c48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c49480 0x557dc9c53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb6b80a3430 0x7fb6b80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6c5ffb640 1 -- 172.21.15.64:0/971582973 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb6bc0cf240 con 0x7fb6c8100a70 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/4121122679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c49480 msgr2=0x557dc9c53800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c49480 0x557dc9c53800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/4121122679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 msgr2=0x557dc9c56130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c56130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/4121122679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc9b621f0 con 0x557dc9c48b60 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c48b60 0x557dc9c48f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009ca0 tx=0x7fb6c0031b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c692f3a49945e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.350+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb6b80a3430 0x7fb6b80a58f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6c004a000 tx=0x7fb6c004d000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.351+0000 7fb6b2ffd640 1 -- 172.21.15.64:0/4121122679 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0052070 con 0x557dc9c48b60 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.351+0000 7fb6b2ffd640 1 -- 172.21.15.64:0/4121122679 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c0036040 con 0x557dc9c48b60 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.351+0000 7fb6b2ffd640 1 -- 172.21.15.64:0/4121122679 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c003d9d0 con 0x557dc9c48b60 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 -- 172.21.15.64:0/4121122679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c48b60 msgr2=0x557dc9c48f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c48b60 0x557dc9c48f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009ca0 tx=0x7fb6c0031b70 comp rx=0 tx=0).stop 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 -- 172.21.15.64:0/4121122679 shutdown_connections 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c49480 0x557dc9c53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 --2- 172.21.15.64:0/4121122679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c48b60 0x557dc9c48f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 -- 172.21.15.64:0/4121122679 >> 172.21.15.64:0/4121122679 conn(0x557dc9c450f0 msgr2=0x557dc9c454d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 -- 172.21.15.64:0/4121122679 shutdown_connections 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 -- 172.21.15.64:0/4121122679 wait complete. 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 -- start start 2024-09-19T05:07:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.352+0000 7fb6d1225740 1 Processor -- start 2024-09-19T05:07:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c50cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c51730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557dc9c56910 con 0x557dc9c53d40 2024-09-19T05:07:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dc9b68f10 con 0x557dc9c49480 2024-09-19T05:07:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557dc9c56a90 con 0x557dc9c48b60 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c51730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c51730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39148/0 (socket says 172.21.15.64:39148) 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/3444112014 learned_addr learned my addr 172.21.15.64:0/3444112014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6d1225740 10 client.0 _ll_register_callbacks cb 0x557dc9b31a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c50cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/3444112014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 msgr2=0x557dc9c50cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c50cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 msgr2=0x557dc9c511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.353+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/3444112014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc991f6a0 con 0x557dc9c53d40 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c51730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb6c002a920 tx=0x7fb6c003f2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a712858c45b3f708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3444112014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c005b060 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c0036040 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3444112014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb6c003d990 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3444112014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6c0052040 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6d1225740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dc9c56910 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.354+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3444112014 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c0036040 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3444112014 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6c0030070 con 0x557dc9c53d40 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6d1225740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6d1225740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6d1225740 20 client.15639 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9c65b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6d1225740 4 client.15639 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dc9bbb7f0 con 0x557dc9c63720 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9c65b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9c65b10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2838edccc3a5fa server_cookie=95436a9d250d8d08 in_seq=0 out_seq=0 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.355+0000 7fb6a7fff640 10 client.15639 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3444112014 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dc9bbb7f0 con 0x557dc9c63720 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6a7fff640 10 client.15639 handle_command_reply: tid=0 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 client.15639 shutdown 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 msgr2=0x557dc9c51730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c51730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb6c002a920 tx=0x7fb6c003f2e0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 shutdown_connections 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9c65b10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c51730 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6a7fff640 0 client.15639 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3444112014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c50cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.356+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 >> 172.21.15.64:0/3444112014 conn(0x557dc9c450f0 msgr2=0x557dc9c4ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.357+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 shutdown_connections 2024-09-19T05:07:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.357+0000 7fb6d1225740 1 -- 172.21.15.64:0/3444112014 wait complete. 2024-09-19T05:07:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.357+0000 7fb6d1225740 20 client.15639 trim_cache size 0 max 16384 2024-09-19T05:07:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 Processor -- start 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 -- start start 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c4b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c51580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c6b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81adf30 con 0x557dc9c53d40 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81ab800 con 0x557dc9c49480 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.377+0000 7fb6d1225740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81a9650 con 0x557dc9c48b60 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c6b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c6b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39156/0 (socket says 172.21.15.64:39156) 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/1985498171 learned_addr learned my addr 172.21.15.64:0/1985498171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:28.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c4b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/1985498171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 msgr2=0x557dc9c4b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c4b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/1985498171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 msgr2=0x557dc9c51580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c51580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 -- 172.21.15.64:0/1985498171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc9c492b0 con 0x557dc9c53d40 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c6b2d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0002230 tx=0x7fb6c0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364dfaca19e09764 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.378+0000 7fb6b27fc640 1 -- 172.21.15.64:0/1985498171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c005b040 con 0x557dc9c53d40 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6b27fc640 1 -- 172.21.15.64:0/1985498171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c0036030 con 0x557dc9c53d40 2024-09-19T05:07:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6b27fc640 1 -- 172.21.15.64:0/1985498171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0004040 con 0x557dc9c53d40 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 -- 172.21.15.64:0/1985498171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 msgr2=0x557dc9c6b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c6b2d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0002230 tx=0x7fb6c0009c20 comp rx=0 tx=0).stop 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 -- 172.21.15.64:0/1985498171 shutdown_connections 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c53d40 0x557dc9c6b2d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c51580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 --2- 172.21.15.64:0/1985498171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c48b60 0x557dc9c4b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 -- 172.21.15.64:0/1985498171 >> 172.21.15.64:0/1985498171 conn(0x557dc9c450f0 msgr2=0x557dc9c52c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 -- 172.21.15.64:0/1985498171 shutdown_connections 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.379+0000 7fb6d1225740 1 -- 172.21.15.64:0/1985498171 wait complete. 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 -- start start 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 Processor -- start 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 0x557dc9c580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c53d40 0x557dc9ccd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81a9cc0 con 0x557dc9c48b60 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81ab130 con 0x557dc9c49480 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81aa390 con 0x557dc9c53d40 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6d1225740 10 client.0 _ll_register_callbacks cb 0x7fb6c0050070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6ccbe8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 0x557dc9c580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6ccbe8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 0x557dc9c580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39164/0 (socket says 172.21.15.64:39164) 2024-09-19T05:07:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.380+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/3661230987 learned_addr learned my addr 172.21.15.64:0/3661230987 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c53d40 0x557dc9ccd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/3661230987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c53d40 msgr2=0x557dc9ccd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c53d40 0x557dc9ccd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 msgr2=0x557dc9c58610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c58610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6ccbe8640 1 -- 172.21.15.64:0/3661230987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc9ccd7e0 con 0x557dc9c48b60 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6cd3e9640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c53d40 0x557dc9ccd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6ccbe8640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 0x557dc9c580d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00be60 tx=0x7fb6bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c0e02c5e7a56e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3661230987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc01e040 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3661230987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb6bc0ccc10 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c0036030 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3661230987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6bc097ca0 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6d1225740 10 client.15645 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.381+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dc9cceae0 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3661230987 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc010c70 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3661230987 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6bc0afb50 con 0x557dc9c48b60 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6d1225740 10 client.15645 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6d1225740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6d1225740 20 client.15645 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9cd0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6d1225740 4 client.15645 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:07:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb6c0036030 con 0x557dc9c63720 2024-09-19T05:07:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.382+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9cd0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.383+0000 7fb6c7fff640 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9cd0fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b7e263d60321e server_cookie=2b4d77f6a8c88078 in_seq=0 out_seq=0 2024-09-19T05:07:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.383+0000 7fb6a7fff640 10 client.15645 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6a7fff640 1 -- 172.21.15.64:0/3661230987 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb6c0036030 con 0x557dc9c63720 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6a7fff640 10 client.15645 handle_command_reply: tid=0 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 client.15645 shutdown 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 msgr2=0x557dc9c580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 0x557dc9c580d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00be60 tx=0x7fb6bc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 shutdown_connections 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dc9c53d40 0x557dc9ccd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dc9c49480 0x557dc9c58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557dc9c63720 0x557dc9cd0fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 --2- 172.21.15.64:0/3661230987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc9c48b60 0x557dc9c580d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 >> 172.21.15.64:0/3661230987 conn(0x557dc9c450f0 msgr2=0x557dc9c52320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6a7fff640 0 client.15645 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 shutdown_connections 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.384+0000 7fb6d1225740 1 -- 172.21.15.64:0/3661230987 wait complete. 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6d1225740 20 client.15645 trim_cache size 0 max 16384 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "5ab08c67-3eaa-49da-bfa2-90b7f5e046db", 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:07:28.390 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:07:28.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb6b80a3430 msgr2=0x7fb6b80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb6b80a3430 0x7fb6b80a58f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6c004a000 tx=0x7fb6c004d000 comp rx=0 tx=0).stop 2024-09-19T05:07:28.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 msgr2=0x7fb6c8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:28.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 0x7fb6c8198370 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00b4a0 tx=0x7fb6bc00b960 comp rx=0 tx=0).stop 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 shutdown_connections 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb6b80a3430 0x7fb6b80a58f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6c8106a70 0x7fb6c819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c8101410 0x7fb6c81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 --2- 172.21.15.64:0/971582973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6c8100a70 0x7fb6c8198370 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.385+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 >> 172.21.15.64:0/971582973 conn(0x7fb6c80fc790 msgr2=0x7fb6c806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.386+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 shutdown_connections 2024-09-19T05:07:28.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:28.386+0000 7fb6cee4c640 1 -- 172.21.15.64:0/971582973 wait complete. 2024-09-19T05:07:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:29 smithi103 ceph-mon[34446]: pgmap v50: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 467 op/s 2024-09-19T05:07:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:29 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:07:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:29 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:29 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:07:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:29 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:29 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:07:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:29 smithi174 ceph-mon[34488]: pgmap v50: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 467 op/s 2024-09-19T05:07:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:29 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:07:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:29 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:29 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:07:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:29 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:29 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:07:29.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:29 smithi064 ceph-mon[30960]: pgmap v50: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 467 op/s 2024-09-19T05:07:29.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:29 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:07:29.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:29 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:29.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:29 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:07:29.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:29 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:29.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:29 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:07:30.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:07:31.050 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:31 smithi103 ceph-mon[34446]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 467 op/s 2024-09-19T05:07:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:31 smithi174 ceph-mon[34488]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 467 op/s 2024-09-19T05:07:31.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:31 smithi064 ceph-mon[30960]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 467 op/s 2024-09-19T05:07:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:33 smithi064 ceph-mon[30960]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 470 op/s 2024-09-19T05:07:33.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:33 smithi103 ceph-mon[34446]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 470 op/s 2024-09-19T05:07:33.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:33 smithi174 ceph-mon[34488]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 KiB/s wr, 470 op/s 2024-09-19T05:07:34.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 -- 172.21.15.64:0/1743260337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 msgr2=0x7f24d81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/1743260337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d81021c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009900 tx=0x7f24c802a410 comp rx=0 tx=0).stop 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 -- 172.21.15.64:0/1743260337 shutdown_connections 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/1743260337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24d8106f10 0x7f24d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/1743260337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d8102790 0x7f24d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/1743260337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d81021c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 -- 172.21.15.64:0/1743260337 >> 172.21.15.64:0/1743260337 conn(0x7f24d80fd8e0 msgr2=0x7f24d80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 -- 172.21.15.64:0/1743260337 shutdown_connections 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.151+0000 7f24ddbe4640 1 -- 172.21.15.64:0/1743260337 wait complete. 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 Processor -- start 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 -- start start 2024-09-19T05:07:34.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d8102790 0x7f24d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24d8106f10 0x7f24d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8111e40 con 0x7f24d8101de0 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8111cc0 con 0x7f24d8102790 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24ddbe4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8111fc0 con 0x7f24d8106f10 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39192/0 (socket says 172.21.15.64:39192) 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.152+0000 7f24d77fe640 1 -- 172.21.15.64:0/2833540151 learned_addr learned my addr 172.21.15.64:0/2833540151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d8102790 0x7f24d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d7fff640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24d8106f10 0x7f24d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d77fe640 1 -- 172.21.15.64:0/2833540151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24d8106f10 msgr2=0x7f24d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d77fe640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24d8106f10 0x7f24d8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d77fe640 1 -- 172.21.15.64:0/2833540151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d8102790 msgr2=0x7f24d8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d77fe640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d8102790 0x7f24d8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d77fe640 1 -- 172.21.15.64:0/2833540151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d81a6870 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d77fe640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d80744f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005940 tx=0x7f24c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d4ff9640 1 -- 172.21.15.64:0/2833540151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c803f030 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d4ff9640 1 -- 172.21.15.64:0/2833540151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c8003aa0 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d81a6b00 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.153+0000 7f24d4ff9640 1 -- 172.21.15.64:0/2833540151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8035070 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.154+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81a7030 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.156+0000 7f24dffbd740 1 Processor -- start 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.156+0000 7f24dffbd740 1 -- start start 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.156+0000 7f24d4ff9640 1 -- 172.21.15.64:0/2833540151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f24c8042030 con 0x7f24d8101de0 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfb6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24dffbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ef9df1c2e0 con 0x55ef9dfb64a0 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24dffbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef9dec3ce0 con 0x55ef9dfb6dc0 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24dffbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef9df182a0 con 0x55ef9dfc1680 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfb6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfb6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39198/0 (socket says 172.21.15.64:39198) 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 -- 172.21.15.64:0/3810089029 learned_addr learned my addr 172.21.15.64:0/3810089029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d77fe640 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 -- 172.21.15.64:0/3810089029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfc1680 msgr2=0x55ef9dfc3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 -- 172.21.15.64:0/3810089029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 msgr2=0x55ef9dfc1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d7fff640 1 -- 172.21.15.64:0/3810089029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9dea8440 con 0x55ef9dfb64a0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.157+0000 7f24d4ff9640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f24c00a3430 0x7f24c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.158+0000 7f24d77fe640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f24c00a3430 0x7f24c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.158+0000 7f24d7fff640 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfb6880 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f24d0007d90 tx=0x7f24d000c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce17bacfb7b1c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.158+0000 7f24d4ff9640 1 -- 172.21.15.64:0/2833540151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f24c80f0270 con 0x7f24d8101de0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.158+0000 7f24bdffb640 1 -- 172.21.15.64:0/3810089029 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0015020 con 0x55ef9dfb64a0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.158+0000 7f24bdffb640 1 -- 172.21.15.64:0/3810089029 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d0011ae0 con 0x55ef9dfb64a0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.158+0000 7f24bdffb640 1 -- 172.21.15.64:0/3810089029 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d00109d0 con 0x55ef9dfb64a0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 -- 172.21.15.64:0/3810089029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 msgr2=0x55ef9dfb6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfb6880 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f24d0007d90 tx=0x7f24d000c560 comp rx=0 tx=0).stop 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24d77fe640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f24c00a3430 0x7f24c00a58f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24c80eda70 tx=0x7f24c80f5040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:34.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 -- 172.21.15.64:0/3810089029 shutdown_connections 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 --2- 172.21.15.64:0/3810089029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfb6880 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 -- 172.21.15.64:0/3810089029 >> 172.21.15.64:0/3810089029 conn(0x55ef9dfb2a30 msgr2=0x55ef9dfb2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 -- 172.21.15.64:0/3810089029 shutdown_connections 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 -- 172.21.15.64:0/3810089029 wait complete. 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 -- start start 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.159+0000 7f24dffbd740 1 Processor -- start 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfbe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfbe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ef9dfc4250 con 0x55ef9dfc1680 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ded7050 con 0x55ef9dfb6dc0 2024-09-19T05:07:34.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef9dfc43d0 con 0x55ef9dfb64a0 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24dffbd740 10 client.0 _ll_register_callbacks cb 0x55ef9df17160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfbe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfbe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39210/0 (socket says 172.21.15.64:39210) 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 -- 172.21.15.64:0/128997287 learned_addr learned my addr 172.21.15.64:0/128997287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfbe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 -- 172.21.15.64:0/128997287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 msgr2=0x55ef9dfc0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 -- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 msgr2=0x55ef9dfbe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfbe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.160+0000 7f24d7fff640 1 -- 172.21.15.64:0/128997287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9dc8d3c0 con 0x55ef9dfc1680 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24d7fff640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfbe7e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f24d00075a0 tx=0x7f24d0010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fd4d8707e4d11a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:34.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24b6ffd640 1 -- 172.21.15.64:0/128997287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0015040 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24b6ffd640 1 -- 172.21.15.64:0/128997287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f24d0010e00 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d0011ae0 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24b6ffd640 1 -- 172.21.15.64:0/128997287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24d0007120 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24dffbd740 10 client.15657 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef9dfcfc20 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24b6ffd640 1 -- 172.21.15.64:0/128997287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d0011ae0 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24b6ffd640 1 -- 172.21.15.64:0/128997287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24d001a970 con 0x55ef9dfc1680 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24dffbd740 10 client.15657 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.161+0000 7f24dffbd740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24dffbd740 20 client.15657 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:34.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24dffbd740 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9dfd0f70 0x55ef9dfd3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24dffbd740 4 client.15657 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef9df29850 con 0x55ef9dfd0f70 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9dfd0f70 0x55ef9dfd3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24b6ffd640 10 client.15657 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.162+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9dfd0f70 0x55ef9dfd3360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c608f2f5d6bf88 server_cookie=73e9b7fd1ab76546 in_seq=0 out_seq=0 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24b6ffd640 1 -- 172.21.15.64:0/128997287 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef9df29850 con 0x55ef9dfd0f70 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24b6ffd640 10 client.15657 handle_command_reply: tid=0 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 client.15657 shutdown 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 msgr2=0x55ef9dfbe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfbe7e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f24d00075a0 tx=0x7f24d0010530 comp rx=0 tx=0).stop 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 shutdown_connections 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9dfd0f70 0x55ef9dfd3360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfbe7e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfbe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 --2- 172.21.15.64:0/128997287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 >> 172.21.15.64:0/128997287 conn(0x55ef9dfb2a30 msgr2=0x55ef9dfb1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24b6ffd640 0 client.15657 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.163+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 shutdown_connections 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.164+0000 7f24dffbd740 1 -- 172.21.15.64:0/128997287 wait complete. 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.164+0000 7f24dffbd740 20 client.15657 trim_cache size 0 max 16384 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 Processor -- start 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 -- start start 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfd8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfd90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81b23a0 con 0x55ef9dfc1680 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81afc70 con 0x55ef9dfb64a0 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24dffbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81adac0 con 0x55ef9dfb6dc0 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfd90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfd90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39220/0 (socket says 172.21.15.64:39220) 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24d7fff640 1 -- 172.21.15.64:0/203387162 learned_addr learned my addr 172.21.15.64:0/203387162 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24d7fff640 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfd8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.184+0000 7f24d7fff640 1 -- 172.21.15.64:0/203387162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb6dc0 msgr2=0x55ef9dfd8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24d7fff640 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfd8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24d7fff640 1 -- 172.21.15.64:0/203387162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb64a0 msgr2=0x55ef9dfc01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24d7fff640 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24d7fff640 1 -- 172.21.15.64:0/203387162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9dfc1300 con 0x55ef9dfc1680 2024-09-19T05:07:34.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfd8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24d7fff640 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfd90a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f24d001ad70 tx=0x7f24d00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb0222da26add22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24bd7fa640 1 -- 172.21.15.64:0/203387162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d000ca60 con 0x55ef9dfc1680 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24bd7fa640 1 -- 172.21.15.64:0/203387162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d00070f0 con 0x55ef9dfc1680 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.185+0000 7f24bd7fa640 1 -- 172.21.15.64:0/203387162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d001ccc0 con 0x55ef9dfc1680 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 -- 172.21.15.64:0/203387162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 msgr2=0x55ef9dfd90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfd90a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f24d001ad70 tx=0x7f24d00078a0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 -- 172.21.15.64:0/203387162 shutdown_connections 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfd90a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfd8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 --2- 172.21.15.64:0/203387162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 -- 172.21.15.64:0/203387162 >> 172.21.15.64:0/203387162 conn(0x55ef9dfb2a30 msgr2=0x55ef9dfbfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 -- 172.21.15.64:0/203387162 shutdown_connections 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 -- 172.21.15.64:0/203387162 wait complete. 2024-09-19T05:07:34.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.186+0000 7f24dffbd740 1 -- start start 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 Processor -- start 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ae130 con 0x55ef9dfc1680 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81af5a0 con 0x55ef9dfb6dc0 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ae800 con 0x55ef9dfb64a0 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24dffbd740 10 client.0 _ll_register_callbacks cb 0x7f24c80f3ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39226/0 (socket says 172.21.15.64:39226) 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d7fff640 1 -- 172.21.15.64:0/1207170947 learned_addr learned my addr 172.21.15.64:0/1207170947 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:34.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d7fff640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d7fff640 1 -- 172.21.15.64:0/1207170947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 msgr2=0x55ef9dfc9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.187+0000 7f24d7fff640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24d7fff640 1 -- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 msgr2=0x55ef9dfc7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24d7fff640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24d7fff640 1 -- 172.21.15.64:0/1207170947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9dfc7e40 con 0x55ef9dfc1680 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24d7fff640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc7640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f24d0010590 tx=0x7f24d0007080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cee401207cc223 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24b6ffd640 1 -- 172.21.15.64:0/1207170947 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0011630 con 0x55ef9dfc1680 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d00070f0 con 0x55ef9dfc1680 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24b6ffd640 1 -- 172.21.15.64:0/1207170947 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f24d00117d0 con 0x55ef9dfc1680 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24b6ffd640 1 -- 172.21.15.64:0/1207170947 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24d000ca60 con 0x55ef9dfc1680 2024-09-19T05:07:34.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24dffbd740 10 client.15663 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef9e03baf0 con 0x55ef9dfc1680 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.188+0000 7f24b6ffd640 1 -- 172.21.15.64:0/1207170947 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d00070f0 con 0x55ef9dfc1680 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24b6ffd640 1 -- 172.21.15.64:0/1207170947 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24d0043410 con 0x55ef9dfc1680 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24dffbd740 10 client.15663 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24dffbd740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24dffbd740 20 client.15663 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24dffbd740 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9e03bfd0 0x55ef9e03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24dffbd740 4 client.15663 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef9df23d60 con 0x55ef9e03bfd0 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.189+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9e03bfd0 0x55ef9e03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24b6ffd640 10 client.15663 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24d6ffd640 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9e03bfd0 0x55ef9e03e3c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906e35df256f0f27 server_cookie=f7f6c037f1b408ac in_seq=0 out_seq=0 2024-09-19T05:07:34.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24b6ffd640 1 -- 172.21.15.64:0/1207170947 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef9df23d60 con 0x55ef9e03bfd0 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24b6ffd640 10 client.15663 handle_command_reply: tid=0 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 client.15663 shutdown 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 msgr2=0x55ef9dfc7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc7640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f24d0010590 tx=0x7f24d0007080 comp rx=0 tx=0).stop 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 shutdown_connections 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ef9e03bfd0 0x55ef9e03e3c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef9dfc1680 0x55ef9dfc7640 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef9dfb6dc0 0x55ef9dfc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 --2- 172.21.15.64:0/1207170947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef9dfb64a0 0x55ef9dfc9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24b6ffd640 0 client.15663 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.190+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 >> 172.21.15.64:0/1207170947 conn(0x55ef9dfb2a30 msgr2=0x55ef9dfc0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:34.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 shutdown_connections 2024-09-19T05:07:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24dffbd740 1 -- 172.21.15.64:0/1207170947 wait complete. 2024-09-19T05:07:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24dffbd740 20 client.15663 trim_cache size 0 max 16384 2024-09-19T05:07:34.845 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:07:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:07:34.845 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:07:34.845 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f24c00a3430 msgr2=0x7f24c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f24c00a3430 0x7f24c00a58f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24c80eda70 tx=0x7f24c80f5040 comp rx=0 tx=0).stop 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 msgr2=0x7f24d80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.191+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d80744f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005940 tx=0x7f24c8004270 comp rx=0 tx=0).stop 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 shutdown_connections 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f24c00a3430 0x7f24c00a58f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24d8106f10 0x7f24d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d8102790 0x7f24d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 --2- 172.21.15.64:0/2833540151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8101de0 0x7f24d80744f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 >> 172.21.15.64:0/2833540151 conn(0x7f24d80fd8e0 msgr2=0x7f24d80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 shutdown_connections 2024-09-19T05:07:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:34.192+0000 7f24ddbe4640 1 -- 172.21.15.64:0/2833540151 wait complete. 2024-09-19T05:07:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:35 smithi103 ceph-mon[34446]: pgmap v53: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-19T05:07:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:35 smithi174 ceph-mon[34488]: pgmap v53: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-19T05:07:35.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:35 smithi064 ceph-mon[30960]: pgmap v53: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-19T05:07:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 -- 172.21.15.64:0/3641827218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab0103620 msgr2=0x7fdab010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 --2- 172.21.15.64:0/3641827218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab0103620 0x7fdab010bb00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009900 tx=0x7fdaa402a480 comp rx=0 tx=0).stop 2024-09-19T05:07:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 -- 172.21.15.64:0/3641827218 shutdown_connections 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 --2- 172.21.15.64:0/3641827218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdab010c0d0 0x7fdab010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 --2- 172.21.15.64:0/3641827218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab0103620 0x7fdab010bb00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 --2- 172.21.15.64:0/3641827218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab0102d00 0x7fdab01030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 -- 172.21.15.64:0/3641827218 >> 172.21.15.64:0/3641827218 conn(0x7fdab0077990 msgr2=0x7fdab00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 -- 172.21.15.64:0/3641827218 shutdown_connections 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.943+0000 7fdab5c27640 1 -- 172.21.15.64:0/3641827218 wait complete. 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 Processor -- start 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 -- start start 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdab0102d00 0x7fdab0100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab0103620 0x7fdab0100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 0x7fdab01013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111d60 con 0x7fdab010c0d0 2024-09-19T05:07:36.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111be0 con 0x7fdab0102d00 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.944+0000 7fdab5c27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111ee0 con 0x7fdab0103620 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 0x7fdab01013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 0x7fdab01013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57008/0 (socket says 172.21.15.64:57008) 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 -- 172.21.15.64:0/362313886 learned_addr learned my addr 172.21.15.64:0/362313886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab0103620 0x7fdab0100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdab0102d00 0x7fdab0100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 -- 172.21.15.64:0/362313886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab0103620 msgr2=0x7fdab0100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab0103620 0x7fdab0100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 -- 172.21.15.64:0/362313886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdab0102d00 msgr2=0x7fdab0100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdab0102d00 0x7fdab0100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 -- 172.21.15.64:0/362313886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01a6750 con 0x7fdab010c0d0 2024-09-19T05:07:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaaffff640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 0x7fdab01013d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800eaa0 tx=0x7fdaa800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaacff9640 1 -- 172.21.15.64:0/362313886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800cdb0 con 0x7fdab010c0d0 2024-09-19T05:07:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaacff9640 1 -- 172.21.15.64:0/362313886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa8005020 con 0x7fdab010c0d0 2024-09-19T05:07:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01a6a40 con 0x7fdab010c0d0 2024-09-19T05:07:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdaacff9640 1 -- 172.21.15.64:0/362313886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80116d0 con 0x7fdab010c0d0 2024-09-19T05:07:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.945+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a6f10 con 0x7fdab010c0d0 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.948+0000 7fdaacff9640 1 -- 172.21.15.64:0/362313886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fdaa80051d0 con 0x7fdab010c0d0 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.948+0000 7fdab89c4740 1 Processor -- start 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.948+0000 7fdab89c4740 1 -- start start 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.948+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfec5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfed0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfed30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdab89c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647dfe2a2f0 con 0x5647dfed0cb0 2024-09-19T05:07:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdab89c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5647dfdd2270 con 0x5647dfec5af0 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdab89c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647dfe262b0 con 0x5647dfec6410 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfed30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfed30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57020/0 (socket says 172.21.15.64:57020) 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1010933200 learned_addr learned my addr 172.21.15.64:0/1010933200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfec5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfed0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1010933200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 msgr2=0x5647dfed0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfed0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1010933200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 msgr2=0x5647dfec5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfec5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1010933200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647dfdd1340 con 0x5647dfed0cb0 2024-09-19T05:07:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfed0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaacff9640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fda980a3460 0x7fda980a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfed30a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009900 tx=0x7fdaa4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56effe9bf0e8588 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fda980a3460 0x7fda980a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.949+0000 7fdaacff9640 1 -- 172.21.15.64:0/362313886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdaa80cfe00 con 0x7fdab010c0d0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fda95ffb640 1 -- 172.21.15.64:0/1010933200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4044020 con 0x5647dfed0cb0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fda95ffb640 1 -- 172.21.15.64:0/1010933200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa4031720 con 0x5647dfed0cb0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fda95ffb640 1 -- 172.21.15.64:0/1010933200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4035040 con 0x5647dfed0cb0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 -- 172.21.15.64:0/1010933200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 msgr2=0x5647dfed30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fda980a3460 0x7fda980a5920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5647dfec2ec0 tx=0x7fdaa000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfed30a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009900 tx=0x7fdaa4005c00 comp rx=0 tx=0).stop 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 -- 172.21.15.64:0/1010933200 shutdown_connections 2024-09-19T05:07:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfed30a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfed0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1010933200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfec5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 -- 172.21.15.64:0/1010933200 >> 172.21.15.64:0/1010933200 conn(0x5647dfec2060 msgr2=0x5647dfec2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.950+0000 7fdab89c4740 1 -- 172.21.15.64:0/1010933200 shutdown_connections 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.951+0000 7fdab89c4740 1 -- 172.21.15.64:0/1010933200 wait complete. 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.951+0000 7fdab89c4740 1 -- start start 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.951+0000 7fdab89c4740 1 Processor -- start 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.951+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec5af0 0x5647dfecdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.951+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec6410 0x5647dfece0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.951+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfece630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdab89c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647dfed3880 con 0x5647dfed0cb0 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdab89c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5647dfdb3290 con 0x5647dfec6410 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdab89c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647dfed3a00 con 0x5647dfec5af0 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdab89c4740 10 client.0 _ll_register_callbacks cb 0x5647dfde4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfece630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfece630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57024/0 (socket says 172.21.15.64:57024) 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 -- 172.21.15.64:0/3704798963 learned_addr learned my addr 172.21.15.64:0/3704798963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec6410 0x5647dfece0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec5af0 0x5647dfecdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 -- 172.21.15.64:0/3704798963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec5af0 msgr2=0x5647dfecdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec5af0 0x5647dfecdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 -- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec6410 msgr2=0x5647dfece0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec6410 0x5647dfece0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 -- 172.21.15.64:0/3704798963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647dfb3d210 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdaaffff640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfece630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80cd5b0 tx=0x7fdaa809bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1f2548bb163629 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fda8effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fda8effd640 1 -- 172.21.15.64:0/3704798963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa801f030 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fda8effd640 1 -- 172.21.15.64:0/3704798963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdaa80a0120 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.952+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa4031720 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fda8effd640 1 -- 172.21.15.64:0/3704798963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdaa8098800 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 10 client.15675 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647dfc09620 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fda8effd640 1 -- 172.21.15.64:0/3704798963 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa4031720 con 0x5647dfed0cb0 2024-09-19T05:07:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fda8effd640 1 -- 172.21.15.64:0/3704798963 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdaa800f040 con 0x5647dfed0cb0 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 10 client.15675 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 20 client.15675 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dfee0bc0 0x5647dfee2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 4 client.15675 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647dfe37a20 con 0x5647dfee0bc0 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.953+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dfee0bc0 0x5647dfee2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.954+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dfee0bc0 0x5647dfee2fb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85068ce4ce375fa server_cookie=30a5a7ee54d24ba4 in_seq=0 out_seq=0 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.954+0000 7fda8effd640 10 client.15675 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fda8effd640 1 -- 172.21.15.64:0/3704798963 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647dfe37a20 con 0x5647dfee0bc0 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fda8effd640 10 client.15675 handle_command_reply: tid=0 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 client.15675 shutdown 2024-09-19T05:07:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 msgr2=0x5647dfece630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfece630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80cd5b0 tx=0x7fdaa809bdf0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 shutdown_connections 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dfee0bc0 0x5647dfee2fb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfece630 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec6410 0x5647dfece0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fda8effd640 0 client.15675 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3704798963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec5af0 0x5647dfecdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 >> 172.21.15.64:0/3704798963 conn(0x5647dfec2060 msgr2=0x5647dfec7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 shutdown_connections 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 1 -- 172.21.15.64:0/3704798963 wait complete. 2024-09-19T05:07:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.955+0000 7fdab89c4740 20 client.15675 trim_cache size 0 max 16384 2024-09-19T05:07:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 Processor -- start 2024-09-19T05:07:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 -- start start 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfecfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfece460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfee88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01b21c0 con 0x5647dfed0cb0 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01afa90 con 0x5647dfec5af0 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdab89c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ad8e0 con 0x5647dfec6410 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfee88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfee88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57036/0 (socket says 172.21.15.64:57036) 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 -- 172.21.15.64:0/3948415014 learned_addr learned my addr 172.21.15.64:0/3948415014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfecfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 -- 172.21.15.64:0/3948415014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 msgr2=0x5647dfece460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfece460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 -- 172.21.15.64:0/3948415014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 msgr2=0x5647dfecfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfecfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.970+0000 7fdaaffff640 1 -- 172.21.15.64:0/3948415014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647dfec6240 con 0x5647dfed0cb0 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.971+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfecfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.971+0000 7fdaaffff640 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfee88e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdaa809be90 tx=0x7fdaa8009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb884a2b9361f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.971+0000 7fda957fa640 1 -- 172.21.15.64:0/3948415014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa807f070 con 0x5647dfed0cb0 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.971+0000 7fda957fa640 1 -- 172.21.15.64:0/3948415014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa809b050 con 0x5647dfed0cb0 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.971+0000 7fda957fa640 1 -- 172.21.15.64:0/3948415014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa801f070 con 0x5647dfed0cb0 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 -- 172.21.15.64:0/3948415014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 msgr2=0x5647dfee88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfee88e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdaa809be90 tx=0x7fdaa8009730 comp rx=0 tx=0).stop 2024-09-19T05:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 -- 172.21.15.64:0/3948415014 shutdown_connections 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfed0cb0 0x5647dfee88e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfec6410 0x5647dfece460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 --2- 172.21.15.64:0/3948415014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfecfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 -- 172.21.15.64:0/3948415014 >> 172.21.15.64:0/3948415014 conn(0x5647dfec2060 msgr2=0x5647dfec7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 -- 172.21.15.64:0/3948415014 shutdown_connections 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 -- 172.21.15.64:0/3948415014 wait complete. 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 -- start start 2024-09-19T05:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.972+0000 7fdab89c4740 1 Processor -- start 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfed8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 0x5647dfed9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfed0cb0 0x5647dfed5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01adf50 con 0x5647dfec6410 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01af3c0 con 0x5647dfec5af0 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01ae620 con 0x5647dfed0cb0 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 10 client.0 _ll_register_callbacks cb 0x7fdaa8009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 0x5647dfed9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 0x5647dfed9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57044/0 (socket says 172.21.15.64:57044) 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1698134172 learned_addr learned my addr 172.21.15.64:0/1698134172 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfed8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1698134172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfed0cb0 msgr2=0x5647dfed5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfed0cb0 0x5647dfed5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 msgr2=0x5647dfed8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfed8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 -- 172.21.15.64:0/1698134172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647dfed61d0 con 0x5647dfec6410 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdaaeffd640 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 0x5647dfed9200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004650 tx=0x7fdaa4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fc9ad272403bab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fda8effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fda8effd640 1 -- 172.21.15.64:0/1698134172 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa404e060 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fda8effd640 1 -- 172.21.15.64:0/1698134172 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdaa4004950 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.973+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa809b050 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fda8effd640 1 -- 172.21.15.64:0/1698134172 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdaa4044070 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 10 client.15681 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647dff4c010 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fda8effd640 1 -- 172.21.15.64:0/1698134172 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa809b050 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fda8effd640 1 -- 172.21.15.64:0/1698134172 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdaa4031b50 con 0x5647dfec6410 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 10 client.15681 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 20 client.15681 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dff4c2f0 0x5647dff4e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 4 client.15681 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647dfe31d70 con 0x5647dff4c2f0 2024-09-19T05:07:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.974+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dff4c2f0 0x5647dff4e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.975+0000 7fda8effd640 10 client.15681 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.975+0000 7fdaaf7fe640 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dff4c2f0 0x5647dff4e710 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e907fd4c761d86 server_cookie=7bede27b8daa7190 in_seq=0 out_seq=0 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fda8effd640 1 -- 172.21.15.64:0/1698134172 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647dfe31d70 con 0x5647dff4c2f0 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fda8effd640 10 client.15681 handle_command_reply: tid=0 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 client.15681 shutdown 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 msgr2=0x5647dfed9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 0x5647dfed9200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004650 tx=0x7fdaa4031400 comp rx=0 tx=0).stop 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 shutdown_connections 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647dfed0cb0 0x5647dfed5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5647dff4c2f0 0x5647dff4e710 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647dfec6410 0x5647dfed9200 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 --2- 172.21.15.64:0/1698134172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5647dfec5af0 0x5647dfed8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 >> 172.21.15.64:0/1698134172 conn(0x5647dfec2060 msgr2=0x5647dfec7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fda8effd640 0 client.15681 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 shutdown_connections 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 1 -- 172.21.15.64:0/1698134172 wait complete. 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.983+0000 7fdab89c4740 20 client.15681 trim_cache size 0 max 16384 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:07:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:07:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:07:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fda980a3460 msgr2=0x7fda980a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fda980a3460 0x7fda980a5920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5647dfec2ec0 tx=0x7fdaa000b390 comp rx=0 tx=0).stop 2024-09-19T05:07:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 msgr2=0x7fdab01013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 0x7fdab01013d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800eaa0 tx=0x7fdaa800ef60 comp rx=0 tx=0).stop 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 shutdown_connections 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fda980a3460 0x7fda980a5920 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdab010c0d0 0x7fdab01013d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab0103620 0x7fdab0100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 --2- 172.21.15.64:0/362313886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdab0102d00 0x7fdab0100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 >> 172.21.15.64:0/362313886 conn(0x7fdab0077990 msgr2=0x7fdab00fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 shutdown_connections 2024-09-19T05:07:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:36.984+0000 7fdab5c27640 1 -- 172.21.15.64:0/362313886 wait complete. 2024-09-19T05:07:37.115 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:07:37.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:07:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:37.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:37 smithi064 ceph-mon[30960]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-19T05:07:37.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:37 smithi103 ceph-mon[34446]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-19T05:07:37.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:37 smithi174 ceph-mon[34488]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-19T05:07:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 -- 172.21.15.64:0/2991206311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 msgr2=0x7f0580101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 --2- 172.21.15.64:0/2991206311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f0580101200 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0570009900 tx=0x7f057002a480 comp rx=0 tx=0).stop 2024-09-19T05:07:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 -- 172.21.15.64:0/2991206311 shutdown_connections 2024-09-19T05:07:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 --2- 172.21.15.64:0/2991206311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0580102ef0 0x7f058010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 --2- 172.21.15.64:0/2991206311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0580102570 0x7f05801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 --2- 172.21.15.64:0/2991206311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f0580101200 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 -- 172.21.15.64:0/2991206311 >> 172.21.15.64:0/2991206311 conn(0x7f05800fc7d0 msgr2=0x7f05800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 -- 172.21.15.64:0/2991206311 shutdown_connections 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.995+0000 7f058645a640 1 -- 172.21.15.64:0/2991206311 wait complete. 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 Processor -- start 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 -- start start 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f05801992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0580102570 0x7f0580199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0580102ef0 0x7f05801a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0580110e80 con 0x7f0580100e20 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0580110d00 con 0x7f0580102570 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f058645a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0580111000 con 0x7f0580102ef0 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f057ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f05801992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:38.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f057ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f05801992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57068/0 (socket says 172.21.15.64:57068) 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f057ffff640 1 -- 172.21.15.64:0/2856430674 learned_addr learned my addr 172.21.15.64:0/2856430674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f05849f7640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0580102ef0 0x7f05801a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.996+0000 7f057f7fe640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0580102570 0x7f0580199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057ffff640 1 -- 172.21.15.64:0/2856430674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0580102ef0 msgr2=0x7f05801a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057ffff640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0580102ef0 0x7f05801a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057ffff640 1 -- 172.21.15.64:0/2856430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0580102570 msgr2=0x7f0580199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057ffff640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0580102570 0x7f0580199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057ffff640 1 -- 172.21.15.64:0/2856430674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05801a4890 con 0x7f0580100e20 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057ffff640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f05801992d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0570004770 tx=0x7f0570003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057003f070 con 0x7f0580100e20 2024-09-19T05:07:38.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05801a4a60 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f057003bd20 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570035030 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.997+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05801a4f40 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:37.999+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0580101200 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.000+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0570042070 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.001+0000 7f057d7fa640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05680a33b0 0x7f05680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.001+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f05700f0470 con 0x7f0580100e20 2024-09-19T05:07:38.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.001+0000 7f057f7fe640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05680a33b0 0x7f05680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:38.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.001+0000 7f057f7fe640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05680a33b0 0x7f05680a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f058006ab00 tx=0x7f057400b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:38.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.005+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05700bc800 con 0x7f0580100e20 2024-09-19T05:07:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.280+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0580109090 con 0x7f0580100e20 2024-09-19T05:07:38.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.281+0000 7f057d7fa640 1 -- 172.21.15.64:0/2856430674 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f05700c0b00 con 0x7f0580100e20 2024-09-19T05:07:38.283 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:07:38.283 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:07:38.283 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.283+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05680a33b0 msgr2=0x7f05680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.283+0000 7f058645a640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05680a33b0 0x7f05680a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f058006ab00 tx=0x7f057400b390 comp rx=0 tx=0).stop 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.283+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 msgr2=0x7f05801992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.283+0000 7f058645a640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f05801992d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0570004770 tx=0x7f0570003ca0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 shutdown_connections 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05680a33b0 0x7f05680a5870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0580102ef0 0x7f05801a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0580102570 0x7f0580199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 --2- 172.21.15.64:0/2856430674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0580100e20 0x7f05801992d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:38.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 >> 172.21.15.64:0/2856430674 conn(0x7f05800fc7d0 msgr2=0x7f05800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:38.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 shutdown_connections 2024-09-19T05:07:38.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:38.284+0000 7f058645a640 1 -- 172.21.15.64:0/2856430674 wait complete. 2024-09-19T05:07:38.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:38 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2856430674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:38.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:38 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2856430674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:38.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:38 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2856430674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:38.993 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:07:39.267 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:39 smithi064 ceph-mon[30960]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T05:07:39.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:39 smithi103 ceph-mon[34446]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T05:07:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:39 smithi174 ceph-mon[34488]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.780+0000 7f4b339cd640 1 -- 172.21.15.64:0/3251964364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c102d30 msgr2=0x7f4b2c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.780+0000 7f4b339cd640 1 --2- 172.21.15.64:0/3251964364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c102d30 0x7f4b2c103110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009930 tx=0x7f4b2802a420 comp rx=0 tx=0).stop 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 -- 172.21.15.64:0/3251964364 shutdown_connections 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 --2- 172.21.15.64:0/3251964364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2c1082a0 0x7f4b2c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 --2- 172.21.15.64:0/3251964364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 --2- 172.21.15.64:0/3251964364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c102d30 0x7f4b2c103110 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 -- 172.21.15.64:0/3251964364 >> 172.21.15.64:0/3251964364 conn(0x7f4b2c0fc7d0 msgr2=0x7f4b2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 -- 172.21.15.64:0/3251964364 shutdown_connections 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 -- 172.21.15.64:0/3251964364 wait complete. 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.781+0000 7f4b339cd640 1 Processor -- start 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 -- start start 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b2c102d30 0x7f4b2c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2c1082a0 0x7f4b2c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c111fa0 con 0x7f4b2c1036e0 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c111e20 con 0x7f4b2c102d30 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b339cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c112120 con 0x7f4b2c1082a0 2024-09-19T05:07:39.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57084/0 (socket says 172.21.15.64:57084) 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 -- 172.21.15.64:0/1011321113 learned_addr learned my addr 172.21.15.64:0/1011321113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2c1082a0 0x7f4b2c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b31769640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b2c102d30 0x7f4b2c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 -- 172.21.15.64:0/1011321113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2c1082a0 msgr2=0x7f4b2c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2c1082a0 0x7f4b2c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 -- 172.21.15.64:0/1011321113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b2c102d30 msgr2=0x7f4b2c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b2c102d30 0x7f4b2c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.782+0000 7f4b30f68640 1 -- 172.21.15.64:0/1011321113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1a69a0 con 0x7f4b2c1036e0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.783+0000 7f4b30f68640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c100ed0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00e9f0 tx=0x7f4b1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.783+0000 7f4b227fc640 1 -- 172.21.15.64:0/1011321113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00ccc0 con 0x7f4b2c1036e0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.783+0000 7f4b227fc640 1 -- 172.21.15.64:0/1011321113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b1c0108e0 con 0x7f4b2c1036e0 2024-09-19T05:07:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.783+0000 7f4b227fc640 1 -- 172.21.15.64:0/1011321113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0187e0 con 0x7f4b2c1036e0 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.783+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1a6c90 con 0x7f4b2c1036e0 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.783+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a7180 con 0x7f4b2c1036e0 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b227fc640 1 -- 172.21.15.64:0/1011321113 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4b1c010a90 con 0x7f4b2c1036e0 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 Processor -- start 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 -- start start 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0db32f0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0d5acf0 con 0x55e0a0e58370 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.786+0000 7f4b35da6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0daf2b0 con 0x55e0a0e4eb40 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e4da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57092/0 (socket says 172.21.15.64:57092) 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 -- 172.21.15.64:0/1115435128 learned_addr learned my addr 172.21.15.64:0/1115435128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b227fc640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b180a3430 0x7f4b180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b30f68640 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b180a3430 0x7f4b180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 -- 172.21.15.64:0/1115435128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 msgr2=0x55e0a0e4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 -- 172.21.15.64:0/1115435128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 msgr2=0x55e0a0e5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 -- 172.21.15.64:0/1115435128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a0d3f3a0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b227fc640 1 -- 172.21.15.64:0/1011321113 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4b1c0cfff0 con 0x7f4b2c1036e0 2024-09-19T05:07:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.787+0000 7f4b31769640 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e4da10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b28006010 tx=0x7f4b28004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba6ce43f9c62253 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.788+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b180a3430 0x7f4b180a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005fd0 tx=0x7f4b24004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.788+0000 7f4b177fe640 1 -- 172.21.15.64:0/1115435128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28035040 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.788+0000 7f4b177fe640 1 -- 172.21.15.64:0/1115435128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b280054f0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.788+0000 7f4b177fe640 1 -- 172.21.15.64:0/1115435128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280035f0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 -- 172.21.15.64:0/1115435128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 msgr2=0x55e0a0e4da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e4da10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b28006010 tx=0x7f4b28004170 comp rx=0 tx=0).stop 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 -- 172.21.15.64:0/1115435128 shutdown_connections 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e4da10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 --2- 172.21.15.64:0/1115435128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 -- 172.21.15.64:0/1115435128 >> 172.21.15.64:0/1115435128 conn(0x55e0a0e16310 msgr2=0x55e0a0e49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 -- 172.21.15.64:0/1115435128 shutdown_connections 2024-09-19T05:07:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 -- 172.21.15.64:0/1115435128 wait complete. 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 -- start start 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.789+0000 7f4b35da6740 1 Processor -- start 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0e5b010 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0d6e060 con 0x55e0a0e58370 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0a0e5b190 con 0x55e0a0e4eb40 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b35da6740 10 client.0 _ll_register_callbacks cb 0x55e0a0ac18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e57d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57094/0 (socket says 172.21.15.64:57094) 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 -- 172.21.15.64:0/3750507609 learned_addr learned my addr 172.21.15.64:0/3750507609 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 -- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 msgr2=0x55e0a0e54fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e54fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 -- 172.21.15.64:0/3750507609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 msgr2=0x55e0a0e55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31769640 1 -- 172.21.15.64:0/3750507609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a0b243c0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.790+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b31769640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e57d10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005c40 tx=0x7f4b28005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49327102f86a858c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b14ff9640 1 -- 172.21.15.64:0/3750507609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28035050 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b14ff9640 1 -- 172.21.15.64:0/3750507609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4b28004d80 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b280054f0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b14ff9640 1 -- 172.21.15.64:0/3750507609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b2805b050 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b35da6740 10 client.15699 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0a0e66a90 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.791+0000 7f4b14ff9640 1 -- 172.21.15.64:0/3750507609 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b280054f0 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b14ff9640 1 -- 172.21.15.64:0/3750507609 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4b28056030 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b35da6740 10 client.15699 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b35da6740 10 client.15699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b35da6740 20 client.15699 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0e67de0 0x55e0a0e6a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b35da6740 4 client.15699 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a0dc0860 con 0x55e0a0e67de0 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.792+0000 7f4b30f68640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0e67de0 0x55e0a0e6a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.794+0000 7f4b14ff9640 10 client.15699 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.794+0000 7f4b30f68640 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0e67de0 0x55e0a0e6a1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a56b4dcdb9d75cc5 server_cookie=5af8ebb9ac95b5ae in_seq=0 out_seq=0 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b14ff9640 1 -- 172.21.15.64:0/3750507609 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0a0dc0860 con 0x55e0a0e67de0 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b14ff9640 10 client.15699 handle_command_reply: tid=0 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 client.15699 shutdown 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 msgr2=0x55e0a0e57d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e57d10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005c40 tx=0x7f4b28005850 comp rx=0 tx=0).stop 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 shutdown_connections 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0e67de0 0x55e0a0e6a1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e58370 0x55e0a0e55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e54fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3750507609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e57d10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 >> 172.21.15.64:0/3750507609 conn(0x55e0a0e16310 msgr2=0x55e0a0e4f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b14ff9640 0 client.15699 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 shutdown_connections 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 1 -- 172.21.15.64:0/3750507609 wait complete. 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.795+0000 7f4b35da6740 20 client.15699 trim_cache size 0 max 16384 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 Processor -- start 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 -- start start 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e50120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e50660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b2230 con 0x55e0a0e58370 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1afb00 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.808+0000 7f4b35da6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1ad950 con 0x55e0a0e4eb40 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57096/0 (socket says 172.21.15.64:57096) 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 -- 172.21.15.64:0/3682707240 learned_addr learned my addr 172.21.15.64:0/3682707240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31769640 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e50120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b30f68640 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e50660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 -- 172.21.15.64:0/3682707240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 msgr2=0x55e0a0e50660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e50660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 -- 172.21.15.64:0/3682707240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 msgr2=0x55e0a0e50120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e50120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 -- 172.21.15.64:0/3682707240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a0e58080 con 0x55e0a0e58370 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6ce10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b24012a80 tx=0x7f4b24009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29d86b8dd7cb2f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b16ffd640 1 -- 172.21.15.64:0/3682707240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2401a070 con 0x55e0a0e58370 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b16ffd640 1 -- 172.21.15.64:0/3682707240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b24026030 con 0x55e0a0e58370 2024-09-19T05:07:39.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.809+0000 7f4b16ffd640 1 -- 172.21.15.64:0/3682707240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24022700 con 0x55e0a0e58370 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 -- 172.21.15.64:0/3682707240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 msgr2=0x55e0a0e6ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6ce10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b24012a80 tx=0x7f4b24009ab0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 -- 172.21.15.64:0/3682707240 shutdown_connections 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6ce10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e50660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 --2- 172.21.15.64:0/3682707240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e50120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 -- 172.21.15.64:0/3682707240 >> 172.21.15.64:0/3682707240 conn(0x55e0a0e16310 msgr2=0x55e0a0e540b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 -- 172.21.15.64:0/3682707240 shutdown_connections 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.810+0000 7f4b35da6740 1 -- 172.21.15.64:0/3682707240 wait complete. 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 -- start start 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 Processor -- start 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e6a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e6ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1adfc0 con 0x55e0a0e58370 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1af430 con 0x55e0a0e4d5d0 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1ae690 con 0x55e0a0e4eb40 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.811+0000 7f4b35da6740 10 client.0 _ll_register_callbacks cb 0x7f4b1c097f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57102/0 (socket says 172.21.15.64:57102) 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 -- 172.21.15.64:0/684022965 learned_addr learned my addr 172.21.15.64:0/684022965 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31769640 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e6a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 -- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 msgr2=0x55e0a0e6ab10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e6ab10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 -- 172.21.15.64:0/684022965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 msgr2=0x55e0a0e6a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e6a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 -- 172.21.15.64:0/684022965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a0e6d320 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b31f6a640 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6cc30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005c10 tx=0x7f4b24014590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a9ed0b016e92dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b14ff9640 1 -- 172.21.15.64:0/684022965 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24038060 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b14ff9640 1 -- 172.21.15.64:0/684022965 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4b240119b0 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.812+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b24026030 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b14ff9640 1 -- 172.21.15.64:0/684022965 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b2401a040 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 10 client.15705 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0a0ed22c0 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b14ff9640 1 -- 172.21.15.64:0/684022965 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b24026030 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b14ff9640 1 -- 172.21.15.64:0/684022965 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4b2401d070 con 0x55e0a0e58370 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 10 client.15705 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:39.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 10 client.15705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 20 client.15705 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0ed2920 0x55e0a0ed4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 4 client.15705 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0a0dbad70 con 0x55e0a0ed2920 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.813+0000 7f4b31769640 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0ed2920 0x55e0a0ed4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b14ff9640 10 client.15705 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b31769640 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0ed2920 0x55e0a0ed4d10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=926b74034f6fb53f server_cookie=a3835dddfd047116 in_seq=0 out_seq=0 2024-09-19T05:07:39.837 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b14ff9640 1 -- 172.21.15.64:0/684022965 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0a0dbad70 con 0x55e0a0ed2920 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b14ff9640 10 client.15705 handle_command_reply: tid=0 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 client.15705 shutdown 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 msgr2=0x55e0a0e6cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6cc30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005c10 tx=0x7f4b24014590 comp rx=0 tx=0).stop 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 shutdown_connections 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e0a0ed2920 0x55e0a0ed4d10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0a0e58370 0x55e0a0e6cc30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0a0e4eb40 0x55e0a0e6ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 --2- 172.21.15.64:0/684022965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0a0e4d5d0 0x55e0a0e6a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 >> 172.21.15.64:0/684022965 conn(0x55e0a0e16310 msgr2=0x55e0a0e484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b14ff9640 0 client.15705 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 shutdown_connections 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.814+0000 7f4b35da6740 1 -- 172.21.15.64:0/684022965 wait complete. 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b35da6740 20 client.15705 trim_cache size 0 max 16384 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b180a3430 msgr2=0x7f4b180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b180a3430 0x7f4b180a58f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005fd0 tx=0x7f4b24004ab0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 msgr2=0x7f4b2c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c100ed0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00e9f0 tx=0x7f4b1c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 shutdown_connections 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4b180a3430 0x7f4b180a58f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2c1082a0 0x7f4b2c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b2c1036e0 0x7f4b2c100ed0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 --2- 172.21.15.64:0/1011321113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b2c102d30 0x7f4b2c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 >> 172.21.15.64:0/1011321113 conn(0x7f4b2c0fc7d0 msgr2=0x7f4b2c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 shutdown_connections 2024-09-19T05:07:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:39.815+0000 7f4b339cd640 1 -- 172.21.15.64:0/1011321113 wait complete. 2024-09-19T05:07:40.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:07:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:41.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 -- 172.21.15.64:0/1284424405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 msgr2=0x7f69641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 --2- 172.21.15.64:0/1284424405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f69641067e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6958009900 tx=0x7f695802a410 comp rx=0 tx=0).stop 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 -- 172.21.15.64:0/1284424405 shutdown_connections 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 --2- 172.21.15.64:0/1284424405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6964106f10 0x7f696410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 --2- 172.21.15.64:0/1284424405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f69641067e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 --2- 172.21.15.64:0/1284424405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6964101de0 0x7f69641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 -- 172.21.15.64:0/1284424405 >> 172.21.15.64:0/1284424405 conn(0x7f69640fd8e0 msgr2=0x7f69640ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 -- 172.21.15.64:0/1284424405 shutdown_connections 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.359+0000 7f69698ba640 1 -- 172.21.15.64:0/1284424405 wait complete. 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.360+0000 7f69698ba640 1 Processor -- start 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.360+0000 7f69698ba640 1 -- start start 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69698ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6964101de0 0x7f69640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69698ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f6964074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69698ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6964106f10 0x7f6964074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69698ba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6964111de0 con 0x7f6964102790 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69698ba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6964111c60 con 0x7f6964101de0 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69698ba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6964111f60 con 0x7f6964106f10 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f6964074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f6964074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57124/0 (socket says 172.21.15.64:57124) 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 -- 172.21.15.64:0/2235764592 learned_addr learned my addr 172.21.15.64:0/2235764592 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f6962ffd640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6964101de0 0x7f69640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69637fe640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6964106f10 0x7f6964074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 -- 172.21.15.64:0/2235764592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6964106f10 msgr2=0x7f6964074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6964106f10 0x7f6964074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 -- 172.21.15.64:0/2235764592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6964101de0 msgr2=0x7f69640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6964101de0 0x7f69640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.361+0000 7f69627fc640 1 -- 172.21.15.64:0/2235764592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69641a6820 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.362+0000 7f69627fc640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f6964074a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6958005940 tx=0x7f69580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.362+0000 7f69688b8640 1 -- 172.21.15.64:0/2235764592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695803f030 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.362+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69641a6ab0 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.362+0000 7f69688b8640 1 -- 172.21.15.64:0/2235764592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6958003cb0 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.362+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69641a6f90 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.362+0000 7f69688b8640 1 -- 172.21.15.64:0/2235764592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958049060 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f69688b8640 1 -- 172.21.15.64:0/2235764592 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6958042030 con 0x7f6964102790 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 Processor -- start 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 -- start start 2024-09-19T05:07:41.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cad9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cad9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55871ca3f2f0 con 0x55871cadab40 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55871c9e6cf0 con 0x55871cae4370 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f696bc93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55871ca3b2b0 con 0x55871cad95d0 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f69637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cad9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f69637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cad9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57132/0 (socket says 172.21.15.64:57132) 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.365+0000 7f69637fe640 1 -- 172.21.15.64:0/2797594699 learned_addr learned my addr 172.21.15.64:0/2797594699 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69627fc640 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f6962ffd640 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cad9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69688b8640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f694c0a33b0 0x7f694c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 -- 172.21.15.64:0/2797594699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 msgr2=0x55871cad9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cad9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 -- 172.21.15.64:0/2797594699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 msgr2=0x55871cae6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 -- 172.21.15.64:0/2797594699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871c9cb3a0 con 0x55871cadab40 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69688b8640 1 -- 172.21.15.64:0/2235764592 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f69580f00c0 con 0x7f6964102790 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f694c0a33b0 0x7f694c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.366+0000 7f69637fe640 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cad9090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f695c010440 tx=0x7f695c010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e911db6b828b3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:41.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f6949ffb640 1 -- 172.21.15.64:0/2797594699 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c01d400 con 0x55871cadab40 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f6949ffb640 1 -- 172.21.15.64:0/2797594699 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695c01da50 con 0x55871cadab40 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f6949ffb640 1 -- 172.21.15.64:0/2797594699 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c01c780 con 0x55871cadab40 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 -- 172.21.15.64:0/2797594699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 msgr2=0x55871cad9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cad9090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f695c010440 tx=0x7f695c010900 comp rx=0 tx=0).stop 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f69637fe640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f694c0a33b0 0x7f694c0a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f695c021000 tx=0x7f695c01e000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 -- 172.21.15.64:0/2797594699 shutdown_connections 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cad9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 --2- 172.21.15.64:0/2797594699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cad9090 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.367+0000 7f696bc93740 1 -- 172.21.15.64:0/2797594699 >> 172.21.15.64:0/2797594699 conn(0x55871caa2310 msgr2=0x55871cad5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 -- 172.21.15.64:0/2797594699 shutdown_connections 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 -- 172.21.15.64:0/2797594699 wait complete. 2024-09-19T05:07:41.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 -- start start 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 Processor -- start 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cae3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cae1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55871cae7070 con 0x55871cadab40 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55871c9fa060 con 0x55871cae4370 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55871cae71f0 con 0x55871cad95d0 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.368+0000 7f696bc93740 10 client.0 _ll_register_callbacks cb 0x55871c74d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cae1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cae1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57136/0 (socket says 172.21.15.64:57136) 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 -- 172.21.15.64:0/3478439789 learned_addr learned my addr 172.21.15.64:0/3478439789 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:41.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69637fe640 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 -- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 msgr2=0x55871cae3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cae3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 -- 172.21.15.64:0/3478439789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 msgr2=0x55871cae1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 -- 172.21.15.64:0/3478439789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871c7b03c0 con 0x55871cadab40 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f69627fc640 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cae1030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f69580bc150 tx=0x7f6958003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee64c65bfaa63143 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f6942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f6942ffd640 1 -- 172.21.15.64:0/3478439789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958035040 con 0x55871cadab40 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f6942ffd640 1 -- 172.21.15.64:0/3478439789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f69580bc6b0 con 0x55871cadab40 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.369+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c01da50 con 0x55871cadab40 2024-09-19T05:07:41.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f6942ffd640 1 -- 172.21.15.64:0/3478439789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69580044e0 con 0x55871cadab40 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55871caf2b70 con 0x55871cadab40 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f6942ffd640 1 -- 172.21.15.64:0/3478439789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695c01da50 con 0x55871cadab40 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f6942ffd640 1 -- 172.21.15.64:0/3478439789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69580c4b80 con 0x55871cadab40 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 10 client.15717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 20 client.15717 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871caf4020 0x55871caf64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 4 client.15717 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871ca4c860 con 0x55871caf4020 2024-09-19T05:07:41.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.370+0000 7f6962ffd640 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871caf4020 0x55871caf64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.371+0000 7f6942ffd640 10 client.15717 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.371+0000 7f6962ffd640 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871caf4020 0x55871caf64b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a62a10ed8edfa86 server_cookie=cf051a0573cdbaee in_seq=0 out_seq=0 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f6942ffd640 1 -- 172.21.15.64:0/3478439789 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55871ca4c860 con 0x55871caf4020 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f6942ffd640 10 client.15717 handle_command_reply: tid=0 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 client.15717 shutdown 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 msgr2=0x55871cae1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cae1030 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f69580bc150 tx=0x7f6958003f20 comp rx=0 tx=0).stop 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 shutdown_connections 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cae1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871caf4020 0x55871caf64b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cadab40 0x55871cae1030 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 --2- 172.21.15.64:0/3478439789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cae3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 >> 172.21.15.64:0/3478439789 conn(0x55871caa2310 msgr2=0x55871cadb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f6942ffd640 0 client.15717 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 shutdown_connections 2024-09-19T05:07:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 1 -- 172.21.15.64:0/3478439789 wait complete. 2024-09-19T05:07:41.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.372+0000 7f696bc93740 20 client.15717 trim_cache size 0 max 16384 2024-09-19T05:07:41.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 Processor -- start 2024-09-19T05:07:41.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 -- start start 2024-09-19T05:07:41.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cadc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cadab40 0x55871cadc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 0x55871cafbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69641b2220 con 0x55871cae4370 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69641afaf0 con 0x55871cadab40 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.392+0000 7f696bc93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69641ad940 con 0x55871cad95d0 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 0x55871cafbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 0x55871cafbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57138/0 (socket says 172.21.15.64:57138) 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 -- 172.21.15.64:0/936527355 learned_addr learned my addr 172.21.15.64:0/936527355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69627fc640 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cadab40 0x55871cadc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f6962ffd640 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cadc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 -- 172.21.15.64:0/936527355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 msgr2=0x55871cadc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cadc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 -- 172.21.15.64:0/936527355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cadab40 msgr2=0x55871cadc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cadab40 0x55871cadc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 -- 172.21.15.64:0/936527355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871cae1970 con 0x55871cae4370 2024-09-19T05:07:41.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.393+0000 7f69637fe640 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 0x55871cafbd80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f695c010e00 tx=0x7f695c028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c264eb1fd599d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f69497fa640 1 -- 172.21.15.64:0/936527355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0134c0 con 0x55871cae4370 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f69497fa640 1 -- 172.21.15.64:0/936527355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695c036070 con 0x55871cae4370 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f69497fa640 1 -- 172.21.15.64:0/936527355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c003c60 con 0x55871cae4370 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 -- 172.21.15.64:0/936527355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 msgr2=0x55871cafbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 0x55871cafbd80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f695c010e00 tx=0x7f695c028000 comp rx=0 tx=0).stop 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 -- 172.21.15.64:0/936527355 shutdown_connections 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cae4370 0x55871cafbd80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cadab40 0x55871cadc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 --2- 172.21.15.64:0/936527355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cad95d0 0x55871cadc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 -- 172.21.15.64:0/936527355 >> 172.21.15.64:0/936527355 conn(0x55871caa2310 msgr2=0x55871cae30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.394+0000 7f696bc93740 1 -- 172.21.15.64:0/936527355 shutdown_connections 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.395+0000 7f696bc93740 1 -- 172.21.15.64:0/936527355 wait complete. 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.395+0000 7f696bc93740 1 -- start start 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.395+0000 7f696bc93740 1 Processor -- start 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.395+0000 7f696bc93740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 0x55871cae88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.395+0000 7f696bc93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cadab40 0x55871cae8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.395+0000 7f696bc93740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cb5dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f696bc93740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69641adfb0 con 0x55871cad95d0 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f696bc93740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69641af420 con 0x55871cae4370 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f696bc93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69641ae680 con 0x55871cadab40 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f696bc93740 10 client.0 _ll_register_callbacks cb 0x7f6958005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 0x55871cae88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 0x55871cae88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57154/0 (socket says 172.21.15.64:57154) 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 -- 172.21.15.64:0/565183428 learned_addr learned my addr 172.21.15.64:0/565183428 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f69637fe640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cb5dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f69627fc640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cadab40 0x55871cae8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 -- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cadab40 msgr2=0x55871cae8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cadab40 0x55871cae8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 -- 172.21.15.64:0/565183428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 msgr2=0x55871cb5dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:41.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cb5dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 -- 172.21.15.64:0/565183428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871cb5e290 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f69627fc640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cadab40 0x55871cae8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6962ffd640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 0x55871cae88a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55871caf6cc0 tx=0x7f6954004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f531aea550ba11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.396+0000 7f6942ffd640 1 -- 172.21.15.64:0/565183428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954020070 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f6942ffd640 1 -- 172.21.15.64:0/565183428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f695401c310 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c036070 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f6942ffd640 1 -- 172.21.15.64:0/565183428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f695402c050 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 10 client.15723 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55871cb5f550 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f6942ffd640 1 -- 172.21.15.64:0/565183428 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695401c9c0 con 0x55871cad95d0 2024-09-19T05:07:41.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f6942ffd640 1 -- 172.21.15.64:0/565183428 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69540108a0 con 0x55871cad95d0 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 10 client.15723 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 10 client.15723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 20 client.15723 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871cb5fc50 0x55871cb62070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 4 client.15723 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.397+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f695c036070 con 0x55871cb5fc50 2024-09-19T05:07:41.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.398+0000 7f69627fc640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871cb5fc50 0x55871cb62070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:41.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.398+0000 7f69627fc640 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871cb5fc50 0x55871cb62070 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f325ae229114329 server_cookie=52c08545efdab3b7 in_seq=0 out_seq=0 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.398+0000 7f6942ffd640 10 client.15723 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.398+0000 7f6942ffd640 1 -- 172.21.15.64:0/565183428 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f695c036070 con 0x55871cb5fc50 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.398+0000 7f6942ffd640 10 client.15723 handle_command_reply: tid=0 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.398+0000 7f696bc93740 1 client.15723 shutdown 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 msgr2=0x55871cae88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 0x55871cae88a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55871caf6cc0 tx=0x7f6954004410 comp rx=0 tx=0).stop 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 shutdown_connections 2024-09-19T05:07:41.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55871cae4370 0x55871cb5dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55871cadab40 0x55871cae8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55871cb5fc50 0x55871cb62070 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f6942ffd640 0 client.15723 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 --2- 172.21.15.64:0/565183428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55871cad95d0 0x55871cae88a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 >> 172.21.15.64:0/565183428 conn(0x55871caa2310 msgr2=0x55871cad4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 shutdown_connections 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 1 -- 172.21.15.64:0/565183428 wait complete. 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f696bc93740 20 client.15723 trim_cache size 0 max 16384 2024-09-19T05:07:41.406 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:07:41.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f694c0a33b0 msgr2=0x7f694c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f69698ba640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f694c0a33b0 0x7f694c0a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f695c021000 tx=0x7f695c01e000 comp rx=0 tx=0).stop 2024-09-19T05:07:41.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 msgr2=0x7f6964074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:41.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.399+0000 7f69698ba640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f6964074a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6958005940 tx=0x7f69580042f0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 shutdown_connections 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f694c0a33b0 0x7f694c0a5870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6964106f10 0x7f6964074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6964102790 0x7f6964074a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 --2- 172.21.15.64:0/2235764592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6964101de0 0x7f69640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 >> 172.21.15.64:0/2235764592 conn(0x7f69640fd8e0 msgr2=0x7f69640fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 shutdown_connections 2024-09-19T05:07:41.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:41.400+0000 7f69698ba640 1 -- 172.21.15.64:0/2235764592 wait complete. 2024-09-19T05:07:41.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:41 smithi064 ceph-mon[30960]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T05:07:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:41 smithi174 ceph-mon[34488]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T05:07:41.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:41 smithi103 ceph-mon[34446]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T05:07:42.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:07:42.426 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:42.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 -- 172.21.15.64:0/1327215372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 msgr2=0x7f0cf81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/1327215372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf81067e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce002a410 comp rx=0 tx=0).stop 2024-09-19T05:07:42.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 -- 172.21.15.64:0/1327215372 shutdown_connections 2024-09-19T05:07:42.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/1327215372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cf8106f10 0x7f0cf810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/1327215372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf81067e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/1327215372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf8101de0 0x7f0cf81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 -- 172.21.15.64:0/1327215372 >> 172.21.15.64:0/1327215372 conn(0x7f0cf80fd8e0 msgr2=0x7f0cf80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 -- 172.21.15.64:0/1327215372 shutdown_connections 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.919+0000 7f0cfce0d640 1 -- 172.21.15.64:0/1327215372 wait complete. 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 Processor -- start 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 -- start start 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf8101de0 0x7f0cf80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cf8106f10 0x7f0cf8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8111e60 con 0x7f0cf8102790 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8111ce0 con 0x7f0cf8106f10 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cfce0d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8111fe0 con 0x7f0cf8101de0 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57172/0 (socket says 172.21.15.64:57172) 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0ceffff640 1 -- 172.21.15.64:0/3109945125 learned_addr learned my addr 172.21.15.64:0/3109945125 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.920+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cf8106f10 0x7f0cf8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0ceffff640 1 -- 172.21.15.64:0/3109945125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf8101de0 msgr2=0x7f0cf80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0ceffff640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf8101de0 0x7f0cf80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0ceffff640 1 -- 172.21.15.64:0/3109945125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cf8106f10 msgr2=0x7f0cf8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0ceffff640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cf8106f10 0x7f0cf8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0ceffff640 1 -- 172.21.15.64:0/3109945125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a68d0 con 0x7f0cf8102790 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0ceffff640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf8074a30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005940 tx=0x7f0ce0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3109945125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce003f030 con 0x7f0cf8102790 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a6b60 con 0x7f0cf8102790 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3109945125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce0003aa0 con 0x7f0cf8102790 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3109945125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0035070 con 0x7f0cf8102790 2024-09-19T05:07:42.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.921+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81a7090 con 0x7f0cf8102790 2024-09-19T05:07:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.924+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3109945125 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0ce0042030 con 0x7f0cf8102790 2024-09-19T05:07:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.924+0000 7f0cff1e6740 1 Processor -- start 2024-09-19T05:07:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.924+0000 7f0cff1e6740 1 -- start start 2024-09-19T05:07:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x56266674f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x56266674fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x56266675c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5626666b52f0 con 0x562666750b40 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56266665ccf0 con 0x56266674f5d0 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5626666b12b0 con 0x56266675a370 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cef7fe640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0ce40a33b0 0x7f0ce40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x56266674f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x56266674f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57174/0 (socket says 172.21.15.64:57174) 2024-09-19T05:07:42.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 -- 172.21.15.64:0/1590663047 learned_addr learned my addr 172.21.15.64:0/1590663047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0ceffff640 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x56266675c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0ce40a33b0 0x7f0ce40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf659c640 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x56266674fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cef7fe640 1 -- 172.21.15.64:0/3109945125 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0ce00efb30 con 0x7f0cf8102790 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 -- 172.21.15.64:0/1590663047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 msgr2=0x56266675c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x56266675c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 -- 172.21.15.64:0/1590663047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 msgr2=0x56266674fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x56266674fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf6d9d640 1 -- 172.21.15.64:0/1590663047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626666413a0 con 0x562666750b40 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.925+0000 7f0cf659c640 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x56266674fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.926+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x56266674f090 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0004140 tx=0x7f0cf0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9392a094b090cb57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.926+0000 7f0cecff9640 1 -- 172.21.15.64:0/1590663047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0012e60 con 0x562666750b40 2024-09-19T05:07:42.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.926+0000 7f0cecff9640 1 -- 172.21.15.64:0/1590663047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf0023070 con 0x562666750b40 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.926+0000 7f0cecff9640 1 -- 172.21.15.64:0/1590663047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0003ad0 con 0x562666750b40 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.926+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0ce40a33b0 0x7f0ce40a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0022000 tx=0x7f0cf0020000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1590663047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 msgr2=0x56266674f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x56266674f090 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0004140 tx=0x7f0cf0003f90 comp rx=0 tx=0).stop 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1590663047 shutdown_connections 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x56266675c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x56266674fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1590663047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x56266674f090 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1590663047 >> 172.21.15.64:0/1590663047 conn(0x562666718310 msgr2=0x56266674bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1590663047 shutdown_connections 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1590663047 wait complete. 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 -- start start 2024-09-19T05:07:42.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.927+0000 7f0cff1e6740 1 Processor -- start 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x562666759c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x562666756f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x562666757480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56266675d070 con 0x562666750b40 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562666670060 con 0x56266674f5d0 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x562666756f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56266675d1f0 con 0x56266675a370 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x562666756f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57188/0 (socket says 172.21.15.64:57188) 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0ceffff640 1 -- 172.21.15.64:0/3432830677 learned_addr learned my addr 172.21.15.64:0/3432830677 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cff1e6740 10 client.0 _ll_register_callbacks cb 0x5626663c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:42.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cf659c640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x562666759c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.928+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x562666757480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0ceffff640 1 -- 172.21.15.64:0/3432830677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 msgr2=0x562666757480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0ceffff640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x562666757480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0ceffff640 1 -- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 msgr2=0x562666759c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0ceffff640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x562666759c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0ceffff640 1 -- 172.21.15.64:0/3432830677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626664263c0 con 0x562666750b40 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cf659c640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x562666759c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0ceffff640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x562666756f40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ed880 tx=0x7f0ce002faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d28f9a7c5a999a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cddffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cddffb640 1 -- 172.21.15.64:0/3432830677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0049040 con 0x562666750b40 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cddffb640 1 -- 172.21.15.64:0/3432830677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0ce0003c50 con 0x562666750b40 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf0023070 con 0x562666750b40 2024-09-19T05:07:42.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cddffb640 1 -- 172.21.15.64:0/3432830677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ce00ba5c0 con 0x562666750b40 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cff1e6740 10 client.15735 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.929+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562666768ac0 con 0x562666750b40 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cddffb640 1 -- 172.21.15.64:0/3432830677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce00bc510 con 0x562666750b40 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cddffb640 1 -- 172.21.15.64:0/3432830677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0ce00b9460 con 0x562666750b40 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cff1e6740 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cff1e6740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cff1e6740 20 client.15735 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562666769f20 0x56266676c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cff1e6740 4 client.15735 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0cf0023070 con 0x562666769f20 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.930+0000 7f0cf659c640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562666769f20 0x56266676c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.931+0000 7f0cf659c640 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562666769f20 0x56266676c330 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6399f87c94dcf7c server_cookie=6687e41dad04ba71 in_seq=0 out_seq=0 2024-09-19T05:07:42.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.931+0000 7f0cddffb640 10 client.15735 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:42.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.931+0000 7f0cddffb640 1 -- 172.21.15.64:0/3432830677 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0cf0023070 con 0x562666769f20 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.931+0000 7f0cddffb640 10 client.15735 handle_command_reply: tid=0 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.931+0000 7f0cff1e6740 1 client.15735 shutdown 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 msgr2=0x562666756f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x562666756f40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ed880 tx=0x7f0ce002faf0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 shutdown_connections 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266675a370 0x562666757480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562666769f20 0x56266676c330 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666750b40 0x562666756f40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/3432830677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56266674f5d0 0x562666759c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 >> 172.21.15.64:0/3432830677 conn(0x562666718310 msgr2=0x562666751930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cddffb640 0 client.15735 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:42.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 shutdown_connections 2024-09-19T05:07:42.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 1 -- 172.21.15.64:0/3432830677 wait complete. 2024-09-19T05:07:42.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.932+0000 7f0cff1e6740 20 client.15735 trim_cache size 0 max 16384 2024-09-19T05:07:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 Processor -- start 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 -- start start 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 0x5626667663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266674f5d0 0x56266674b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x562666752120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b2370 con 0x562666718310 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81afc40 con 0x562666750b40 2024-09-19T05:07:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ada90 con 0x56266674f5d0 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 0x5626667663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 0x5626667663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57192/0 (socket says 172.21.15.64:57192) 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 -- 172.21.15.64:0/385528766 learned_addr learned my addr 172.21.15.64:0/385528766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 -- 172.21.15.64:0/385528766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266674f5d0 msgr2=0x56266674b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266674f5d0 0x56266674b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 -- 172.21.15.64:0/385528766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 msgr2=0x562666752120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x562666752120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.949+0000 7f0cf659c640 1 -- 172.21.15.64:0/385528766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266674bed0 con 0x562666718310 2024-09-19T05:07:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.950+0000 7f0cf659c640 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 0x5626667663a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006fd0 tx=0x7f0ce800b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5948328fc0eac4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.950+0000 7f0cdffff640 1 -- 172.21.15.64:0/385528766 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8021070 con 0x562666718310 2024-09-19T05:07:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.950+0000 7f0cdffff640 1 -- 172.21.15.64:0/385528766 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce8015db0 con 0x562666718310 2024-09-19T05:07:42.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.950+0000 7f0cdffff640 1 -- 172.21.15.64:0/385528766 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8024d30 con 0x562666718310 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 -- 172.21.15.64:0/385528766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 msgr2=0x5626667663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 0x5626667663a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006fd0 tx=0x7f0ce800b7f0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 -- 172.21.15.64:0/385528766 shutdown_connections 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x562666752120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56266674f5d0 0x56266674b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/385528766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562666718310 0x5626667663a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 -- 172.21.15.64:0/385528766 >> 172.21.15.64:0/385528766 conn(0x5626667575e0 msgr2=0x562666756030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 -- 172.21.15.64:0/385528766 shutdown_connections 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 -- 172.21.15.64:0/385528766 wait complete. 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 -- start start 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.951+0000 7f0cff1e6740 1 Processor -- start 2024-09-19T05:07:42.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562666718310 0x562666762d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 0x56266675e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x56266675e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ae100 con 0x56266674f5d0 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81af570 con 0x562666750b40 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ae7d0 con 0x562666718310 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cff1e6740 10 client.0 _ll_register_callbacks cb 0x7f0ce0009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 0x56266675e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 0x56266675e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57208/0 (socket says 172.21.15.64:57208) 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 -- 172.21.15.64:0/1392505008 learned_addr learned my addr 172.21.15.64:0/1392505008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x56266675e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cf659c640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562666718310 0x562666762d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 -- 172.21.15.64:0/1392505008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562666718310 msgr2=0x562666762d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562666718310 0x562666762d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 -- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 msgr2=0x56266675e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x56266675e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 -- 172.21.15.64:0/1392505008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266675ee70 con 0x56266674f5d0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cf6d9d640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x56266675e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cf659c640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562666718310 0x562666762d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0ceffff640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 0x56266675e080 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ed330 tx=0x7f0ce00b80b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a736474b21491 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.952+0000 7f0cddffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cddffb640 1 -- 172.21.15.64:0/1392505008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0049040 con 0x56266674f5d0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cddffb640 1 -- 172.21.15.64:0/1392505008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0ce0003c50 con 0x56266674f5d0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce8015db0 con 0x56266674f5d0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cddffb640 1 -- 172.21.15.64:0/1392505008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ce00b9cf0 con 0x56266674f5d0 2024-09-19T05:07:42.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5626667d4cb0 con 0x56266674f5d0 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cddffb640 1 -- 172.21.15.64:0/1392505008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce00bab00 con 0x56266674f5d0 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cddffb640 1 -- 172.21.15.64:0/1392505008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0ce00c45f0 con 0x56266674f5d0 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 20 client.15741 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56266675a370 0x5626667d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 4 client.15741 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.953+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0ce8015db0 con 0x56266675a370 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cf659c640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56266675a370 0x5626667d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cddffb640 10 client.15741 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:42.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cf659c640 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56266675a370 0x5626667d72a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1645f0983a807fc server_cookie=56ceeaf27a0b613d in_seq=0 out_seq=0 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cddffb640 1 -- 172.21.15.64:0/1392505008 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0ce8015db0 con 0x56266675a370 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cddffb640 10 client.15741 handle_command_reply: tid=0 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cff1e6740 1 client.15741 shutdown 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 msgr2=0x56266675e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 0x56266675e080 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00ed330 tx=0x7f0ce00b80b0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 shutdown_connections 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562666750b40 0x56266675e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.954+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56266675a370 0x5626667d72a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56266674f5d0 0x56266675e080 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cddffb640 0 client.15741 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cff1e6740 1 --2- 172.21.15.64:0/1392505008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562666718310 0x562666762d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 >> 172.21.15.64:0/1392505008 conn(0x5626667575e0 msgr2=0x5626667587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 shutdown_connections 2024-09-19T05:07:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cff1e6740 1 -- 172.21.15.64:0/1392505008 wait complete. 2024-09-19T05:07:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cff1e6740 20 client.15741 trim_cache size 0 max 16384 2024-09-19T05:07:42.962 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:07:42.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0ce40a33b0 msgr2=0x7f0ce40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0ce40a33b0 0x7f0ce40a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0022000 tx=0x7f0cf0020000 comp rx=0 tx=0).stop 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 msgr2=0x7f0cf8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.955+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf8074a30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005940 tx=0x7f0ce0004270 comp rx=0 tx=0).stop 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 shutdown_connections 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0ce40a33b0 0x7f0ce40a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0cf8106f10 0x7f0cf8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cf8102790 0x7f0cf8074a30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 --2- 172.21.15.64:0/3109945125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf8101de0 0x7f0cf80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 >> 172.21.15.64:0/3109945125 conn(0x7f0cf80fd8e0 msgr2=0x7f0cf80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 shutdown_connections 2024-09-19T05:07:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:42.956+0000 7f0cfce0d640 1 -- 172.21.15.64:0/3109945125 wait complete. 2024-09-19T05:07:43.594 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:43 smithi064 ceph-mon[30960]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T05:07:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:43 smithi174 ceph-mon[34488]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T05:07:43.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:43 smithi103 ceph-mon[34446]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T05:07:44.624 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:07:44.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:07:44.958 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:45.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.479+0000 7fa259b92640 1 -- 172.21.15.64:0/937884583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 msgr2=0x7fa254106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.479+0000 7fa259b92640 1 --2- 172.21.15.64:0/937884583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa254106340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009ca0 tx=0x7fa24c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 -- 172.21.15.64:0/937884583 shutdown_connections 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 --2- 172.21.15.64:0/937884583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa254106340 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 --2- 172.21.15.64:0/937884583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541011e0 0x7fa254105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 --2- 172.21.15.64:0/937884583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2541008c0 0x7fa254100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 -- 172.21.15.64:0/937884583 >> 172.21.15.64:0/937884583 conn(0x7fa2540fc7d0 msgr2=0x7fa2540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 -- 172.21.15.64:0/937884583 shutdown_connections 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 -- 172.21.15.64:0/937884583 wait complete. 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.480+0000 7fa259b92640 1 Processor -- start 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 -- start start 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541008c0 0x7fa2541a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2541011e0 0x7fa2541a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa25419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa254118510 con 0x7fa254105ec0 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa254118390 con 0x7fa2541008c0 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa259b92640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa254118690 con 0x7fa2541011e0 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa253fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa25419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa253fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa25419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58284/0 (socket says 172.21.15.64:58284) 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa253fff640 1 -- 172.21.15.64:0/2117069805 learned_addr learned my addr 172.21.15.64:0/2117069805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.481+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541008c0 0x7fa2541a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa252ffd640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2541011e0 0x7fa2541a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa253fff640 1 -- 172.21.15.64:0/2117069805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2541011e0 msgr2=0x7fa2541a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa253fff640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2541011e0 0x7fa2541a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa253fff640 1 -- 172.21.15.64:0/2117069805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541008c0 msgr2=0x7fa2541a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa253fff640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541008c0 0x7fa2541a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa253fff640 1 -- 172.21.15.64:0/2117069805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25419efe0 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541008c0 0x7fa2541a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa253fff640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa25419e840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa24c031040 tx=0x7fa24c032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa250ff9640 1 -- 172.21.15.64:0/2117069805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c003b70 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa250ff9640 1 -- 172.21.15.64:0/2117069805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24c003d10 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.482+0000 7fa250ff9640 1 -- 172.21.15.64:0/2117069805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c030ab0 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.483+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25419f270 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.483+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541ab520 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 Processor -- start 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa250ff9640 1 -- 172.21.15.64:0/2117069805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa24c030c90 con 0x7fa254105ec0 2024-09-19T05:07:45.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 -- start start 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a35f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a40800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a43130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f9199b280 con 0x564f91a40d40 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f91942b10 con 0x564f91a36480 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.486+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f91997240 con 0x564f91a35b60 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a43130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a43130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58292/0 (socket says 172.21.15.64:58292) 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 -- 172.21.15.64:0/1712909177 learned_addr learned my addr 172.21.15.64:0/1712909177 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa2537fe640 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a40800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 -- 172.21.15.64:0/1712909177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 msgr2=0x564f91a35f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a35f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 -- 172.21.15.64:0/1712909177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 msgr2=0x564f91a40800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a40800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa252ffd640 1 -- 172.21.15.64:0/1712909177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f9194f1f0 con 0x564f91a40d40 2024-09-19T05:07:45.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa250ff9640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa23c0a3430 0x7fa23c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.487+0000 7fa250ff9640 1 -- 172.21.15.64:0/2117069805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa24c0f0fd0 con 0x7fa254105ec0 2024-09-19T05:07:45.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.488+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa23c0a3430 0x7fa23c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.488+0000 7fa252ffd640 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a43130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa248009c70 tx=0x7fa2480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b974baaa57460035 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.488+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa23c0a3430 0x7fa23c0a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x564f91a32f50 tx=0x7fa244005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.489+0000 7fa239ffb640 1 -- 172.21.15.64:0/1712909177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248004050 con 0x564f91a40d40 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.489+0000 7fa239ffb640 1 -- 172.21.15.64:0/1712909177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa248016070 con 0x564f91a40d40 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.489+0000 7fa239ffb640 1 -- 172.21.15.64:0/1712909177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24801b6d0 con 0x564f91a40d40 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.489+0000 7fa25bf6b740 1 -- 172.21.15.64:0/1712909177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 msgr2=0x564f91a43130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.489+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a43130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa248009c70 tx=0x7fa2480117b0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 -- 172.21.15.64:0/1712909177 shutdown_connections 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a43130 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a40800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/1712909177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a35f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 -- 172.21.15.64:0/1712909177 >> 172.21.15.64:0/1712909177 conn(0x564f91a320f0 msgr2=0x564f91a324d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 -- 172.21.15.64:0/1712909177 shutdown_connections 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 -- 172.21.15.64:0/1712909177 wait complete. 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 -- start start 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.490+0000 7fa25bf6b740 1 Processor -- start 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a35b60 0x564f91a3dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a3e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a40d40 0x564f91a3e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f91a43910 con 0x564f91a36480 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f91955f10 con 0x564f91a35b60 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f91a43a90 con 0x564f91a40d40 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a3e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a3e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58300/0 (socket says 172.21.15.64:58300) 2024-09-19T05:07:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 -- 172.21.15.64:0/95095443 learned_addr learned my addr 172.21.15.64:0/95095443 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa25bf6b740 10 client.0 _ll_register_callbacks cb 0x564f9191ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa2537fe640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a35b60 0x564f91a3dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa253fff640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a40d40 0x564f91a3e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 -- 172.21.15.64:0/95095443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a40d40 msgr2=0x564f91a3e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a40d40 0x564f91a3e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 -- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a35b60 msgr2=0x564f91a3dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a35b60 0x564f91a3dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.491+0000 7fa252ffd640 1 -- 172.21.15.64:0/95095443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9170c6a0 con 0x564f91a36480 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa252ffd640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a3e1d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa248001eb0 tx=0x7fa248009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dce89cdae5068b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa232ffd640 1 -- 172.21.15.64:0/95095443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248020070 con 0x564f91a36480 2024-09-19T05:07:45.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa232ffd640 1 -- 172.21.15.64:0/95095443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa2480041f0 con 0x564f91a36480 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa248016070 con 0x564f91a36480 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa232ffd640 1 -- 172.21.15.64:0/95095443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2480097d0 con 0x564f91a36480 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa25bf6b740 10 client.15753 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.492+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f91a4f5f0 con 0x564f91a36480 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa232ffd640 1 -- 172.21.15.64:0/95095443 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa248016070 con 0x564f91a36480 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa232ffd640 1 -- 172.21.15.64:0/95095443 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2480124e0 con 0x564f91a36480 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa25bf6b740 10 client.15753 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa25bf6b740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa25bf6b740 20 client.15753 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91a509a0 0x564f91a52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa25bf6b740 4 client.15753 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:45.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f919a87f0 con 0x564f91a509a0 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.493+0000 7fa2537fe640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91a509a0 0x564f91a52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.494+0000 7fa2537fe640 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91a509a0 0x564f91a52d90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f6d35a1183e510 server_cookie=41a0b08332c758c8 in_seq=0 out_seq=0 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.494+0000 7fa232ffd640 10 client.15753 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.494+0000 7fa232ffd640 1 -- 172.21.15.64:0/95095443 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f919a87f0 con 0x564f91a509a0 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.494+0000 7fa232ffd640 10 client.15753 handle_command_reply: tid=0 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 client.15753 shutdown 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 msgr2=0x564f91a3e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a3e1d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa248001eb0 tx=0x7fa248009f90 comp rx=0 tx=0).stop 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 shutdown_connections 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a40d40 0x564f91a3e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91a509a0 0x564f91a52d90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a3e1d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/95095443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a35b60 0x564f91a3dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 >> 172.21.15.64:0/95095443 conn(0x564f91a320f0 msgr2=0x564f91a31270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:45.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa232ffd640 0 client.15753 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:45.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 shutdown_connections 2024-09-19T05:07:45.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 1 -- 172.21.15.64:0/95095443 wait complete. 2024-09-19T05:07:45.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.495+0000 7fa25bf6b740 20 client.15753 trim_cache size 0 max 16384 2024-09-19T05:07:45.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 Processor -- start 2024-09-19T05:07:45.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 -- start start 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a55db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b6640 con 0x564f91a40d40 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b3f10 con 0x564f91a36480 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b1d60 con 0x564f91a35b60 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa253fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa253fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58314/0 (socket says 172.21.15.64:58314) 2024-09-19T05:07:45.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa253fff640 1 -- 172.21.15.64:0/721734420 learned_addr learned my addr 172.21.15.64:0/721734420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:45.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.517+0000 7fa252ffd640 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a55db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa2537fe640 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa253fff640 1 -- 172.21.15.64:0/721734420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 msgr2=0x564f91a3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa253fff640 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a3faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa253fff640 1 -- 172.21.15.64:0/721734420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 msgr2=0x564f91a55db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa253fff640 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a55db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa253fff640 1 -- 172.21.15.64:0/721734420 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f91a3c220 con 0x564f91a40d40 2024-09-19T05:07:45.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa253fff640 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a562f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0b98b0 tx=0x7fa24c0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fc82edaffe608d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa2397fa640 1 -- 172.21.15.64:0/721734420 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0d5020 con 0x564f91a40d40 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa2397fa640 1 -- 172.21.15.64:0/721734420 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24c036050 con 0x564f91a40d40 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.518+0000 7fa2397fa640 1 -- 172.21.15.64:0/721734420 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0d6220 con 0x564f91a40d40 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 -- 172.21.15.64:0/721734420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 msgr2=0x564f91a562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a562f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0b98b0 tx=0x7fa24c0dd040 comp rx=0 tx=0).stop 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 -- 172.21.15.64:0/721734420 shutdown_connections 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a40d40 0x564f91a562f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a36480 0x564f91a55db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/721734420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 -- 172.21.15.64:0/721734420 >> 172.21.15.64:0/721734420 conn(0x564f91a320f0 msgr2=0x564f91a3f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 -- 172.21.15.64:0/721734420 shutdown_connections 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.519+0000 7fa25bf6b740 1 -- 172.21.15.64:0/721734420 wait complete. 2024-09-19T05:07:45.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 -- start start 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 Processor -- start 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a54250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a40d40 0x564f91aba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b23d0 con 0x564f91a36480 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b3840 con 0x564f91a40d40 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa252ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa25bf6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2541b2aa0 con 0x564f91a35b60 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa252ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a54790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58320/0 (socket says 172.21.15.64:58320) 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.520+0000 7fa252ffd640 1 -- 172.21.15.64:0/2973987662 learned_addr learned my addr 172.21.15.64:0/2973987662 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa253fff640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a40d40 0x564f91aba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a54250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa25bf6b740 10 client.0 _ll_register_callbacks cb 0x7fa244009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa252ffd640 1 -- 172.21.15.64:0/2973987662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 msgr2=0x564f91a54250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa252ffd640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a54250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa252ffd640 1 -- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a40d40 msgr2=0x564f91aba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:45.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa252ffd640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a40d40 0x564f91aba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa252ffd640 1 -- 172.21.15.64:0/2973987662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f91aba820 con 0x564f91a36480 2024-09-19T05:07:45.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa253fff640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a40d40 0x564f91aba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:45.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa252ffd640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a54790 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa2480117f0 tx=0x7fa248009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03e7f5ccd40de64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.521+0000 7fa232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa232ffd640 1 -- 172.21.15.64:0/2973987662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248020070 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa232ffd640 1 -- 172.21.15.64:0/2973987662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa248003b00 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24c036050 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa232ffd640 1 -- 172.21.15.64:0/2973987662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa248012d20 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa25bf6b740 10 client.15759 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f91abbaa0 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa232ffd640 1 -- 172.21.15.64:0/2973987662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24c036050 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa232ffd640 1 -- 172.21.15.64:0/2973987662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa248004540 con 0x564f91a36480 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa25bf6b740 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa25bf6b740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.522+0000 7fa25bf6b740 20 client.15759 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.523+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91abc140 0x564f91abe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.523+0000 7fa25bf6b740 4 client.15759 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.523+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f919a2d00 con 0x564f91abc140 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.523+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91abc140 0x564f91abe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.523+0000 7fa2537fe640 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91abc140 0x564f91abe560 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f2fe5c0e690306 server_cookie=5878ad79d3a8f49d in_seq=0 out_seq=0 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.523+0000 7fa232ffd640 10 client.15759 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.524+0000 7fa232ffd640 1 -- 172.21.15.64:0/2973987662 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f919a2d00 con 0x564f91abc140 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.524+0000 7fa232ffd640 10 client.15759 handle_command_reply: tid=0 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.524+0000 7fa25bf6b740 1 client.15759 shutdown 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 msgr2=0x564f91a54790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a54790 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa2480117f0 tx=0x7fa248009420 comp rx=0 tx=0).stop 2024-09-19T05:07:45.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 shutdown_connections 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f91a40d40 0x564f91aba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564f91abc140 0x564f91abe560 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f91a36480 0x564f91a54790 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 --2- 172.21.15.64:0/2973987662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f91a35b60 0x564f91a54250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa232ffd640 0 client.15759 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 >> 172.21.15.64:0/2973987662 conn(0x564f91a320f0 msgr2=0x564f91a37dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 shutdown_connections 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 1 -- 172.21.15.64:0/2973987662 wait complete. 2024-09-19T05:07:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.525+0000 7fa25bf6b740 20 client.15759 trim_cache size 0 max 16384 2024-09-19T05:07:45.531 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:07:45.531 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:07:45.532 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "6080efa2-f5a2-44e4-8455-2bbe76ce06ab", 2024-09-19T05:07:45.532 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:07:45.532 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:07:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa23c0a3430 msgr2=0x7fa23c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa23c0a3430 0x7fa23c0a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x564f91a32f50 tx=0x7fa244005f90 comp rx=0 tx=0).stop 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 msgr2=0x7fa25419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa25419e840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa24c031040 tx=0x7fa24c032f70 comp rx=0 tx=0).stop 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 shutdown_connections 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa23c0a3430 0x7fa23c0a58f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa254105ec0 0x7fa25419e840 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2541011e0 0x7fa2541a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 --2- 172.21.15.64:0/2117069805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2541008c0 0x7fa2541a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 >> 172.21.15.64:0/2117069805 conn(0x7fa2540fc7d0 msgr2=0x7fa254103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:45.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 shutdown_connections 2024-09-19T05:07:45.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:45.526+0000 7fa259b92640 1 -- 172.21.15.64:0/2117069805 wait complete. 2024-09-19T05:07:45.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:45 smithi064 ceph-mon[30960]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-19T05:07:45.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:45 smithi174 ceph-mon[34488]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-19T05:07:45.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:45 smithi103 ceph-mon[34446]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-19T05:07:46.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:07:46.524 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1813939226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102570 msgr2=0x7f7ab01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1813939226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102570 0x7f7ab01029b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009900 tx=0x7f7a9c02a480 comp rx=0 tx=0).stop 2024-09-19T05:07:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1813939226 shutdown_connections 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1813939226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0102ef0 0x7f7ab010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1813939226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102570 0x7f7ab01029b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1813939226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0100e20 0x7f7ab0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1813939226 >> 172.21.15.64:0/1813939226 conn(0x7f7ab00fc7d0 msgr2=0x7f7ab00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1813939226 shutdown_connections 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.042+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1813939226 wait complete. 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 Processor -- start 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 -- start start 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0100e20 0x7f7ab019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0102570 0x7f7ab019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 0x7f7ab019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0110e00 con 0x7f7ab0102ef0 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0110c80 con 0x7f7ab0100e20 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.043+0000 7f7ab6c96640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0110f80 con 0x7f7ab0102570 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 0x7f7ab019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 0x7f7ab019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58342/0 (socket says 172.21.15.64:58342) 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 -- 172.21.15.64:0/1129388676 learned_addr learned my addr 172.21.15.64:0/1129388676 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0100e20 0x7f7ab019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7aaffff640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0102570 0x7f7ab019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 -- 172.21.15.64:0/1129388676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0102570 msgr2=0x7f7ab019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0102570 0x7f7ab019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 -- 172.21.15.64:0/1129388676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0100e20 msgr2=0x7f7ab019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0100e20 0x7f7ab019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 -- 172.21.15.64:0/1129388676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab019f650 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab5233640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 0x7f7ab019ef70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009fd0 tx=0x7f7aa800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7aadffb640 1 -- 172.21.15.64:0/1129388676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8017070 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab019f940 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.044+0000 7f7aadffb640 1 -- 172.21.15.64:0/1129388676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa8003e20 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.045+0000 7f7aadffb640 1 -- 172.21.15.64:0/1129388676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8010460 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.045+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab01a7220 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.047+0000 7f7aadffb640 1 -- 172.21.15.64:0/1129388676 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7aa8003970 con 0x7f7ab0102ef0 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 Processor -- start 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 -- start start 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7aadffb640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7aa00a33b0 0x7f7aa00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637563204a0 0x563756320880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x56375632b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 0x56375632da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5637562862e0 con 0x56375632b680 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56375622dce0 con 0x5637563204a0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab906f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637562822a0 con 0x563756320dc0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7aaffff640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7aa00a33b0 0x7f7aa00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7aadffb640 1 -- 172.21.15.64:0/1129388676 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7aa80cfa70 con 0x7f7ab0102ef0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab5233640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 0x56375632da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab5233640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 0x56375632da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58348/0 (socket says 172.21.15.64:58348) 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab5233640 1 -- 172.21.15.64:0/4239284233 learned_addr learned my addr 172.21.15.64:0/4239284233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637563204a0 0x563756320880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7aaffff640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x56375632b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab5233640 1 -- 172.21.15.64:0/4239284233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 msgr2=0x56375632b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.048+0000 7f7ab5233640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x56375632b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7ab5233640 1 -- 172.21.15.64:0/4239284233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637563204a0 msgr2=0x563756320880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7ab5233640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637563204a0 0x563756320880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7ab5233640 1 -- 172.21.15.64:0/4239284233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563756212440 con 0x56375632b680 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7aaffff640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x56375632b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637563204a0 0x563756320880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7ab5233640 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 0x56375632da70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80042f0 tx=0x7f7aa809bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e291733bec2becf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7aaffff640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7aa00a33b0 0x7f7aa00a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009230 tx=0x7f7a9c031cd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7a9affd640 1 -- 172.21.15.64:0/4239284233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8020040 con 0x56375632b680 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7a9affd640 1 -- 172.21.15.64:0/4239284233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa80a00a0 con 0x56375632b680 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.049+0000 7f7a9affd640 1 -- 172.21.15.64:0/4239284233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8013050 con 0x56375632b680 2024-09-19T05:07:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 -- 172.21.15.64:0/4239284233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 msgr2=0x56375632da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 0x56375632da70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80042f0 tx=0x7f7aa809bc80 comp rx=0 tx=0).stop 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 -- 172.21.15.64:0/4239284233 shutdown_connections 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56375632b680 0x56375632da70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x56375632b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 --2- 172.21.15.64:0/4239284233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637563204a0 0x563756320880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 -- 172.21.15.64:0/4239284233 >> 172.21.15.64:0/4239284233 conn(0x56375631ca30 msgr2=0x56375631ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 -- 172.21.15.64:0/4239284233 shutdown_connections 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 -- 172.21.15.64:0/4239284233 wait complete. 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.050+0000 7f7ab906f740 1 -- start start 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 Processor -- start 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x56375632b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756328880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56375632e250 con 0x5637563204a0 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563756241050 con 0x563756320dc0 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56375632e3d0 con 0x56375632b680 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab4a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x56375632b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab906f740 10 client.0 _ll_register_callbacks cb 0x563756281160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab4a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x56375632b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58354/0 (socket says 172.21.15.64:58354) 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab4a32640 1 -- 172.21.15.64:0/58346810 learned_addr learned my addr 172.21.15.64:0/58346810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7ab5233640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756328880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.051+0000 7f7aaffff640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab4a32640 1 -- 172.21.15.64:0/58346810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 msgr2=0x563756328880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756328880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab4a32640 1 -- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 msgr2=0x5637563282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab4a32640 1 -- 172.21.15.64:0/58346810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563755ff73c0 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x56375632b030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400bfb0 tx=0x7f7aa40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e26e28a7d65e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7a8ffff640 1 -- 172.21.15.64:0/58346810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004220 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7a8ffff640 1 -- 172.21.15.64:0/58346810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7aa40043c0 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa80a00a0 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7a8ffff640 1 -- 172.21.15.64:0/58346810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7aa4005020 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab906f740 10 client.15771 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.052+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563756339c80 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7a8ffff640 1 -- 172.21.15.64:0/58346810 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa80a00a0 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7a8ffff640 1 -- 172.21.15.64:0/58346810 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7aa40107c0 con 0x5637563204a0 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7ab906f740 10 client.15771 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7ab906f740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7ab906f740 20 client.15771 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7ab906f740 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56375633b090 0x56375633d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7ab906f740 4 client.15771 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563756293850 con 0x56375633b090 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.053+0000 7f7aaffff640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56375633b090 0x56375633d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.054+0000 7f7a8ffff640 10 client.15771 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.054+0000 7f7aaffff640 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56375633b090 0x56375633d4f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1513c87eb75148fc server_cookie=d957d432c4fcdf2c in_seq=0 out_seq=0 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.054+0000 7f7a8ffff640 1 -- 172.21.15.64:0/58346810 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563756293850 con 0x56375633b090 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.054+0000 7f7a8ffff640 10 client.15771 handle_command_reply: tid=0 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.054+0000 7f7ab906f740 1 client.15771 shutdown 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 msgr2=0x56375632b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x56375632b030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400bfb0 tx=0x7f7aa40128e0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 shutdown_connections 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756328880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56375633b090 0x56375633d4f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 --2- 172.21.15.64:0/58346810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x56375632b030 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 >> 172.21.15.64:0/58346810 conn(0x56375631ca30 msgr2=0x563756322710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7a8ffff640 0 client.15771 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 shutdown_connections 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 1 -- 172.21.15.64:0/58346810 wait complete. 2024-09-19T05:07:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.055+0000 7f7ab906f740 20 client.15771 trim_cache size 0 max 16384 2024-09-19T05:07:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 Processor -- start 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 -- start start 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x563756326bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56375632b680 0x563756342e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01b2300 con 0x5637563204a0 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01afbd0 con 0x56375632b680 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab906f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01ada20 con 0x563756320dc0 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab4a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab4a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58368/0 (socket says 172.21.15.64:58368) 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2668775627 learned_addr learned my addr 172.21.15.64:0/2668775627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7aaffff640 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x563756326bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.077+0000 7f7ab5233640 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56375632b680 0x563756342e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2668775627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 msgr2=0x563756326bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x563756326bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2668775627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56375632b680 msgr2=0x563756342e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56375632b680 0x563756342e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2668775627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563756327290 con 0x5637563204a0 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab5233640 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56375632b680 0x563756342e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563374b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b890 tx=0x7f7aa40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356be536a77bdf59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7a9a7fc640 1 -- 172.21.15.64:0/2668775627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40109c0 con 0x5637563204a0 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7a9a7fc640 1 -- 172.21.15.64:0/2668775627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa4010e50 con 0x5637563204a0 2024-09-19T05:07:47.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.078+0000 7f7a9a7fc640 1 -- 172.21.15.64:0/2668775627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4024470 con 0x5637563204a0 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 -- 172.21.15.64:0/2668775627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 msgr2=0x5637563374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563374b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b890 tx=0x7f7aa40035f0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 -- 172.21.15.64:0/2668775627 shutdown_connections 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56375632b680 0x563756342e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563756320dc0 0x563756326bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2668775627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563374b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 -- 172.21.15.64:0/2668775627 >> 172.21.15.64:0/2668775627 conn(0x56375631ca30 msgr2=0x5637563238b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 -- 172.21.15.64:0/2668775627 shutdown_connections 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 -- 172.21.15.64:0/2668775627 wait complete. 2024-09-19T05:07:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 -- start start 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.079+0000 7f7ab906f740 1 Processor -- start 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756325d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01ae090 con 0x5637563204a0 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01af500 con 0x563756320dc0 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01ae760 con 0x56375632b680 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab906f740 10 client.0 _ll_register_callbacks cb 0x7f7aa400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563252b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58382/0 (socket says 172.21.15.64:58382) 2024-09-19T05:07:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2087257229 learned_addr learned my addr 172.21.15.64:0/2087257229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab5233640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756325d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7aaffff640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2087257229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 msgr2=0x563756325d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756325d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 msgr2=0x5637563257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.080+0000 7f7ab4a32640 1 -- 172.21.15.64:0/2087257229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375632b3a0 con 0x5637563204a0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7ab4a32640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563252b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005d00 tx=0x7f7aa400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70592ce01bf4698 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7a8ffff640 1 -- 172.21.15.64:0/2087257229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004d50 con 0x5637563204a0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa4010e50 con 0x5637563204a0 2024-09-19T05:07:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7a8ffff640 1 -- 172.21.15.64:0/2087257229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7aa40051c0 con 0x5637563204a0 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7a8ffff640 1 -- 172.21.15.64:0/2087257229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7aa4042050 con 0x5637563204a0 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7ab906f740 10 client.15777 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637563a5ec0 con 0x5637563204a0 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.081+0000 7f7a8ffff640 1 -- 172.21.15.64:0/2087257229 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7aa4010e50 con 0x5637563204a0 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7a8ffff640 1 -- 172.21.15.64:0/2087257229 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7aa4024b80 con 0x5637563204a0 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7ab906f740 10 client.15777 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7ab906f740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7ab906f740 20 client.15777 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637563a6570 0x5637563a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7ab906f740 4 client.15777 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56375628dd60 con 0x5637563a6570 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.082+0000 7f7aaffff640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637563a6570 0x5637563a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7aaffff640 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637563a6570 0x5637563a8990 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a54b7e8769a85f6 server_cookie=164a43b2721c419d in_seq=0 out_seq=0 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7a8ffff640 10 client.15777 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7a8ffff640 1 -- 172.21.15.64:0/2087257229 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56375628dd60 con 0x5637563a6570 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7a8ffff640 10 client.15777 handle_command_reply: tid=0 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 client.15777 shutdown 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 msgr2=0x5637563252b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563252b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005d00 tx=0x7f7aa400bab0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 shutdown_connections 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56375632b680 0x563756325d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563756320dc0 0x5637563257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637563a6570 0x5637563a8990 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 --2- 172.21.15.64:0/2087257229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637563204a0 0x5637563252b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 >> 172.21.15.64:0/2087257229 conn(0x56375631ca30 msgr2=0x563756322710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7a8ffff640 0 client.15777 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 shutdown_connections 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 1 -- 172.21.15.64:0/2087257229 wait complete. 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.083+0000 7f7ab906f740 20 client.15777 trim_cache size 0 max 16384 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout: "6080efa2-f5a2-44e4-8455-2bbe76ce06ab": { 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "6080efa2-f5a2-44e4-8455-2bbe76ce06ab", 2024-09-19T05:07:47.091 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7aa00a33b0 msgr2=0x7f7aa00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7aa00a33b0 0x7f7aa00a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009230 tx=0x7f7a9c031cd0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 msgr2=0x7f7ab019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 0x7f7ab019ef70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009fd0 tx=0x7f7aa800b260 comp rx=0 tx=0).stop 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 shutdown_connections 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7aa00a33b0 0x7f7aa00a5870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0102ef0 0x7f7ab019ef70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0102570 0x7f7ab019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 --2- 172.21.15.64:0/1129388676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0100e20 0x7f7ab019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 >> 172.21.15.64:0/1129388676 conn(0x7f7ab00fc7d0 msgr2=0x7f7ab006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 shutdown_connections 2024-09-19T05:07:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:07:47.084+0000 7f7ab6c96640 1 -- 172.21.15.64:0/1129388676 wait complete. 2024-09-19T05:07:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:47 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:07:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:47 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:47 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:07:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:47 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:47.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:47 smithi174 ceph-mon[34488]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-19T05:07:47.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:47 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:07:47.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:47 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:47.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:47 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:07:47.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:47 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:47.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:47 smithi064 ceph-mon[30960]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-19T05:07:47.866 INFO:tasks.cephfs.filesystem:scrub status for tag:6080efa2-f5a2-44e4-8455-2bbe76ce06ab - {'path': '/', 'tag': '6080efa2-f5a2-44e4-8455-2bbe76ce06ab', 'options': 'recursive,force'} 2024-09-19T05:07:47.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:47 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:07:47.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:47 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:47.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:47 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:07:47.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:47 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:47.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:47 smithi103 ceph-mon[34446]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-19T05:07:49.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:49 smithi174 ceph-mon[34488]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 937 B/s wr, 6 op/s 2024-09-19T05:07:49.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:49 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:07:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:49 smithi064 ceph-mon[30960]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 937 B/s wr, 6 op/s 2024-09-19T05:07:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:49 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:07:49.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:49 smithi103 ceph-mon[34446]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 937 B/s wr, 6 op/s 2024-09-19T05:07:49.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:49 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:07:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 -- 172.21.15.64:0/2797687150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe93410c0c0 msgr2=0x7fe93410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 --2- 172.21.15.64:0/2797687150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe93410c0c0 0x7fe93410e560 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009ca0 tx=0x7fe92c02a950 comp rx=0 tx=0).stop 2024-09-19T05:07:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 -- 172.21.15.64:0/2797687150 shutdown_connections 2024-09-19T05:07:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 --2- 172.21.15.64:0/2797687150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe93410c0c0 0x7fe93410e560 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 --2- 172.21.15.64:0/2797687150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934103610 0x7fe93410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 --2- 172.21.15.64:0/2797687150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe934102cf0 0x7fe9341030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 -- 172.21.15.64:0/2797687150 >> 172.21.15.64:0/2797687150 conn(0x7fe9340fc6e0 msgr2=0x7fe9340feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 -- 172.21.15.64:0/2797687150 shutdown_connections 2024-09-19T05:07:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.081+0000 7fe93a82e640 1 -- 172.21.15.64:0/2797687150 wait complete. 2024-09-19T05:07:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 Processor -- start 2024-09-19T05:07:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 -- start start 2024-09-19T05:07:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934102cf0 0x7fe934100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 0x7fe934100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93410c0c0 0x7fe9341013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111d80 con 0x7fe934103610 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111c00 con 0x7fe934102cf0 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe93a82e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111f00 con 0x7fe93410c0c0 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe9337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 0x7fe934100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe9337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 0x7fe934100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58402/0 (socket says 172.21.15.64:58402) 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe9337fe640 1 -- 172.21.15.64:0/3115404720 learned_addr learned my addr 172.21.15.64:0/3115404720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.082+0000 7fe933fff640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934102cf0 0x7fe934100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9337fe640 1 -- 172.21.15.64:0/3115404720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93410c0c0 msgr2=0x7fe9341013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe938dcb640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93410c0c0 0x7fe9341013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9337fe640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93410c0c0 0x7fe9341013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9337fe640 1 -- 172.21.15.64:0/3115404720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934102cf0 msgr2=0x7fe934100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9337fe640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934102cf0 0x7fe934100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9337fe640 1 -- 172.21.15.64:0/3115404720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9341a66f0 con 0x7fe934103610 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe938dcb640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93410c0c0 0x7fe9341013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe933fff640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934102cf0 0x7fe934100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9337fe640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 0x7fe934100e80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe92800e9f0 tx=0x7fe92800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9317fa640 1 -- 172.21.15.64:0/3115404720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92800cd80 con 0x7fe934103610 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9341a69e0 con 0x7fe934103610 2024-09-19T05:07:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9317fa640 1 -- 172.21.15.64:0/3115404720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9280108e0 con 0x7fe934103610 2024-09-19T05:07:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe9317fa640 1 -- 172.21.15.64:0/3115404720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928018920 con 0x7fe934103610 2024-09-19T05:07:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.083+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a6f80 con 0x7fe934103610 2024-09-19T05:07:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 Processor -- start 2024-09-19T05:07:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 -- start start 2024-09-19T05:07:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae4af0 0x561c89ae4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 0x561c89aef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89af20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561c89a492f0 con 0x561c89ae5410 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c899f1270 con 0x561c89ae4af0 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c89a452b0 con 0x561c89aefcb0 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 0x561c89aef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe9317fa640 1 -- 172.21.15.64:0/3115404720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe928018ac0 con 0x7fe934103610 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 0x561c89aef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58404/0 (socket says 172.21.15.64:58404) 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe933fff640 1 -- 172.21.15.64:0/582812820 learned_addr learned my addr 172.21.15.64:0/582812820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.086+0000 7fe938dcb640 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae4af0 0x561c89ae4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe933fff640 1 -- 172.21.15.64:0/582812820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 msgr2=0x561c89af20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe933fff640 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89af20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe933fff640 1 -- 172.21.15.64:0/582812820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae4af0 msgr2=0x561c89ae4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe933fff640 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae4af0 0x561c89ae4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe933fff640 1 -- 172.21.15.64:0/582812820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c899f0340 con 0x561c89ae5410 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe933fff640 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 0x561c89aef770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe924010480 tx=0x7fe924010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503a4c93860af607 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe91a7fc640 1 -- 172.21.15.64:0/582812820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924011760 con 0x561c89ae5410 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe91a7fc640 1 -- 172.21.15.64:0/582812820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe924011db0 con 0x561c89ae5410 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe91a7fc640 1 -- 172.21.15.64:0/582812820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92400e430 con 0x561c89ae5410 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe9317fa640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe91c0a3460 0x7fe91c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe938dcb640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe91c0a3460 0x7fe91c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.087+0000 7fe9317fa640 1 -- 172.21.15.64:0/3115404720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe9280d0050 con 0x7fe934103610 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe938dcb640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe91c0a3460 0x7fe91c0a5920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009a60 tx=0x7fe92c03c040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 -- 172.21.15.64:0/582812820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 msgr2=0x561c89aef770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 0x561c89aef770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe924010480 tx=0x7fe924010940 comp rx=0 tx=0).stop 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 -- 172.21.15.64:0/582812820 shutdown_connections 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89af20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae5410 0x561c89aef770 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/582812820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae4af0 0x561c89ae4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 -- 172.21.15.64:0/582812820 >> 172.21.15.64:0/582812820 conn(0x561c89ae1060 msgr2=0x561c89ae1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 -- 172.21.15.64:0/582812820 shutdown_connections 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 -- 172.21.15.64:0/582812820 wait complete. 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.088+0000 7fe93d5ca740 1 -- start start 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 Processor -- start 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561c89af2880 con 0x561c89ae4af0 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c899d2290 con 0x561c89ae5410 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c89af2a00 con 0x561c89aefcb0 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe93d5ca740 10 client.0 _ll_register_callbacks cb 0x561c89a03140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aecbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58408/0 (socket says 172.21.15.64:58408) 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe933fff640 1 -- 172.21.15.64:0/2842919095 learned_addr learned my addr 172.21.15.64:0/2842919095 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.089+0000 7fe9337fe640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe938dcb640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe933fff640 1 -- 172.21.15.64:0/2842919095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 msgr2=0x561c89aed660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe933fff640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe933fff640 1 -- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 msgr2=0x561c89aed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe933fff640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe933fff640 1 -- 172.21.15.64:0/2842919095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8975c210 con 0x561c89ae4af0 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe9337fe640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe933fff640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aecbe0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe92401df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bb77d94a854b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe9137fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe9137fe640 1 -- 172.21.15.64:0/2842919095 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924011760 con 0x561c89ae4af0 2024-09-19T05:07:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe9137fe640 1 -- 172.21.15.64:0/2842919095 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe924011900 con 0x561c89ae4af0 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe924011db0 con 0x561c89ae4af0 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe9137fe640 1 -- 172.21.15.64:0/2842919095 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe924034050 con 0x561c89ae4af0 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe93d5ca740 10 client.15789 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.090+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c89828620 con 0x561c89ae4af0 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe9137fe640 1 -- 172.21.15.64:0/2842919095 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe924011db0 con 0x561c89ae4af0 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe9137fe640 1 -- 172.21.15.64:0/2842919095 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe92402f030 con 0x561c89ae4af0 2024-09-19T05:07:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe93d5ca740 10 client.15789 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe93d5ca740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe93d5ca740 20 client.15789 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89affca0 0x561c89b02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe93d5ca740 4 client.15789 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c89a56a20 con 0x561c89affca0 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe9337fe640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89affca0 0x561c89b02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe9337fe640 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89affca0 0x561c89b02090 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9682521431ae9d5b server_cookie=a03c55c2b1560346 in_seq=0 out_seq=0 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.091+0000 7fe9137fe640 10 client.15789 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe9137fe640 1 -- 172.21.15.64:0/2842919095 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c89a56a20 con 0x561c89affca0 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe9137fe640 10 client.15789 handle_command_reply: tid=0 2024-09-19T05:07:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 client.15789 shutdown 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 msgr2=0x561c89aecbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aecbe0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe92401df20 comp rx=0 tx=0).stop 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 shutdown_connections 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89affca0 0x561c89b02090 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2842919095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aecbe0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.092+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 >> 172.21.15.64:0/2842919095 conn(0x561c89ae1060 msgr2=0x561c89ae6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.093+0000 7fe9137fe640 0 client.15789 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.093+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 shutdown_connections 2024-09-19T05:07:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.093+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2842919095 wait complete. 2024-09-19T05:07:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.093+0000 7fe93d5ca740 20 client.15789 trim_cache size 0 max 16384 2024-09-19T05:07:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 Processor -- start 2024-09-19T05:07:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 -- start start 2024-09-19T05:07:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aeeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341b2270 con 0x561c89ae4af0 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341afb40 con 0x561c89ae5410 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.109+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ad990 con 0x561c89aefcb0 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aeeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aeeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58422/0 (socket says 172.21.15.64:58422) 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 -- 172.21.15.64:0/1221949672 learned_addr learned my addr 172.21.15.64:0/1221949672 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe9337fe640 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe938dcb640 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 -- 172.21.15.64:0/1221949672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 msgr2=0x561c89aed650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 -- 172.21.15.64:0/1221949672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 msgr2=0x561c89aed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 -- 172.21.15.64:0/1221949672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c89adc2d0 con 0x561c89ae4af0 2024-09-19T05:07:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe9337fe640 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe933fff640 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aeeb50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe924010980 tx=0x7fe92400bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51926f658387fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe919ffb640 1 -- 172.21.15.64:0/1221949672 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924027400 con 0x561c89ae4af0 2024-09-19T05:07:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe919ffb640 1 -- 172.21.15.64:0/1221949672 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92401cdb0 con 0x561c89ae4af0 2024-09-19T05:07:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.110+0000 7fe919ffb640 1 -- 172.21.15.64:0/1221949672 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240118d0 con 0x561c89ae4af0 2024-09-19T05:07:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 -- 172.21.15.64:0/1221949672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 msgr2=0x561c89aeeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aeeb50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe924010980 tx=0x7fe92400bae0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 -- 172.21.15.64:0/1221949672 shutdown_connections 2024-09-19T05:07:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89aefcb0 0x561c89aed650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89aed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/1221949672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89ae4af0 0x561c89aeeb50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 -- 172.21.15.64:0/1221949672 >> 172.21.15.64:0/1221949672 conn(0x561c89ae1060 msgr2=0x561c89aee440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 -- 172.21.15.64:0/1221949672 shutdown_connections 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 -- 172.21.15.64:0/1221949672 wait complete. 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.111+0000 7fe93d5ca740 1 -- start start 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 Processor -- start 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89ae4af0 0x561c89af7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89af81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 0x561c89ae8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ae000 con 0x561c89aefcb0 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341af470 con 0x561c89ae5410 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ae6d0 con 0x561c89ae4af0 2024-09-19T05:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe93d5ca740 10 client.0 _ll_register_callbacks cb 0x7fe928005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 0x561c89ae8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 0x561c89ae8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58434/0 (socket says 172.21.15.64:58434) 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 -- 172.21.15.64:0/2231666436 learned_addr learned my addr 172.21.15.64:0/2231666436 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe933fff640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89ae4af0 0x561c89af7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe9337fe640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89af81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 -- 172.21.15.64:0/2231666436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89ae4af0 msgr2=0x561c89af7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89ae4af0 0x561c89af7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 -- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 msgr2=0x561c89af81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89af81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe938dcb640 1 -- 172.21.15.64:0/2231666436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c89af8890 con 0x561c89aefcb0 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.112+0000 7fe9337fe640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89af81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe938dcb640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 0x561c89ae8ed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe92c02f7d0 tx=0x7fe92c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b9da90ea92b323 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe9137fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe9137fe640 1 -- 172.21.15.64:0/2231666436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c055060 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe9137fe640 1 -- 172.21.15.64:0/2231666436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe92c004450 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92401cdb0 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe9137fe640 1 -- 172.21.15.64:0/2231666436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe92c034050 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe93d5ca740 10 client.15795 fetch_fsmap learned FSMap version 25 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c89b69c60 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.113+0000 7fe9137fe640 1 -- 172.21.15.64:0/2231666436 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92401cdb0 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe9137fe640 1 -- 172.21.15.64:0/2231666436 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe92c044070 con 0x561c89aefcb0 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe93d5ca740 10 client.15795 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe93d5ca740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe93d5ca740 20 client.15795 populate_metadata read hostname 'smithi064' 2024-09-19T05:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89b6a3e0 0x561c89b6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe93d5ca740 4 client.15795 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c89a50d70 con 0x561c89b6a3e0 2024-09-19T05:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.114+0000 7fe933fff640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89b6a3e0 0x561c89b6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.115+0000 7fe933fff640 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89b6a3e0 0x561c89b6c800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe240aa9e80b263 server_cookie=3a513eaf6e90ac59 in_seq=0 out_seq=0 2024-09-19T05:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.115+0000 7fe9137fe640 10 client.15795 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.118+0000 7fe9137fe640 1 -- 172.21.15.64:0/2231666436 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c89a50d70 con 0x561c89b6a3e0 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.118+0000 7fe9137fe640 10 client.15795 handle_command_reply: tid=0 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.118+0000 7fe93d5ca740 1 client.15795 shutdown 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 msgr2=0x561c89ae8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 0x561c89ae8ed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe92c02f7d0 tx=0x7fe92c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 shutdown_connections 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561c89b6a3e0 0x561c89b6c800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c89aefcb0 0x561c89ae8ed0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c89ae5410 0x561c89af81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 --2- 172.21.15.64:0/2231666436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c89ae4af0 0x561c89af7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 >> 172.21.15.64:0/2231666436 conn(0x561c89ae1060 msgr2=0x561c89ae6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe9137fe640 0 client.15795 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 shutdown_connections 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 1 -- 172.21.15.64:0/2231666436 wait complete. 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93d5ca740 20 client.15795 trim_cache size 0 max 16384 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe91c0a3460 msgr2=0x7fe91c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93a82e640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe91c0a3460 0x7fe91c0a5920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009a60 tx=0x7fe92c03c040 comp rx=0 tx=0).stop 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 msgr2=0x7fe934100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.119+0000 7fe93a82e640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 0x7fe934100e80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe92800e9f0 tx=0x7fe92800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 shutdown_connections 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe91c0a3460 0x7fe91c0a5920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93410c0c0 0x7fe9341013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe934103610 0x7fe934100e80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 --2- 172.21.15.64:0/3115404720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe934102cf0 0x7fe934100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 >> 172.21.15.64:0/3115404720 conn(0x7fe9340fc6e0 msgr2=0x7fe9340fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 shutdown_connections 2024-09-19T05:07:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:07:50.120+0000 7fe93a82e640 1 -- 172.21.15.64:0/3115404720 wait complete. 2024-09-19T05:07:51.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:51 smithi174 ceph-mon[34488]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-19T05:07:51.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:51 smithi064 ceph-mon[30960]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-19T05:07:51.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:51 smithi103 ceph-mon[34446]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-19T05:07:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:53 smithi174 ceph-mon[34488]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:07:53.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:53 smithi064 ceph-mon[30960]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:07:53.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:53 smithi103 ceph-mon[34446]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:07:55.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:55 smithi174 ceph-mon[34488]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-19T05:07:55.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:55 smithi064 ceph-mon[30960]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-19T05:07:55.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:55 smithi103 ceph-mon[34446]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-19T05:07:57.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:57 smithi174 ceph-mon[34488]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T05:07:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:57 smithi064 ceph-mon[30960]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T05:07:57.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:57 smithi103 ceph-mon[34446]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T05:07:59.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:07:59 smithi174 ceph-mon[34488]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T05:07:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:07:59 smithi064 ceph-mon[30960]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T05:07:59.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:07:59 smithi103 ceph-mon[34446]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-19T05:08:01.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:01 smithi174 ceph-mon[34488]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-19T05:08:01.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:01 smithi064 ceph-mon[30960]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-19T05:08:01.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:01 smithi103 ceph-mon[34446]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-19T05:08:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 -- 172.21.15.64:0/1928844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c107b70 msgr2=0x7f152c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 --2- 172.21.15.64:0/1928844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c107b70 0x7f152c107f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f151c009930 tx=0x7f151c02a420 comp rx=0 tx=0).stop 2024-09-19T05:08:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 -- 172.21.15.64:0/1928844928 shutdown_connections 2024-09-19T05:08:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 --2- 172.21.15.64:0/1928844928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f152c102470 0x7f152c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 --2- 172.21.15.64:0/1928844928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f152c101ad0 0x7f152c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 --2- 172.21.15.64:0/1928844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c107b70 0x7f152c107f50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 -- 172.21.15.64:0/1928844928 >> 172.21.15.64:0/1928844928 conn(0x7f152c0fd7d0 msgr2=0x7f152c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.212+0000 7f1531bf8640 1 -- 172.21.15.64:0/1928844928 shutdown_connections 2024-09-19T05:08:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.213+0000 7f1531bf8640 1 -- 172.21.15.64:0/1928844928 wait complete. 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.213+0000 7f1531bf8640 1 Processor -- start 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.213+0000 7f1531bf8640 1 -- start start 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f1531bf8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f152c101ad0 0x7f152c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f1531bf8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 0x7f152c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f1531bf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f152c107b70 0x7f152c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f1531bf8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111d90 con 0x7f152c102470 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f1531bf8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111c10 con 0x7f152c107b70 2024-09-19T05:08:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f1531bf8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f152c111f10 con 0x7f152c101ad0 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 0x7f152c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 0x7f152c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43390/0 (socket says 172.21.15.64:43390) 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 -- 172.21.15.64:0/4287361803 learned_addr learned my addr 172.21.15.64:0/4287361803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152b7fe640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f152c101ad0 0x7f152c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152bfff640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f152c107b70 0x7f152c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 -- 172.21.15.64:0/4287361803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f152c101ad0 msgr2=0x7f152c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f152c101ad0 0x7f152c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 -- 172.21.15.64:0/4287361803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f152c107b70 msgr2=0x7f152c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f152c107b70 0x7f152c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 -- 172.21.15.64:0/4287361803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c19d2b0 con 0x7f152c102470 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.214+0000 7f152affd640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 0x7f152c198800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f152000eab0 tx=0x7f152000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.215+0000 7f1528ff9640 1 -- 172.21.15.64:0/4287361803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000ce40 con 0x7f152c102470 2024-09-19T05:08:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.215+0000 7f1528ff9640 1 -- 172.21.15.64:0/4287361803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1520011920 con 0x7f152c102470 2024-09-19T05:08:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.215+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c19d5a0 con 0x7f152c102470 2024-09-19T05:08:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.215+0000 7f1528ff9640 1 -- 172.21.15.64:0/4287361803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520010a00 con 0x7f152c102470 2024-09-19T05:08:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.215+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c1a4e30 con 0x7f152c102470 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.217+0000 7f1528ff9640 1 -- 172.21.15.64:0/4287361803 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1520011ad0 con 0x7f152c102470 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.217+0000 7f1534a67740 1 Processor -- start 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 -- start start 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749caf0 0x55f95749ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f9574012f0 con 0x55f95749d410 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9573a9270 con 0x55f95749caf0 2024-09-19T05:08:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1534a67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9573fd2b0 con 0x55f9574a7cb0 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43404/0 (socket says 172.21.15.64:43404) 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 -- 172.21.15.64:0/2538011025 learned_addr learned my addr 172.21.15.64:0/2538011025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152bfff640 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749caf0 0x55f95749ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152affd640 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 -- 172.21.15.64:0/2538011025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 msgr2=0x55f9574aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 -- 172.21.15.64:0/2538011025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749caf0 msgr2=0x55f95749ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749caf0 0x55f95749ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f1528ff9640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f15140a3460 0x7f15140a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.218+0000 7f152b7fe640 1 -- 172.21.15.64:0/2538011025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9573a8340 con 0x55f95749d410 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.219+0000 7f152bfff640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f15140a3460 0x7f15140a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.219+0000 7f1528ff9640 1 -- 172.21.15.64:0/4287361803 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f15200d03f0 con 0x7f152c102470 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.219+0000 7f152b7fe640 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a7770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f151c031440 tx=0x7f151c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7302f089bff1e41c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.219+0000 7f1511ffb640 1 -- 172.21.15.64:0/2538011025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c03f070 con 0x55f95749d410 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.219+0000 7f1511ffb640 1 -- 172.21.15.64:0/2538011025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c032e20 con 0x55f95749d410 2024-09-19T05:08:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.219+0000 7f1511ffb640 1 -- 172.21.15.64:0/2538011025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035070 con 0x55f95749d410 2024-09-19T05:08:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 -- 172.21.15.64:0/2538011025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 msgr2=0x55f9574a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f152bfff640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f15140a3460 0x7f15140a5920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1524005fd0 tx=0x7f152400b5b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a7770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f151c031440 tx=0x7f151c032c30 comp rx=0 tx=0).stop 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 -- 172.21.15.64:0/2538011025 shutdown_connections 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a7770 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 --2- 172.21.15.64:0/2538011025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749caf0 0x55f95749ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 -- 172.21.15.64:0/2538011025 >> 172.21.15.64:0/2538011025 conn(0x55f957499060 msgr2=0x55f957499440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 -- 172.21.15.64:0/2538011025 shutdown_connections 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 -- 172.21.15.64:0/2538011025 wait complete. 2024-09-19T05:08:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.220+0000 7f1534a67740 1 -- start start 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 Processor -- start 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749d410 0x55f9574a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f9574aa880 con 0x55f95749caf0 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f95738a290 con 0x55f95749d410 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9574aaa00 con 0x55f9574a7cb0 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f1534a67740 10 client.0 _ll_register_callbacks cb 0x55f9573bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43418/0 (socket says 172.21.15.64:43418) 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 -- 172.21.15.64:0/2078792069 learned_addr learned my addr 172.21.15.64:0/2078792069 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152bfff640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152affd640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749d410 0x55f9574a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 -- 172.21.15.64:0/2078792069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 msgr2=0x55f9574a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 -- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749d410 msgr2=0x55f9574a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749d410 0x55f9574a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.221+0000 7f152b7fe640 1 -- 172.21.15.64:0/2078792069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f957114210 con 0x55f95749caf0 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f152b7fe640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a4b70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f151c02a940 tx=0x7f151c031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59d601587ab8e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f150affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f150affd640 1 -- 172.21.15.64:0/2078792069 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c04b060 con 0x55f95749caf0 2024-09-19T05:08:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f150affd640 1 -- 172.21.15.64:0/2078792069 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f151c004440 con 0x55f95749caf0 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c032e20 con 0x55f95749caf0 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f150affd640 1 -- 172.21.15.64:0/2078792069 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f151c03f030 con 0x55f95749caf0 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f1534a67740 10 client.15807 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9571e0620 con 0x55f95749caf0 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f150affd640 1 -- 172.21.15.64:0/2078792069 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c032e20 con 0x55f95749caf0 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f150affd640 1 -- 172.21.15.64:0/2078792069 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f151c03b7d0 con 0x55f95749caf0 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f1534a67740 10 client.15807 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f1534a67740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.222+0000 7f1534a67740 20 client.15807 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.223+0000 7f1534a67740 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f9574b7a60 0x55f9574b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.223+0000 7f1534a67740 4 client.15807 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.223+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95740ea20 con 0x55f9574b7a60 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.223+0000 7f152affd640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f9574b7a60 0x55f9574b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.223+0000 7f150affd640 10 client.15807 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.223+0000 7f152affd640 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f9574b7a60 0x55f9574b9e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacb5131e956132 server_cookie=3138d78750b95975 in_seq=0 out_seq=0 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f150affd640 1 -- 172.21.15.64:0/2078792069 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f95740ea20 con 0x55f9574b7a60 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f150affd640 10 client.15807 handle_command_reply: tid=0 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 client.15807 shutdown 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 msgr2=0x55f9574a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a4b70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f151c02a940 tx=0x7f151c031890 comp rx=0 tx=0).stop 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 shutdown_connections 2024-09-19T05:08:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9574a7cb0 0x55f9574a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f95749d410 0x55f9574a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f9574b7a60 0x55f9574b9e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 --2- 172.21.15.64:0/2078792069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a4b70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f150affd640 0 client.15807 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 >> 172.21.15.64:0/2078792069 conn(0x55f957499060 msgr2=0x55f95749edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 shutdown_connections 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 1 -- 172.21.15.64:0/2078792069 wait complete. 2024-09-19T05:08:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.224+0000 7f1534a67740 20 client.15807 trim_cache size 0 max 16384 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 Processor -- start 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 -- start start 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749d410 0x55f9574bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1b0020 con 0x55f95749caf0 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ad8f0 con 0x55f9574a7cb0 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f1534a67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ab740 con 0x55f95749d410 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.240+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43420/0 (socket says 172.21.15.64:43420) 2024-09-19T05:08:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 -- 172.21.15.64:0/1110617401 learned_addr learned my addr 172.21.15.64:0/1110617401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152bfff640 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152affd640 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749d410 0x55f9574bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 -- 172.21.15.64:0/1110617401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749d410 msgr2=0x55f9574bf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749d410 0x55f9574bf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 -- 172.21.15.64:0/1110617401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 msgr2=0x55f9574bf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574bf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 -- 172.21.15.64:0/1110617401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f957498e90 con 0x55f95749caf0 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f152b7fe640 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a5420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f151c000c00 tx=0x7f151c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1925f1bd2fac34ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f15117fa640 1 -- 172.21.15.64:0/1110617401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c04b030 con 0x55f95749caf0 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f15117fa640 1 -- 172.21.15.64:0/1110617401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c03b8e0 con 0x55f95749caf0 2024-09-19T05:08:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.241+0000 7f15117fa640 1 -- 172.21.15.64:0/1110617401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035040 con 0x55f95749caf0 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 -- 172.21.15.64:0/1110617401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 msgr2=0x55f9574a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a5420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f151c000c00 tx=0x7f151c032c30 comp rx=0 tx=0).stop 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 -- 172.21.15.64:0/1110617401 shutdown_connections 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749d410 0x55f9574bf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 --2- 172.21.15.64:0/1110617401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749caf0 0x55f9574a5420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 -- 172.21.15.64:0/1110617401 >> 172.21.15.64:0/1110617401 conn(0x55f957499060 msgr2=0x55f9574a6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 -- 172.21.15.64:0/1110617401 shutdown_connections 2024-09-19T05:08:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.242+0000 7f1534a67740 1 -- 172.21.15.64:0/1110617401 wait complete. 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 -- start start 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 Processor -- start 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749caf0 0x55f9574a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1abdb0 con 0x55f95749d410 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ad220 con 0x55f9574a7cb0 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ac480 con 0x55f95749caf0 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f1534a67740 10 client.0 _ll_register_callbacks cb 0x7f151c032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f152affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f152affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43424/0 (socket says 172.21.15.64:43424) 2024-09-19T05:08:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f152affd640 1 -- 172.21.15.64:0/3830972500 learned_addr learned my addr 172.21.15.64:0/3830972500 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f152b7fe640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749caf0 0x55f9574a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.243+0000 7f152bfff640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152affd640 1 -- 172.21.15.64:0/3830972500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749caf0 msgr2=0x55f9574a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152affd640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749caf0 0x55f9574a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152affd640 1 -- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 msgr2=0x55f9574a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152affd640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152affd640 1 -- 172.21.15.64:0/3830972500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9575213c0 con 0x55f95749d410 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152bfff640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f152affd640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a1f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1520098b30 tx=0x7f15200094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccba304a7491e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f150affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f150affd640 1 -- 172.21.15.64:0/3830972500 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152009cdd0 con 0x55f95749d410 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f150affd640 1 -- 172.21.15.64:0/3830972500 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1520098d40 con 0x55f95749d410 2024-09-19T05:08:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c03b8e0 con 0x55f95749d410 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f150affd640 1 -- 172.21.15.64:0/3830972500 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f152009c6d0 con 0x55f95749d410 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f1534a67740 10 client.15813 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f957522610 con 0x55f95749d410 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.244+0000 7f150affd640 1 -- 172.21.15.64:0/3830972500 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c03b8e0 con 0x55f95749d410 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f150affd640 1 -- 172.21.15.64:0/3830972500 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f152000f040 con 0x55f95749d410 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f1534a67740 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f1534a67740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f1534a67740 20 client.15813 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f1534a67740 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f957522d90 0x55f9575251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f1534a67740 4 client.15813 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f957408d70 con 0x55f957522d90 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f152b7fe640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f957522d90 0x55f9575251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:03.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f152b7fe640 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f957522d90 0x55f9575251b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed2e06c19cc6bddb server_cookie=d1c32c1081fa95ec in_seq=0 out_seq=0 2024-09-19T05:08:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.245+0000 7f150affd640 10 client.15813 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f150affd640 1 -- 172.21.15.64:0/3830972500 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f957408d70 con 0x55f957522d90 2024-09-19T05:08:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f150affd640 10 client.15813 handle_command_reply: tid=0 2024-09-19T05:08:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 client.15813 shutdown 2024-09-19T05:08:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 msgr2=0x55f9574a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a1f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1520098b30 tx=0x7f15200094a0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 shutdown_connections 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9574a7cb0 0x55f9574a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f957522d90 0x55f9575251b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f95749d410 0x55f9574a1f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f150affd640 0 client.15813 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 --2- 172.21.15.64:0/3830972500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f95749caf0 0x55f9574a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 >> 172.21.15.64:0/3830972500 conn(0x55f957499060 msgr2=0x55f95749e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 shutdown_connections 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 1 -- 172.21.15.64:0/3830972500 wait complete. 2024-09-19T05:08:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.479+0000 7f1534a67740 20 client.15813 trim_cache size 0 max 16384 2024-09-19T05:08:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:08:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:08:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:08:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:08:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f15140a3460 msgr2=0x7f15140a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f15140a3460 0x7f15140a5920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1524005fd0 tx=0x7f152400b5b0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 msgr2=0x7f152c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 0x7f152c198800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f152000eab0 tx=0x7f152000ef70 comp rx=0 tx=0).stop 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 shutdown_connections 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f15140a3460 0x7f15140a5920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f152c107b70 0x7f152c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f152c102470 0x7f152c198800 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 --2- 172.21.15.64:0/4287361803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f152c101ad0 0x7f152c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 >> 172.21.15.64:0/4287361803 conn(0x7f152c0fd7d0 msgr2=0x7f152c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 shutdown_connections 2024-09-19T05:08:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:03.480+0000 7f1531bf8640 1 -- 172.21.15.64:0/4287361803 wait complete. 2024-09-19T05:08:03.771 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:03 smithi103 ceph-mon[34446]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-19T05:08:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:03 smithi174 ceph-mon[34488]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-19T05:08:03.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:03 smithi064 ceph-mon[30960]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-19T05:08:04.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:08:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:04 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:08:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:04 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:08:04.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:04.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:04 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:05.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:05 smithi174 ceph-mon[34488]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-19T05:08:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:08:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:08:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:05.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:05 smithi064 ceph-mon[30960]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-19T05:08:05.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:08:05.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:08:05.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:05.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:05 smithi103 ceph-mon[34446]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-19T05:08:05.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:08:05.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:08:05.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:08:07.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:07 smithi174 ceph-mon[34488]: pgmap v69: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 235 KiB/s wr, 25 op/s 2024-09-19T05:08:07.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:07 smithi064 ceph-mon[30960]: pgmap v69: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 235 KiB/s wr, 25 op/s 2024-09-19T05:08:07.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:07 smithi103 ceph-mon[34446]: pgmap v69: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 235 KiB/s wr, 25 op/s 2024-09-19T05:08:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:09 smithi174 ceph-mon[34488]: pgmap v70: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:09.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:09 smithi064 ceph-mon[30960]: pgmap v70: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:09.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:09 smithi103 ceph-mon[34446]: pgmap v70: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:11 smithi174 ceph-mon[34488]: pgmap v71: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:11.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:11 smithi064 ceph-mon[30960]: pgmap v71: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:11.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:11 smithi103 ceph-mon[34446]: pgmap v71: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:13.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:13 smithi174 ceph-mon[34488]: pgmap v72: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 234 KiB/s wr, 24 op/s 2024-09-19T05:08:13.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:13 smithi064 ceph-mon[30960]: pgmap v72: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 234 KiB/s wr, 24 op/s 2024-09-19T05:08:13.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:13 smithi103 ceph-mon[34446]: pgmap v72: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 234 KiB/s wr, 24 op/s 2024-09-19T05:08:15.782 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:15 smithi174 ceph-mon[34488]: pgmap v73: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:15.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:15 smithi064 ceph-mon[30960]: pgmap v73: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:15.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:15 smithi103 ceph-mon[34446]: pgmap v73: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 234 KiB/s wr, 22 op/s 2024-09-19T05:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 -- 172.21.15.64:0/2966101221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c102c80 msgr2=0x7f635c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 --2- 172.21.15.64:0/2966101221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c102c80 0x7f635c103060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6358009900 tx=0x7f635802a480 comp rx=0 tx=0).stop 2024-09-19T05:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 -- 172.21.15.64:0/2966101221 shutdown_connections 2024-09-19T05:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 --2- 172.21.15.64:0/2966101221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f635c10c0d0 0x7f635c10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 --2- 172.21.15.64:0/2966101221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f635c1035a0 0x7f635c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 --2- 172.21.15.64:0/2966101221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c102c80 0x7f635c103060 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 -- 172.21.15.64:0/2966101221 >> 172.21.15.64:0/2966101221 conn(0x7f635c077990 msgr2=0x7f635c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 -- 172.21.15.64:0/2966101221 shutdown_connections 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.590+0000 7f6363480640 1 -- 172.21.15.64:0/2966101221 wait complete. 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 Processor -- start 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 -- start start 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f635c102c80 0x7f635c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f635c1035a0 0x7f635c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 0x7f635c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f635c111d60 con 0x7f635c10c0d0 2024-09-19T05:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f635c111be0 con 0x7f635c102c80 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6363480640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f635c111ee0 con 0x7f635c1035a0 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.591+0000 7f6361a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 0x7f635c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 0x7f635c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58552/0 (socket says 172.21.15.64:58552) 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 -- 172.21.15.64:0/3103388561 learned_addr learned my addr 172.21.15.64:0/3103388561 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f636121c640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f635c102c80 0x7f635c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6360a1b640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f635c1035a0 0x7f635c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 -- 172.21.15.64:0/3103388561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f635c1035a0 msgr2=0x7f635c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f635c1035a0 0x7f635c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 -- 172.21.15.64:0/3103388561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f635c102c80 msgr2=0x7f635c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f635c102c80 0x7f635c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 -- 172.21.15.64:0/3103388561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c19d260 con 0x7f635c10c0d0 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6361a1d640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 0x7f635c19cb30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f635400eaa0 tx=0x7f635400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f63527fc640 1 -- 172.21.15.64:0/3103388561 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635400cdb0 con 0x7f635c10c0d0 2024-09-19T05:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c19d550 con 0x7f635c10c0d0 2024-09-19T05:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f63527fc640 1 -- 172.21.15.64:0/3103388561 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6354005020 con 0x7f635c10c0d0 2024-09-19T05:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f63527fc640 1 -- 172.21.15.64:0/3103388561 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540116d0 con 0x7f635c10c0d0 2024-09-19T05:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.592+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c1a4de0 con 0x7f635c10c0d0 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.595+0000 7f636620e740 1 Processor -- start 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f63527fc640 1 -- 172.21.15.64:0/3103388561 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f63540051d0 con 0x7f635c10c0d0 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 -- start start 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a23ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5654f79882f0 con 0x5654f7a23af0 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654f7930270 con 0x5654f7a24410 2024-09-19T05:08:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636620e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5654f79842b0 con 0x5654f7a2ecb0 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f6360a1b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a23ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f6360a1b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a23ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58566/0 (socket says 172.21.15.64:58566) 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f6360a1b640 1 -- 172.21.15.64:0/3459317075 learned_addr learned my addr 172.21.15.64:0/3459317075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.596+0000 7f636121c640 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 -- 172.21.15.64:0/3459317075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 msgr2=0x5654f7a310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 -- 172.21.15.64:0/3459317075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 msgr2=0x5654f7a2e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 -- 172.21.15.64:0/3459317075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f792f340 con 0x5654f7a23af0 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f63527fc640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f63480a3460 0x7f63480a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f6360a1b640 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a23ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f634c003dd0 tx=0x7f634c0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9be963cd7fbc26a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f636121c640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f63480a3460 0x7f63480a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f63527fc640 1 -- 172.21.15.64:0/3103388561 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f63540cfeb0 con 0x7f635c10c0d0 2024-09-19T05:08:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f63477fe640 1 -- 172.21.15.64:0/3459317075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c019020 con 0x5654f7a23af0 2024-09-19T05:08:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f63477fe640 1 -- 172.21.15.64:0/3459317075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f634c0110d0 con 0x5654f7a23af0 2024-09-19T05:08:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.597+0000 7f63477fe640 1 -- 172.21.15.64:0/3459317075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c012650 con 0x5654f7a23af0 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636121c640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f63480a3460 0x7f63480a5920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6358031440 tx=0x7f6358005c00 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 -- 172.21.15.64:0/3459317075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 msgr2=0x5654f7a23ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a23ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f634c003dd0 tx=0x7f634c0043a0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 -- 172.21.15.64:0/3459317075 shutdown_connections 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 --2- 172.21.15.64:0/3459317075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a23ed0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 -- 172.21.15.64:0/3459317075 >> 172.21.15.64:0/3459317075 conn(0x5654f7a20060 msgr2=0x5654f7a20440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 -- 172.21.15.64:0/3459317075 shutdown_connections 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.598+0000 7f636620e740 1 -- 172.21.15.64:0/3459317075 wait complete. 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 -- start start 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 Processor -- start 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a2bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5654f7a35c90 con 0x5654f7a2ecb0 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654f7911290 con 0x5654f7a24410 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5654f7a35e10 con 0x5654f7a23af0 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.599+0000 7f636620e740 10 client.0 _ll_register_callbacks cb 0x5654f7942140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a2c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58580/0 (socket says 172.21.15.64:58580) 2024-09-19T05:08:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 -- 172.21.15.64:0/4215276453 learned_addr learned my addr 172.21.15.64:0/4215276453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f636121c640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a2bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6360a1b640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 -- 172.21.15.64:0/4215276453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 msgr2=0x5654f7a2bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a2bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 -- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 msgr2=0x5654f7a2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 -- 172.21.15.64:0/4215276453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f769b210 con 0x5654f7a2ecb0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6361a1d640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a2c620 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f63540cdc00 tx=0x7f635409c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e445092ecfab2f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6344ff9640 1 -- 172.21.15.64:0/4215276453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635401f030 con 0x5654f7a2ecb0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f634c0110d0 con 0x5654f7a2ecb0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6344ff9640 1 -- 172.21.15.64:0/4215276453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f63540bc070 con 0x5654f7a2ecb0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f6344ff9640 1 -- 172.21.15.64:0/4215276453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63540bc270 con 0x5654f7a2ecb0 2024-09-19T05:08:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f636620e740 10 client.15825 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.600+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654f7767620 con 0x5654f7a2ecb0 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f6344ff9640 1 -- 172.21.15.64:0/4215276453 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f634c0110d0 con 0x5654f7a2ecb0 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f6344ff9640 1 -- 172.21.15.64:0/4215276453 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f635400f040 con 0x5654f7a2ecb0 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636620e740 10 client.15825 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636620e740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636620e740 20 client.15825 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636620e740 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7a43060 0x5654f7a45450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636620e740 4 client.15825 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f7995a20 con 0x5654f7a43060 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.601+0000 7f636121c640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7a43060 0x5654f7a45450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.602+0000 7f636121c640 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7a43060 0x5654f7a45450 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14dfa7307b347c95 server_cookie=a66ae0e64bfd6c60 in_seq=0 out_seq=0 2024-09-19T05:08:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.602+0000 7f6344ff9640 10 client.15825 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.602+0000 7f6344ff9640 1 -- 172.21.15.64:0/4215276453 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654f7995a20 con 0x5654f7a43060 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.602+0000 7f6344ff9640 10 client.15825 handle_command_reply: tid=0 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.602+0000 7f636620e740 1 client.15825 shutdown 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 msgr2=0x5654f7a2c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a2c620 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f63540cdc00 tx=0x7f635409c9c0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 shutdown_connections 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7a43060 0x5654f7a45450 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a2c620 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 --2- 172.21.15.64:0/4215276453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a2bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 >> 172.21.15.64:0/4215276453 conn(0x5654f7a20060 msgr2=0x5654f7a25dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f6344ff9640 0 client.15825 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 shutdown_connections 2024-09-19T05:08:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 1 -- 172.21.15.64:0/4215276453 wait complete. 2024-09-19T05:08:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.603+0000 7f636620e740 20 client.15825 trim_cache size 0 max 16384 2024-09-19T05:08:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.618+0000 7f636620e740 1 Processor -- start 2024-09-19T05:08:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.618+0000 7f636620e740 1 -- start start 2024-09-19T05:08:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636620e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a4ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636620e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636620e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a28e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636620e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1ade20 con 0x5654f7a2ecb0 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636620e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1ab6f0 con 0x5654f7a24410 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636620e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1a9540 con 0x5654f7a23af0 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a28e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a28e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58582/0 (socket says 172.21.15.64:58582) 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 -- 172.21.15.64:0/1140828511 learned_addr learned my addr 172.21.15.64:0/1140828511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6360a1b640 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f636121c640 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a4ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 -- 172.21.15.64:0/1140828511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 msgr2=0x5654f7a4ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a4ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 -- 172.21.15.64:0/1140828511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 msgr2=0x5654f7a4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a4b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.619+0000 7f6361a1d640 1 -- 172.21.15.64:0/1140828511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f7a1fe90 con 0x5654f7a2ecb0 2024-09-19T05:08:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.620+0000 7f6361a1d640 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a28e00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f635409c120 tx=0x7f6354005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f8cc06c854535b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.620+0000 7f6346ffd640 1 -- 172.21.15.64:0/1140828511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635401f070 con 0x5654f7a2ecb0 2024-09-19T05:08:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.620+0000 7f6346ffd640 1 -- 172.21.15.64:0/1140828511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63540bc0c0 con 0x5654f7a2ecb0 2024-09-19T05:08:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.620+0000 7f6346ffd640 1 -- 172.21.15.64:0/1140828511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540a02b0 con 0x5654f7a2ecb0 2024-09-19T05:08:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 -- 172.21.15.64:0/1140828511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 msgr2=0x5654f7a28e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a28e00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f635409c120 tx=0x7f6354005770 comp rx=0 tx=0).stop 2024-09-19T05:08:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 -- 172.21.15.64:0/1140828511 shutdown_connections 2024-09-19T05:08:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a2ecb0 0x5654f7a28e00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 --2- 172.21.15.64:0/1140828511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a23af0 0x5654f7a4ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 -- 172.21.15.64:0/1140828511 >> 172.21.15.64:0/1140828511 conn(0x5654f7a20060 msgr2=0x5654f7a2dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 -- 172.21.15.64:0/1140828511 shutdown_connections 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 -- 172.21.15.64:0/1140828511 wait complete. 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 -- start start 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.621+0000 7f636620e740 1 Processor -- start 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1a9bb0 con 0x5654f7a23af0 2024-09-19T05:08:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1ab020 con 0x5654f7a24410 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1aa280 con 0x5654f7a2ecb0 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636620e740 10 client.0 _ll_register_callbacks cb 0x7f6354005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a28ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58584/0 (socket says 172.21.15.64:58584) 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 -- 172.21.15.64:0/698801525 learned_addr learned my addr 172.21.15.64:0/698801525 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f6360a1b640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f6361a1d640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 -- 172.21.15.64:0/698801525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 msgr2=0x5654f7a29760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a29760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 -- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 msgr2=0x5654f7a29220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a29220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 -- 172.21.15.64:0/698801525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f7aacda0 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f6360a1b640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f636121c640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a28ce0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f635802a9a0 tx=0x7f635803f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdebad6effe2bf16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.622+0000 7f6344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f6344ff9640 1 -- 172.21.15.64:0/698801525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358035040 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f6344ff9640 1 -- 172.21.15.64:0/698801525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6358004450 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63540bc0c0 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f6344ff9640 1 -- 172.21.15.64:0/698801525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6358005040 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 10 client.15831 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654f7aae0f0 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f6344ff9640 1 -- 172.21.15.64:0/698801525 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63540bc0c0 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f6344ff9640 1 -- 172.21.15.64:0/698801525 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f63580035f0 con 0x5654f7a23af0 2024-09-19T05:08:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 10 client.15831 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 20 client.15831 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7aae820 0x5654f7ab0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 4 client.15831 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.623+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654f798fd70 con 0x5654f7aae820 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.624+0000 7f6360a1b640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7aae820 0x5654f7ab0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.624+0000 7f6344ff9640 10 client.15831 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.624+0000 7f6360a1b640 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7aae820 0x5654f7ab0c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0352eb42684e30 server_cookie=a4bad48cb8c80c46 in_seq=0 out_seq=0 2024-09-19T05:08:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f6344ff9640 1 -- 172.21.15.64:0/698801525 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654f798fd70 con 0x5654f7aae820 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f6344ff9640 10 client.15831 handle_command_reply: tid=0 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 client.15831 shutdown 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 msgr2=0x5654f7a28ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a28ce0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f635802a9a0 tx=0x7f635803f070 comp rx=0 tx=0).stop 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 shutdown_connections 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654f7a2ecb0 0x5654f7a29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f7a24410 0x5654f7a29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5654f7aae820 0x5654f7ab0c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f6344ff9640 0 client.15831 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 --2- 172.21.15.64:0/698801525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5654f7a23af0 0x5654f7a28ce0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 >> 172.21.15.64:0/698801525 conn(0x5654f7a20060 msgr2=0x5654f7a2dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 shutdown_connections 2024-09-19T05:08:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.628+0000 7f636620e740 1 -- 172.21.15.64:0/698801525 wait complete. 2024-09-19T05:08:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f636620e740 20 client.15831 trim_cache size 0 max 16384 2024-09-19T05:08:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:08:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:08:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:08:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f63480a3460 msgr2=0x7f63480a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f63480a3460 0x7f63480a5920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6358031440 tx=0x7f6358005c00 comp rx=0 tx=0).stop 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 msgr2=0x7f635c19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 0x7f635c19cb30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f635400eaa0 tx=0x7f635400ef60 comp rx=0 tx=0).stop 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 shutdown_connections 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f63480a3460 0x7f63480a5920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635c10c0d0 0x7f635c19cb30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f635c1035a0 0x7f635c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 --2- 172.21.15.64:0/3103388561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f635c102c80 0x7f635c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 >> 172.21.15.64:0/3103388561 conn(0x7f635c077990 msgr2=0x7f635c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 shutdown_connections 2024-09-19T05:08:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:16.629+0000 7f6363480640 1 -- 172.21.15.64:0/3103388561 wait complete. 2024-09-19T05:08:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:17 smithi174 ceph-mon[34488]: pgmap v74: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 234 KiB/s wr, 24 op/s 2024-09-19T05:08:17.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:08:17.891 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:17 smithi064 ceph-mon[30960]: pgmap v74: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 234 KiB/s wr, 24 op/s 2024-09-19T05:08:17.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:17 smithi103 ceph-mon[34446]: pgmap v74: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 234 KiB/s wr, 24 op/s 2024-09-19T05:08:18.162 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 -- 172.21.15.64:0/110734325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 msgr2=0x7f268c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 --2- 172.21.15.64:0/110734325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c106810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2674009900 tx=0x7f267402a410 comp rx=0 tx=0).stop 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 -- 172.21.15.64:0/110734325 shutdown_connections 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 --2- 172.21.15.64:0/110734325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f268c106f40 0x7f268c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 --2- 172.21.15.64:0/110734325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c106810 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 --2- 172.21.15.64:0/110734325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 0x7f268c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.675+0000 7f26916cc640 1 -- 172.21.15.64:0/110734325 >> 172.21.15.64:0/110734325 conn(0x7f268c0fd8d0 msgr2=0x7f268c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.676+0000 7f26916cc640 1 -- 172.21.15.64:0/110734325 shutdown_connections 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.676+0000 7f26916cc640 1 -- 172.21.15.64:0/110734325 wait complete. 2024-09-19T05:08:18.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.676+0000 7f26916cc640 1 Processor -- start 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.676+0000 7f26916cc640 1 -- start start 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.676+0000 7f26916cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 0x7f268c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f26916cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f26916cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f268c106f40 0x7f268c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f26916cc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f268c111e50 con 0x7f268c1027c0 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f26916cc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f268c111cd0 con 0x7f268c101e10 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f26916cc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f268c111fd0 con 0x7f268c106f40 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58590/0 (socket says 172.21.15.64:58590) 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 -- 172.21.15.64:0/1211329393 learned_addr learned my addr 172.21.15.64:0/1211329393 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f268affd640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 0x7f268c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f268b7fe640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f268c106f40 0x7f268c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 -- 172.21.15.64:0/1211329393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f268c106f40 msgr2=0x7f268c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f268c106f40 0x7f268c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 -- 172.21.15.64:0/1211329393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 msgr2=0x7f268c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 0x7f268c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 -- 172.21.15.64:0/1211329393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c1a68b0 con 0x7f268c1027c0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f268affd640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 0x7f268c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.677+0000 7f2683fff640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c074a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2674005940 tx=0x7f2674004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.678+0000 7f2688ff9640 1 -- 172.21.15.64:0/1211329393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267403f030 con 0x7f268c1027c0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.678+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1a6b40 con 0x7f268c1027c0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.678+0000 7f2688ff9640 1 -- 172.21.15.64:0/1211329393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2674003aa0 con 0x7f268c1027c0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.678+0000 7f2688ff9640 1 -- 172.21.15.64:0/1211329393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674035070 con 0x7f268c1027c0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.678+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1a7070 con 0x7f268c1027c0 2024-09-19T05:08:18.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.680+0000 7f2688ff9640 1 -- 172.21.15.64:0/1211329393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2674042030 con 0x7f268c1027c0 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 Processor -- start 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 -- start start 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2688ff9640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f26780a3430 0x7f26780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f268b7fe640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f26780a3430 0x7f26780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2688ff9640 1 -- 172.21.15.64:0/1211329393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f26740f0370 con 0x7f268c1027c0 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644c472a2e0 con 0x5644c47c44a0 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644c46d1ce0 con 0x5644c47c4dc0 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f2693aa5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644c47262a0 con 0x5644c47cf680 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.681+0000 7f268affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58604/0 (socket says 172.21.15.64:58604) 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 -- 172.21.15.64:0/546939780 learned_addr learned my addr 172.21.15.64:0/546939780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f2683fff640 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268b7fe640 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 -- 172.21.15.64:0/546939780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 msgr2=0x5644c47d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268b7fe640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f26780a3430 0x7f26780a58f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2684005fd0 tx=0x7f2684009600 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 -- 172.21.15.64:0/546939780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 msgr2=0x5644c47cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 -- 172.21.15.64:0/546939780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c46b6440 con 0x5644c47c44a0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f268affd640 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47c4880 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f267c0104d0 tx=0x7f267c010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977619f8b77053e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f26817fa640 1 -- 172.21.15.64:0/546939780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c011760 con 0x5644c47c44a0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f26817fa640 1 -- 172.21.15.64:0/546939780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c011db0 con 0x5644c47c44a0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.682+0000 7f26817fa640 1 -- 172.21.15.64:0/546939780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01ce20 con 0x5644c47c44a0 2024-09-19T05:08:18.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 -- 172.21.15.64:0/546939780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 msgr2=0x5644c47c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47c4880 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f267c0104d0 tx=0x7f267c010990 comp rx=0 tx=0).stop 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 -- 172.21.15.64:0/546939780 shutdown_connections 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 --2- 172.21.15.64:0/546939780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47c4880 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 -- 172.21.15.64:0/546939780 >> 172.21.15.64:0/546939780 conn(0x5644c47c0a30 msgr2=0x5644c47c0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 -- 172.21.15.64:0/546939780 shutdown_connections 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.683+0000 7f2693aa5740 1 -- 172.21.15.64:0/546939780 wait complete. 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 -- start start 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 Processor -- start 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644c47d2250 con 0x5644c47c44a0 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644c46e5050 con 0x5644c47c4dc0 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644c47d23d0 con 0x5644c47cf680 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.684+0000 7f2693aa5740 10 client.0 _ll_register_callbacks cb 0x5644c4725160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:47620/0 (socket says 172.21.15.64:47620) 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 -- 172.21.15.64:0/808574952 learned_addr learned my addr 172.21.15.64:0/808574952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f2683fff640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268affd640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 -- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 msgr2=0x5644c47cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 -- 172.21.15.64:0/808574952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 msgr2=0x5644c47ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268b7fe640 1 -- 172.21.15.64:0/808574952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c449b3c0 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.685+0000 7f268affd640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f268b7fe640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47cc6d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2684012bf0 tx=0x7f2684004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869d7cbf805e637b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f26727fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f26727fc640 1 -- 172.21.15.64:0/808574952 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684011e30 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f26727fc640 1 -- 172.21.15.64:0/808574952 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2684003970 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c011db0 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f26727fc640 1 -- 172.21.15.64:0/808574952 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2684003b70 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f2693aa5740 10 client.25315 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644c47ddbd0 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.686+0000 7f26727fc640 1 -- 172.21.15.64:0/808574952 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c011db0 con 0x5644c47cf680 2024-09-19T05:08:18.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f26727fc640 1 -- 172.21.15.64:0/808574952 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f268400f040 con 0x5644c47cf680 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f2693aa5740 10 client.25315 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f2693aa5740 10 client.25315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f2693aa5740 20 client.25315 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f2693aa5740 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c47defe0 0x5644c47e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f2693aa5740 4 client.25315 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c4737850 con 0x5644c47defe0 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.687+0000 7f268affd640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c47defe0 0x5644c47e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.688+0000 7f268affd640 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c47defe0 0x5644c47e1480 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17a4457fe542063 server_cookie=9e3ef97977dfb77c in_seq=0 out_seq=0 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.688+0000 7f26727fc640 10 client.25315 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.688+0000 7f26727fc640 1 -- 172.21.15.64:0/808574952 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644c4737850 con 0x5644c47defe0 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.688+0000 7f26727fc640 10 client.25315 handle_command_reply: tid=0 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.688+0000 7f2693aa5740 1 client.25315 shutdown 2024-09-19T05:08:18.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 msgr2=0x5644c47cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47cc6d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2684012bf0 tx=0x7f2684004a40 comp rx=0 tx=0).stop 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 shutdown_connections 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47cf680 0x5644c47cc6d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c47defe0 0x5644c47e1480 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 --2- 172.21.15.64:0/808574952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c44a0 0x5644c47ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 >> 172.21.15.64:0/808574952 conn(0x5644c47c0a30 msgr2=0x5644c47c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f26727fc640 0 client.25315 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 shutdown_connections 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 1 -- 172.21.15.64:0/808574952 wait complete. 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.689+0000 7f2693aa5740 20 client.25315 trim_cache size 0 max 16384 2024-09-19T05:08:18.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.710+0000 7f2693aa5740 1 Processor -- start 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.710+0000 7f2693aa5740 1 -- start start 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 0x5644c47e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47cf680 0x5644c47e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2693aa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1b22b0 con 0x5644c47c4dc0 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2693aa5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1afb80 con 0x5644c47cf680 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2693aa5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1ad9d0 con 0x5644c47c44a0 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 0x5644c47e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 0x5644c47e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58628/0 (socket says 172.21.15.64:58628) 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 -- 172.21.15.64:0/3062760015 learned_addr learned my addr 172.21.15.64:0/3062760015 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f268affd640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f268b7fe640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47cf680 0x5644c47e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 -- 172.21.15.64:0/3062760015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 msgr2=0x5644c47ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 -- 172.21.15.64:0/3062760015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47cf680 msgr2=0x5644c47e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47cf680 0x5644c47e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 -- 172.21.15.64:0/3062760015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c47c42b0 con 0x5644c47c4dc0 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f268affd640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f268b7fe640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47cf680 0x5644c47e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.711+0000 7f2683fff640 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 0x5644c47e6da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f26740bc690 tx=0x7f26740bc240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdd06fdd07861c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.712+0000 7f2680ff9640 1 -- 172.21.15.64:0/3062760015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740d4070 con 0x5644c47c4dc0 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.712+0000 7f2680ff9640 1 -- 172.21.15.64:0/3062760015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267403be20 con 0x5644c47c4dc0 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.712+0000 7f2680ff9640 1 -- 172.21.15.64:0/3062760015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674049040 con 0x5644c47c4dc0 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 -- 172.21.15.64:0/3062760015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 msgr2=0x5644c47e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 0x5644c47e6da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f26740bc690 tx=0x7f26740bc240 comp rx=0 tx=0).stop 2024-09-19T05:08:18.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 -- 172.21.15.64:0/3062760015 shutdown_connections 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47cf680 0x5644c47e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47c4dc0 0x5644c47e6da0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 --2- 172.21.15.64:0/3062760015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 -- 172.21.15.64:0/3062760015 >> 172.21.15.64:0/3062760015 conn(0x5644c47c0a30 msgr2=0x5644c47cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 -- 172.21.15.64:0/3062760015 shutdown_connections 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 -- 172.21.15.64:0/3062760015 wait complete. 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 -- start start 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 Processor -- start 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.713+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47d4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2693aa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 0x5644c47d5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2693aa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1ae040 con 0x5644c47cf680 2024-09-19T05:08:18.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2693aa5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1af4b0 con 0x5644c47c4dc0 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2693aa5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1ae710 con 0x5644c47c44a0 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2693aa5740 10 client.0 _ll_register_callbacks cb 0x7f2684016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 0x5644c47d5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2683fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47d4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2683fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47d4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:60514/0 (socket says 172.21.15.64:60514) 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f2683fff640 1 -- 172.21.15.64:0/1445951924 learned_addr learned my addr 172.21.15.64:0/1445951924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 0x5644c47d5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58632/0 (socket says 172.21.15.64:58632) 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 -- 172.21.15.64:0/1445951924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 msgr2=0x5644c47d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47d7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 -- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 msgr2=0x5644c47d4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47d4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 -- 172.21.15.64:0/1445951924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c47d5c00 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268affd640 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f268b7fe640 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 0x5644c47d5510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26840318c0 tx=0x7f26840112b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3107aecd1ade670 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.714+0000 7f26727fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f26727fc640 1 -- 172.21.15.64:0/1445951924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684037040 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f26727fc640 1 -- 172.21.15.64:0/1445951924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2684031480 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267403be20 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f26727fc640 1 -- 172.21.15.64:0/1445951924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f26840117b0 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 10 client.15849 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644c4849be0 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f26727fc640 1 -- 172.21.15.64:0/1445951924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267403be20 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f26727fc640 1 -- 172.21.15.64:0/1445951924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2684005780 con 0x5644c47cf680 2024-09-19T05:08:18.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 20 client.15849 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c484a300 0x5644c484c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 4 client.15849 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.715+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644c4731d60 con 0x5644c484a300 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.716+0000 7f268affd640 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c484a300 0x5644c484c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.716+0000 7f268affd640 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c484a300 0x5644c484c6f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7647b606e2e1f498 server_cookie=e2eec5c6483d3825 in_seq=0 out_seq=0 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.716+0000 7f26727fc640 10 client.15849 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.716+0000 7f26727fc640 1 -- 172.21.15.64:0/1445951924 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5644c4731d60 con 0x5644c484a300 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.716+0000 7f26727fc640 10 client.15849 handle_command_reply: tid=0 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.716+0000 7f2693aa5740 1 client.15849 shutdown 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 msgr2=0x5644c47d5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 0x5644c47d5510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26840318c0 tx=0x7f26840112b0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 shutdown_connections 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5644c484a300 0x5644c484c6f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644c47cf680 0x5644c47d5510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f26727fc640 0 client.15849 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644c47c4dc0 0x5644c47d4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 --2- 172.21.15.64:0/1445951924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644c47c44a0 0x5644c47d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 >> 172.21.15.64:0/1445951924 conn(0x5644c47c0a30 msgr2=0x5644c47ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 shutdown_connections 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 1 -- 172.21.15.64:0/1445951924 wait complete. 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.717+0000 7f2693aa5740 20 client.15849 trim_cache size 0 max 16384 2024-09-19T05:08:18.735 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:08:18.736 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:08:18.736 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:08:18.736 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:08:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f26780a3430 msgr2=0x7f26780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f26780a3430 0x7f26780a58f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2684005fd0 tx=0x7f2684009600 comp rx=0 tx=0).stop 2024-09-19T05:08:18.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 msgr2=0x7f268c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c074a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2674005940 tx=0x7f2674004270 comp rx=0 tx=0).stop 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 shutdown_connections 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f26780a3430 0x7f26780a58f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f268c106f40 0x7f268c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c1027c0 0x7f268c074a30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 --2- 172.21.15.64:0/1211329393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f268c101e10 0x7f268c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 >> 172.21.15.64:0/1211329393 conn(0x7f268c0fd8d0 msgr2=0x7f268c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 shutdown_connections 2024-09-19T05:08:18.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:18.718+0000 7f26916cc640 1 -- 172.21.15.64:0/1211329393 wait complete. 2024-09-19T05:08:19.550 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:19 smithi064 ceph-mon[30960]: pgmap v75: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:08:19.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:08:19.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:08:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:19 smithi174 ceph-mon[34488]: pgmap v75: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:08:19.865 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:19.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:19 smithi103 ceph-mon[34446]: pgmap v75: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:08:20.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 -- 172.21.15.64:0/394627500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24101de0 msgr2=0x7fed241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 --2- 172.21.15.64:0/394627500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24101de0 0x7fed241021c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fed14009900 tx=0x7fed1402a410 comp rx=0 tx=0).stop 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 -- 172.21.15.64:0/394627500 shutdown_connections 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 --2- 172.21.15.64:0/394627500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed24106f10 0x7fed2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 --2- 172.21.15.64:0/394627500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed24102790 0x7fed241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 --2- 172.21.15.64:0/394627500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24101de0 0x7fed241021c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 -- 172.21.15.64:0/394627500 >> 172.21.15.64:0/394627500 conn(0x7fed240fd880 msgr2=0x7fed240ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 -- 172.21.15.64:0/394627500 shutdown_connections 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 -- 172.21.15.64:0/394627500 wait complete. 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.456+0000 7fed2945c640 1 Processor -- start 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 -- start start 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed24101de0 0x7fed24198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed24102790 0x7fed24198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 0x7fed2419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed24111e20 con 0x7fed24106f10 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed24111ca0 con 0x7fed24101de0 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed2945c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed24111fa0 con 0x7fed24102790 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 0x7fed2419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 0x7fed2419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58642/0 (socket says 172.21.15.64:58642) 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 -- 172.21.15.64:0/1039537948 learned_addr learned my addr 172.21.15.64:0/1039537948 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed227fc640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed24102790 0x7fed24198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed22ffd640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed24101de0 0x7fed24198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 -- 172.21.15.64:0/1039537948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed24102790 msgr2=0x7fed24198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed24102790 0x7fed24198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 -- 172.21.15.64:0/1039537948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed24101de0 msgr2=0x7fed24198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed24101de0 0x7fed24198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.457+0000 7fed237fe640 1 -- 172.21.15.64:0/1039537948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2419d2f0 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.458+0000 7fed237fe640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 0x7fed2419cc10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00eb10 tx=0x7fed1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.458+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c004b20 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.458+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed1c004cc0 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.458+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2419d5e0 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.458+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c010690 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.458+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241a4ec0 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.459+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed241021c0 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.461+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fed1c005020 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.461+0000 7fed0bfff640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fed0c0a3430 0x7fed0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.461+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fed1c0d02f0 con 0x7fed24106f10 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.461+0000 7fed22ffd640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fed0c0a3430 0x7fed0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.462+0000 7fed22ffd640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fed0c0a3430 0x7fed0c0a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed14031440 tx=0x7fed14005b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:20.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.467+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed1c09ccd0 con 0x7fed24106f10 2024-09-19T05:08:20.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.714+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed2402d7c0 con 0x7fed24106f10 2024-09-19T05:08:20.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.715+0000 7fed0bfff640 1 -- 172.21.15.64:0/1039537948 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fed1c09c5d0 con 0x7fed24106f10 2024-09-19T05:08:20.716 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:08:20.717 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:08:20.717 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fed0c0a3430 msgr2=0x7fed0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fed0c0a3430 0x7fed0c0a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed14031440 tx=0x7fed14005b50 comp rx=0 tx=0).stop 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 msgr2=0x7fed2419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 0x7fed2419cc10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00eb10 tx=0x7fed1c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 shutdown_connections 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fed0c0a3430 0x7fed0c0a58f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed24106f10 0x7fed2419cc10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed24102790 0x7fed24198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 --2- 172.21.15.64:0/1039537948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed24101de0 0x7fed24198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.717+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 >> 172.21.15.64:0/1039537948 conn(0x7fed240fd880 msgr2=0x7fed240fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.718+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 shutdown_connections 2024-09-19T05:08:20.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:20.718+0000 7fed2945c640 1 -- 172.21.15.64:0/1039537948 wait complete. 2024-09-19T05:08:21.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:08:21.811 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:21.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:21 smithi064 ceph-mon[30960]: pgmap v76: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T05:08:21.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:21 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1039537948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:21.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:21 smithi103 ceph-mon[34446]: pgmap v76: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T05:08:21.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:21 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1039537948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:22.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:21 smithi174 ceph-mon[34488]: pgmap v76: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T05:08:22.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:21 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1039537948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:22.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 -- 172.21.15.64:0/219875890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1082a0 msgr2=0x7f870c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 --2- 172.21.15.64:0/219875890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1082a0 0x7f870c10ec40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8704009cd0 tx=0x7f870402a910 comp rx=0 tx=0).stop 2024-09-19T05:08:22.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 -- 172.21.15.64:0/219875890 shutdown_connections 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 --2- 172.21.15.64:0/219875890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1082a0 0x7f870c10ec40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 --2- 172.21.15.64:0/219875890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870c1036e0 0x7f870c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 --2- 172.21.15.64:0/219875890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f870c102d30 0x7f870c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 -- 172.21.15.64:0/219875890 >> 172.21.15.64:0/219875890 conn(0x7f870c0fc7d0 msgr2=0x7f870c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 -- 172.21.15.64:0/219875890 shutdown_connections 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.366+0000 7f8711318640 1 -- 172.21.15.64:0/219875890 wait complete. 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 Processor -- start 2024-09-19T05:08:22.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 -- start start 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870c102d30 0x7f870c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 0x7f870c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f870c1082a0 0x7f870c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f870c111f90 con 0x7f870c1036e0 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f870c111e10 con 0x7f870c1082a0 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.367+0000 7f8711318640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f870c112110 con 0x7f870c102d30 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 0x7f870c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 0x7f870c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58664/0 (socket says 172.21.15.64:58664) 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f870c1082a0 0x7f870c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 -- 172.21.15.64:0/2470285802 learned_addr learned my addr 172.21.15.64:0/2470285802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870affd640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870c102d30 0x7f870c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 -- 172.21.15.64:0/2470285802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870c102d30 msgr2=0x7f870c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870c102d30 0x7f870c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 -- 172.21.15.64:0/2470285802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f870c1082a0 msgr2=0x7f870c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f870c1082a0 0x7f870c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 -- 172.21.15.64:0/2470285802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c1a6940 con 0x7f870c1036e0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f870a7fc640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 0x7f870c100ed0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8700006fd0 tx=0x7f870000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f86f3fff640 1 -- 172.21.15.64:0/2470285802 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700011830 con 0x7f870c1036e0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f86f3fff640 1 -- 172.21.15.64:0/2470285802 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8700016070 con 0x7f870c1036e0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1a6c30 con 0x7f870c1036e0 2024-09-19T05:08:22.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.368+0000 7f86f3fff640 1 -- 172.21.15.64:0/2470285802 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700010df0 con 0x7f870c1036e0 2024-09-19T05:08:22.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.369+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a7120 con 0x7f870c1036e0 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f86f3fff640 1 -- 172.21.15.64:0/2470285802 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f87000119d0 con 0x7f870c1036e0 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 Processor -- start 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 -- start start 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926e9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926e9ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926ea9370 0x55e926eab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e926e042f0 con 0x55e926e9fb40 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e926dabcf0 con 0x55e926e9e5d0 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f87136f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e926e002b0 con 0x55e926ea9370 2024-09-19T05:08:22.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.372+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926e9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926e9e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58672/0 (socket says 172.21.15.64:58672) 2024-09-19T05:08:22.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 -- 172.21.15.64:0/1986777548 learned_addr learned my addr 172.21.15.64:0/1986777548 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:22.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870a7fc640 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926ea9370 0x55e926eab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870affd640 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926e9ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f86f3fff640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f86f40a33b0 0x7f86f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 -- 172.21.15.64:0/1986777548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926ea9370 msgr2=0x55e926eab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926ea9370 0x55e926eab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 -- 172.21.15.64:0/1986777548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 msgr2=0x55e926e9ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926e9ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 -- 172.21.15.64:0/1986777548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e926d903a0 con 0x55e926e9fb40 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f86f3fff640 1 -- 172.21.15.64:0/2470285802 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f87000cf200 con 0x7f870c1036e0 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f86f40a33b0 0x7f86f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.373+0000 7f870b7fe640 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926e9e090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8704035000 tx=0x7f8704009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dde5f2e0d0dac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.374+0000 7f86f17fa640 1 -- 172.21.15.64:0/1986777548 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704042040 con 0x55e926e9fb40 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.374+0000 7f86f17fa640 1 -- 172.21.15.64:0/1986777548 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8704036070 con 0x55e926e9fb40 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.374+0000 7f86f17fa640 1 -- 172.21.15.64:0/1986777548 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004840 con 0x55e926e9fb40 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.374+0000 7f870b7fe640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f86f40a33b0 0x7f86f40a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f870404d000 tx=0x7f870404e000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.374+0000 7f87136f1740 1 -- 172.21.15.64:0/1986777548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 msgr2=0x55e926e9e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926e9e090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8704035000 tx=0x7f8704009990 comp rx=0 tx=0).stop 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 -- 172.21.15.64:0/1986777548 shutdown_connections 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926ea9370 0x55e926eab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926e9ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 --2- 172.21.15.64:0/1986777548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926e9e090 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 -- 172.21.15.64:0/1986777548 >> 172.21.15.64:0/1986777548 conn(0x55e926e67310 msgr2=0x55e926e9ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 -- 172.21.15.64:0/1986777548 shutdown_connections 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 -- 172.21.15.64:0/1986777548 wait complete. 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 -- start start 2024-09-19T05:08:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.375+0000 7f87136f1740 1 Processor -- start 2024-09-19T05:08:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9e5d0 0x55e926ea8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926ea5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926ea9370 0x55e926ea6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e926eac070 con 0x55e926e9fb40 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926ea5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e926dbf060 con 0x55e926ea9370 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e926eac1f0 con 0x55e926e9e5d0 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926ea5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58680/0 (socket says 172.21.15.64:58680) 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 -- 172.21.15.64:0/4240720903 learned_addr learned my addr 172.21.15.64:0/4240720903 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870affd640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9e5d0 0x55e926ea8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f87136f1740 10 client.0 _ll_register_callbacks cb 0x55e926b128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870b7fe640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926ea9370 0x55e926ea6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 -- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9e5d0 msgr2=0x55e926ea8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9e5d0 0x55e926ea8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 -- 172.21.15.64:0/4240720903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926ea9370 msgr2=0x55e926ea6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926ea9370 0x55e926ea6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 -- 172.21.15.64:0/4240720903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e926b753c0 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f870a7fc640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926ea5fb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8700009fd0 tx=0x7f870009b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b169d75af1ce947a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.376+0000 7f86eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f86eeff5640 1 -- 172.21.15.64:0/4240720903 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870009bcf0 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f86eeff5640 1 -- 172.21.15.64:0/4240720903 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f87000ccbd0 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8704036070 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f86eeff5640 1 -- 172.21.15.64:0/4240720903 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f870009b5f0 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 10 client.15867 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e926eb7aa0 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f86eeff5640 1 -- 172.21.15.64:0/4240720903 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8704036070 con 0x55e926e9fb40 2024-09-19T05:08:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f86eeff5640 1 -- 172.21.15.64:0/4240720903 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8700097340 con 0x55e926e9fb40 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 10 client.15867 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 20 client.15867 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926ebb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 4 client.15867 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.377+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e926e11860 con 0x55e926eb8f60 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.378+0000 7f870affd640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926ebb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.378+0000 7f86eeff5640 10 client.15867 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.378+0000 7f870affd640 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926ebb350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9914c71161e1e820 server_cookie=b5899594e1534379 in_seq=0 out_seq=0 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f86eeff5640 1 -- 172.21.15.64:0/4240720903 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e926e11860 con 0x55e926eb8f60 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f86eeff5640 10 client.15867 handle_command_reply: tid=0 2024-09-19T05:08:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 client.15867 shutdown 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 msgr2=0x55e926ea5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926ea5fb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8700009fd0 tx=0x7f870009b3c0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 shutdown_connections 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926ebb350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926ea9370 0x55e926ea6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f86eeff5640 0 client.15867 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926e9fb40 0x55e926ea5fb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 --2- 172.21.15.64:0/4240720903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9e5d0 0x55e926ea8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 >> 172.21.15.64:0/4240720903 conn(0x55e926e67310 msgr2=0x55e926ea0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 shutdown_connections 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 1 -- 172.21.15.64:0/4240720903 wait complete. 2024-09-19T05:08:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.379+0000 7f87136f1740 20 client.15867 trim_cache size 0 max 16384 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 Processor -- start 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 -- start start 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926eb5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ec0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926ec1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1b22b0 con 0x55e926ea9370 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1afb80 con 0x55e926e9e5d0 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f87136f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ad9d0 con 0x55e926e9fb40 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926ec1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926ec1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58688/0 (socket says 172.21.15.64:58688) 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f870b7fe640 1 -- 172.21.15.64:0/1968055843 learned_addr learned my addr 172.21.15.64:0/1968055843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f870a7fc640 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ec0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.396+0000 7f870affd640 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926eb5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870b7fe640 1 -- 172.21.15.64:0/1968055843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 msgr2=0x55e926ec0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870b7fe640 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ec0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870b7fe640 1 -- 172.21.15.64:0/1968055843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 msgr2=0x55e926eb5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870b7fe640 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926eb5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870b7fe640 1 -- 172.21.15.64:0/1968055843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e926e9ed50 con 0x55e926ea9370 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870a7fc640 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ec0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f870b7fe640 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926ec1160 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8704009c20 tx=0x7f870402a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ccb2ea42bd220e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f86f0ff9640 1 -- 172.21.15.64:0/1968055843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704042040 con 0x55e926ea9370 2024-09-19T05:08:22.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f86f0ff9640 1 -- 172.21.15.64:0/1968055843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8704036070 con 0x55e926ea9370 2024-09-19T05:08:22.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.397+0000 7f86f0ff9640 1 -- 172.21.15.64:0/1968055843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87040306e0 con 0x55e926ea9370 2024-09-19T05:08:22.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 -- 172.21.15.64:0/1968055843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 msgr2=0x55e926ec1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926ec1160 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8704009c20 tx=0x7f870402a910 comp rx=0 tx=0).stop 2024-09-19T05:08:22.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 -- 172.21.15.64:0/1968055843 shutdown_connections 2024-09-19T05:08:22.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926ec1160 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ec0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 --2- 172.21.15.64:0/1968055843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926eb5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 -- 172.21.15.64:0/1968055843 >> 172.21.15.64:0/1968055843 conn(0x55e926e67310 msgr2=0x55e926ea1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 -- 172.21.15.64:0/1968055843 shutdown_connections 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 -- 172.21.15.64:0/1968055843 wait complete. 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.398+0000 7f87136f1740 1 -- start start 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 Processor -- start 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926ead160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ead6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926eadbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ae040 con 0x55e926ea9370 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1af4b0 con 0x55e926e9e5d0 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ae710 con 0x55e926e9fb40 2024-09-19T05:08:22.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926ead160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f87136f1740 10 client.0 _ll_register_callbacks cb 0x7f87000971d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926eadbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926ead160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:60592/0 (socket says 172.21.15.64:60592) 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870affd640 1 -- 172.21.15.64:0/2321465820 learned_addr learned my addr 172.21.15.64:0/2321465820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870a7fc640 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ead6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870b7fe640 1 -- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 msgr2=0x55e926ead6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870b7fe640 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ead6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870b7fe640 1 -- 172.21.15.64:0/2321465820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 msgr2=0x55e926ead160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870b7fe640 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926ead160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.399+0000 7f870b7fe640 1 -- 172.21.15.64:0/2321465820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e926f22bf0 con 0x55e926ea9370 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f870b7fe640 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926eadbe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f87040049d0 tx=0x7f8704004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e280ea8b714ca7a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:22.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f86eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f86eeff5640 1 -- 172.21.15.64:0/2321465820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704042040 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f86eeff5640 1 -- 172.21.15.64:0/2321465820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f8704003970 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8704036070 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f86eeff5640 1 -- 172.21.15.64:0/2321465820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8704003b70 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f87136f1740 10 client.15873 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e926f23e50 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.400+0000 7f86eeff5640 1 -- 172.21.15.64:0/2321465820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f870405e030 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f86eeff5640 1 -- 172.21.15.64:0/2321465820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f87040664e0 con 0x55e926ea9370 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f87136f1740 10 client.15873 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f87136f1740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f87136f1740 20 client.15873 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f87136f1740 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926f26080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f87136f1740 4 client.15873 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:22.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8704036070 con 0x55e926eb8f60 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f870affd640 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926f26080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f870affd640 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926f26080 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfe8c1333428577e server_cookie=2bad4abc4d567e71 in_seq=0 out_seq=0 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.401+0000 7f86eeff5640 10 client.15873 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f86eeff5640 1 -- 172.21.15.64:0/2321465820 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8704036070 con 0x55e926eb8f60 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f86eeff5640 10 client.15873 handle_command_reply: tid=0 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 client.15873 shutdown 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 msgr2=0x55e926eadbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926eadbe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f87040049d0 tx=0x7f8704004040 comp rx=0 tx=0).stop 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 shutdown_connections 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55e926eb8f60 0x55e926f26080 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e926ea9370 0x55e926eadbe0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e926e9fb40 0x55e926ead6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 --2- 172.21.15.64:0/2321465820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e926e9e5d0 0x55e926ead160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 >> 172.21.15.64:0/2321465820 conn(0x55e926e67310 msgr2=0x55e926ea0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f86eeff5640 0 client.15873 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 shutdown_connections 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 1 -- 172.21.15.64:0/2321465820 wait complete. 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.402+0000 7f87136f1740 20 client.15873 trim_cache size 0 max 16384 2024-09-19T05:08:22.409 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f86f40a33b0 msgr2=0x7f86f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f86f40a33b0 0x7f86f40a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f870404d000 tx=0x7f870404e000 comp rx=0 tx=0).stop 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 msgr2=0x7f870c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 0x7f870c100ed0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8700006fd0 tx=0x7f870000b8e0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 shutdown_connections 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f86f40a33b0 0x7f86f40a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f870c1082a0 0x7f870c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f870c1036e0 0x7f870c100ed0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 --2- 172.21.15.64:0/2470285802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870c102d30 0x7f870c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:22.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 >> 172.21.15.64:0/2470285802 conn(0x7f870c0fc7d0 msgr2=0x7f870c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:22.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 shutdown_connections 2024-09-19T05:08:22.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:22.403+0000 7f8711318640 1 -- 172.21.15.64:0/2470285802 wait complete. 2024-09-19T05:08:23.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:08:23.400 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:23.808 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:23 smithi064 ceph-mon[30960]: pgmap v77: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:08:23.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 -- 172.21.15.64:0/462923148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc101380 msgr2=0x7ff2fc10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 --2- 172.21.15.64:0/462923148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc101380 0x7ff2fc10dfb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009900 tx=0x7ff2f002a410 comp rx=0 tx=0).stop 2024-09-19T05:08:23.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 -- 172.21.15.64:0/462923148 shutdown_connections 2024-09-19T05:08:23.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 --2- 172.21.15.64:0/462923148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 --2- 172.21.15.64:0/462923148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc101380 0x7ff2fc10dfb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 --2- 172.21.15.64:0/462923148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 0x7ff2fc100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 -- 172.21.15.64:0/462923148 >> 172.21.15.64:0/462923148 conn(0x7ff2fc0fc7d0 msgr2=0x7ff2fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:23.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 -- 172.21.15.64:0/462923148 shutdown_connections 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.903+0000 7ff302558640 1 -- 172.21.15.64:0/462923148 wait complete. 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.904+0000 7ff302558640 1 Processor -- start 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.904+0000 7ff302558640 1 -- start start 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff302558640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 0x7ff2fc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff302558640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc101380 0x7ff2fc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff302558640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff302558640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1140f0 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff302558640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc113f70 con 0x7ff2fc1009d0 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff302558640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc114270 con 0x7ff2fc101380 2024-09-19T05:08:23.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58724/0 (socket says 172.21.15.64:58724) 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 -- 172.21.15.64:0/3215859608 learned_addr learned my addr 172.21.15.64:0/3215859608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 0x7ff2fc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc101380 0x7ff2fc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 -- 172.21.15.64:0/3215859608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc101380 msgr2=0x7ff2fc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc101380 0x7ff2fc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 -- 172.21.15.64:0/3215859608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 msgr2=0x7ff2fc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 0x7ff2fc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 -- 172.21.15.64:0/3215859608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc19f5b0 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 0x7ff2fc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc101380 0x7ff2fc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff300af5640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc19eed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400ccb0 tx=0x7ff2f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.905+0000 7ff2f97fa640 1 -- 172.21.15.64:0/3215859608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f450 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.906+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc19f8a0 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.906+0000 7ff2f97fa640 1 -- 172.21.15.64:0/3215859608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f400faa0 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.906+0000 7ff2f97fa640 1 -- 172.21.15.64:0/3215859608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4018b10 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.906+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1a7180 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.908+0000 7ff2f97fa640 1 -- 172.21.15.64:0/3215859608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff2f400f5f0 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 Processor -- start 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff2f97fa640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff2e40a3430 0x7ff2e40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 -- start start 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff2f97fa640 1 -- 172.21.15.64:0/3215859608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff2f40d04b0 con 0x7ff2fc10e4f0 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff2e40a3430 0x7ff2e40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c211cb12f0 con 0x55c211d56370 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c211c58cf0 con 0x55c211d4cb40 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff304931740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c211cad2b0 con 0x55c211d4b5d0 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff300af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.909+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d4b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:60628/0 (socket says 172.21.15.64:60628) 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2fbfff640 1 -- 172.21.15.64:0/1053486140 learned_addr learned my addr 172.21.15.64:0/1053486140 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58728/0 (socket says 172.21.15.64:58728) 2024-09-19T05:08:23.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff2e40a3430 0x7ff2e40a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0031440 tx=0x7ff2f0005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 -- 172.21.15.64:0/1053486140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 msgr2=0x55c211d4ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d4ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 -- 172.21.15.64:0/1053486140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 msgr2=0x55c211d4b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d4b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 -- 172.21.15.64:0/1053486140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c211c3d3a0 con 0x55c211d56370 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff300af5640 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d58760 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40cdc00 tx=0x7ff2f409c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37023f6e0d4e1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2e27fc640 1 -- 172.21.15.64:0/1053486140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40a0b40 con 0x55c211d56370 2024-09-19T05:08:23.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2e27fc640 1 -- 172.21.15.64:0/1053486140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f40a5400 con 0x55c211d56370 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.910+0000 7ff2e27fc640 1 -- 172.21.15.64:0/1053486140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40a4f70 con 0x55c211d56370 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 -- 172.21.15.64:0/1053486140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 msgr2=0x55c211d58760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d58760 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40cdc00 tx=0x7ff2f409c1b0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 -- 172.21.15.64:0/1053486140 shutdown_connections 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d58760 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d4ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 --2- 172.21.15.64:0/1053486140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 -- 172.21.15.64:0/1053486140 >> 172.21.15.64:0/1053486140 conn(0x55c211d14310 msgr2=0x55c211d47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.911+0000 7ff304931740 1 -- 172.21.15.64:0/1053486140 shutdown_connections 2024-09-19T05:08:23.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.912+0000 7ff304931740 1 -- 172.21.15.64:0/1053486140 wait complete. 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.912+0000 7ff304931740 1 -- start start 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.912+0000 7ff304931740 1 Processor -- start 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.912+0000 7ff304931740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d53100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.912+0000 7ff304931740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d53640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.912+0000 7ff304931740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff304931740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c211d59070 con 0x55c211d56370 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff304931740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c211c6c060 con 0x55c211d4cb40 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff304931740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c211d591f0 con 0x55c211d4b5d0 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58742/0 (socket says 172.21.15.64:58742) 2024-09-19T05:08:23.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 -- 172.21.15.64:0/4138292268 learned_addr learned my addr 172.21.15.64:0/4138292268 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff304931740 10 client.0 _ll_register_callbacks cb 0x55c2119bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d53100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d53640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 -- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 msgr2=0x55c211d53100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d53100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 -- 172.21.15.64:0/4138292268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 msgr2=0x55c211d53640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d53640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 -- 172.21.15.64:0/4138292268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c211a223c0 con 0x55c211d56370 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff300af5640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d53b80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40cdcb0 tx=0x7ff2f40cde80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f046a82d9d1c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff2db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:23.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.913+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4138292268 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40b9070 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4138292268 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff2f40a4f70 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f40a5400 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4138292268 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2f40a0b40 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 10 client.15885 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c211d64c00 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4138292268 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f40a5400 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4138292268 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2f40984e0 con 0x55c211d56370 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 10 client.15885 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:08:23.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 20 client.15885 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211d65f50 0x55c211d68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 4 client.15885 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.914+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c211cbe860 con 0x55c211d65f50 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.915+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211d65f50 0x55c211d68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.915+0000 7ff2db7fe640 10 client.15885 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.915+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211d65f50 0x55c211d68340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc575d27c5ef8602 server_cookie=33e891b383c9b0ee in_seq=0 out_seq=0 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4138292268 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c211cbe860 con 0x55c211d65f50 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff2db7fe640 10 client.15885 handle_command_reply: tid=0 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 client.15885 shutdown 2024-09-19T05:08:23.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 msgr2=0x55c211d53b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d53b80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40cdcb0 tx=0x7ff2f40cde80 comp rx=0 tx=0).stop 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 shutdown_connections 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211d65f50 0x55c211d68340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d56370 0x55c211d53b80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d4cb40 0x55c211d53640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 --2- 172.21.15.64:0/4138292268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4b5d0 0x55c211d53100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff2db7fe640 0 client.15885 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 >> 172.21.15.64:0/4138292268 conn(0x55c211d14310 msgr2=0x55c211d4d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 shutdown_connections 2024-09-19T05:08:23.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 1 -- 172.21.15.64:0/4138292268 wait complete. 2024-09-19T05:08:23.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.916+0000 7ff304931740 20 client.15885 trim_cache size 0 max 16384 2024-09-19T05:08:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.932+0000 7ff304931740 1 Processor -- start 2024-09-19T05:08:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.932+0000 7ff304931740 1 -- start start 2024-09-19T05:08:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff304931740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff304931740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff304931740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff304931740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b2160 con 0x55c211d4b5d0 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff304931740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1afa30 con 0x55c211d56370 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff304931740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ad880 con 0x55c211d4cb40 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58748/0 (socket says 172.21.15.64:58748) 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 -- 172.21.15.64:0/3171690553 learned_addr learned my addr 172.21.15.64:0/3171690553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff300af5640 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 -- 172.21.15.64:0/3171690553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 msgr2=0x55c211d539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 -- 172.21.15.64:0/3171690553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 msgr2=0x55c211d57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 -- 172.21.15.64:0/3171690553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c211d47510 con 0x55c211d4b5d0 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff300af5640 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.933+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d519d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55c211d591c0 tx=0x7ff2ec005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361bd81fe3a92b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.934+0000 7ff2e1ffb640 1 -- 172.21.15.64:0/3171690553 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004030 con 0x55c211d4b5d0 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.934+0000 7ff2e1ffb640 1 -- 172.21.15.64:0/3171690553 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2ec0041d0 con 0x55c211d4b5d0 2024-09-19T05:08:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.934+0000 7ff2e1ffb640 1 -- 172.21.15.64:0/3171690553 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec01c750 con 0x55c211d4b5d0 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.934+0000 7ff304931740 1 -- 172.21.15.64:0/3171690553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 msgr2=0x55c211d519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.934+0000 7ff304931740 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d519d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55c211d591c0 tx=0x7ff2ec005850 comp rx=0 tx=0).stop 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.934+0000 7ff304931740 1 -- 172.21.15.64:0/3171690553 shutdown_connections 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 --2- 172.21.15.64:0/3171690553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d519d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 -- 172.21.15.64:0/3171690553 >> 172.21.15.64:0/3171690553 conn(0x55c211d14310 msgr2=0x55c211d55180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 -- 172.21.15.64:0/3171690553 shutdown_connections 2024-09-19T05:08:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 -- 172.21.15.64:0/3171690553 wait complete. 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 -- start start 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.935+0000 7ff304931740 1 Processor -- start 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d67330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d67870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d67db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1adef0 con 0x55c211d4b5d0 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1af360 con 0x55c211d56370 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ae5c0 con 0x55c211d4cb40 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d67330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff304931740 10 client.0 _ll_register_callbacks cb 0x7ff2ec00b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:23.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d67330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58760/0 (socket says 172.21.15.64:58760) 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 -- 172.21.15.64:0/4108717883 learned_addr learned my addr 172.21.15.64:0/4108717883 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d67870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff300af5640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d67db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 -- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 msgr2=0x55c211d67870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d67870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 -- 172.21.15.64:0/4108717883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 msgr2=0x55c211d67db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d67db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff2fbfff640 1 -- 172.21.15.64:0/4108717883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c211dcfad0 con 0x55c211d4b5d0 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.936+0000 7ff300af5640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d67db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff2fbfff640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d67330 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00bea0 tx=0x7ff2ec004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1789bec44a700c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:23.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff2db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4108717883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0035f0 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4108717883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff2ec003790 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ec0041d0 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4108717883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec003990 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff304931740 10 client.15891 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c211dd0dd0 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.937+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4108717883 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2ec0041d0 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4108717883 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2ec010490 con 0x55c211d4b5d0 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff304931740 10 client.15891 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:23.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff304931740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff304931740 20 client.15891 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff304931740 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211dd1480 0x55c211dd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff304931740 4 client.15891 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c211cb8d70 con 0x55c211dd1480 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211dd1480 0x55c211dd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff2db7fe640 10 client.15891 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.938+0000 7ff2fb7fe640 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211dd1480 0x55c211dd3870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db38c21169ba7eea server_cookie=c14931dad3ae8f69 in_seq=0 out_seq=0 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff2db7fe640 1 -- 172.21.15.64:0/4108717883 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c211cb8d70 con 0x55c211dd1480 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff2db7fe640 10 client.15891 handle_command_reply: tid=0 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 client.15891 shutdown 2024-09-19T05:08:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 msgr2=0x55c211d67330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d67330 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00bea0 tx=0x7ff2ec004160 comp rx=0 tx=0).stop 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 shutdown_connections 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c211d56370 0x55c211d67db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c211d4cb40 0x55c211d67870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c211dd1480 0x55c211dd3870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 --2- 172.21.15.64:0/4108717883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c211d4b5d0 0x55c211d67330 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 >> 172.21.15.64:0/4108717883 conn(0x55c211d14310 msgr2=0x55c211d464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff2db7fe640 0 client.15891 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 shutdown_connections 2024-09-19T05:08:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 1 -- 172.21.15.64:0/4108717883 wait complete. 2024-09-19T05:08:23.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.939+0000 7ff304931740 20 client.15891 trim_cache size 0 max 16384 2024-09-19T05:08:23.948 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff2e40a3430 msgr2=0x7ff2e40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff2e40a3430 0x7ff2e40a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0031440 tx=0x7ff2f0005a20 comp rx=0 tx=0).stop 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 msgr2=0x7ff2fc19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc19eed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400ccb0 tx=0x7ff2f400a620 comp rx=0 tx=0).stop 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 shutdown_connections 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff2e40a3430 0x7ff2e40a58f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc10e4f0 0x7ff2fc19eed0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2fc101380 0x7ff2fc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 --2- 172.21.15.64:0/3215859608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2fc1009d0 0x7ff2fc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 >> 172.21.15.64:0/3215859608 conn(0x7ff2fc0fc7d0 msgr2=0x7ff2fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:23.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 shutdown_connections 2024-09-19T05:08:23.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:23.940+0000 7ff302558640 1 -- 172.21.15.64:0/3215859608 wait complete. 2024-09-19T05:08:23.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:23 smithi103 ceph-mon[34446]: pgmap v77: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:08:24.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:23 smithi174 ceph-mon[34488]: pgmap v77: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:08:24.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:08:25.010 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:25.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 -- 172.21.15.64:0/3299452977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8980101360 msgr2=0x7f898010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 --2- 172.21.15.64:0/3299452977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8980101360 0x7f898010df90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8970009900 tx=0x7f897002a410 comp rx=0 tx=0).stop 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 -- 172.21.15.64:0/3299452977 shutdown_connections 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 --2- 172.21.15.64:0/3299452977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898010e4d0 0x7f89801108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 --2- 172.21.15.64:0/3299452977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8980101360 0x7f898010df90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 --2- 172.21.15.64:0/3299452977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89801009b0 0x7f8980100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 -- 172.21.15.64:0/3299452977 >> 172.21.15.64:0/3299452977 conn(0x7f89800fc7d0 msgr2=0x7f89800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 -- 172.21.15.64:0/3299452977 shutdown_connections 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.522+0000 7f8987fc7640 1 -- 172.21.15.64:0/3299452977 wait complete. 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 Processor -- start 2024-09-19T05:08:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 -- start start 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89801009b0 0x7f898019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8980101360 0x7f898019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 0x7f898019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8980114130 con 0x7f898010e4d0 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8980113fb0 con 0x7f8980101360 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8987fc7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89801142b0 con 0x7f89801009b0 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8986564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 0x7f898019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8986564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 0x7f898019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47492/0 (socket says 172.21.15.64:47492) 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8986564640 1 -- 172.21.15.64:0/948732541 learned_addr learned my addr 172.21.15.64:0/948732541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.523+0000 7f8985562640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8980101360 0x7f898019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8985d63640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89801009b0 0x7f898019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8986564640 1 -- 172.21.15.64:0/948732541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89801009b0 msgr2=0x7f898019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8986564640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89801009b0 0x7f898019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8986564640 1 -- 172.21.15.64:0/948732541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8980101360 msgr2=0x7f898019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8986564640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8980101360 0x7f898019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8986564640 1 -- 172.21.15.64:0/948732541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898019f6c0 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8986564640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 0x7f898019efe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f897800eaa0 tx=0x7f897800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8976ffd640 1 -- 172.21.15.64:0/948732541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978004d20 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8976ffd640 1 -- 172.21.15.64:0/948732541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8978005020 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898019f9b0 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801a7290 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.524+0000 7f8976ffd640 1 -- 172.21.15.64:0/948732541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89780116b0 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f8976ffd640 1 -- 172.21.15.64:0/948732541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f89780051d0 con 0x7f898010e4d0 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 Processor -- start 2024-09-19T05:08:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 -- start start 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4cb40 0x55d4cee4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee56370 0x55d4cee58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4cedb12f0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d4ced58cf0 con 0x55d4cee56370 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.527+0000 7f898a3a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4cedad2b0 con 0x55d4cee4cb40 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee4ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47500/0 (socket says 172.21.15.64:47500) 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 -- 172.21.15.64:0/444101312 learned_addr learned my addr 172.21.15.64:0/444101312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985562640 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee56370 0x55d4cee58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 -- 172.21.15.64:0/444101312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4cb40 msgr2=0x55d4cee4b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4cb40 0x55d4cee4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 -- 172.21.15.64:0/444101312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee56370 msgr2=0x55d4cee58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee56370 0x55d4cee58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 -- 172.21.15.64:0/444101312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ced3d3a0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8976ffd640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f896c0a33b0 0x7f896c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985d63640 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee4ba10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f897c009a70 tx=0x7f897c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd298ec1722b32a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8985562640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f896c0a33b0 0x7f896c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f8976ffd640 1 -- 172.21.15.64:0/948732541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f89780cff00 con 0x7f898010e4d0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f896bfff640 1 -- 172.21.15.64:0/444101312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c017070 con 0x55d4cee4b5d0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.528+0000 7f896bfff640 1 -- 172.21.15.64:0/444101312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c013470 con 0x55d4cee4b5d0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f896bfff640 1 -- 172.21.15.64:0/444101312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c012630 con 0x55d4cee4b5d0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f8985562640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f896c0a33b0 0x7f896c0a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8970031440 tx=0x7f8970005850 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 -- 172.21.15.64:0/444101312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 msgr2=0x55d4cee4ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee4ba10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f897c009a70 tx=0x7f897c009f30 comp rx=0 tx=0).stop 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 -- 172.21.15.64:0/444101312 shutdown_connections 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee56370 0x55d4cee58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee4ba10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 --2- 172.21.15.64:0/444101312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4cb40 0x55d4cee4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.529+0000 7f898a3a0740 1 -- 172.21.15.64:0/444101312 >> 172.21.15.64:0/444101312 conn(0x55d4cee14310 msgr2=0x55d4cee47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.530+0000 7f898a3a0740 1 -- 172.21.15.64:0/444101312 shutdown_connections 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.530+0000 7f898a3a0740 1 -- 172.21.15.64:0/444101312 wait complete. 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.530+0000 7f898a3a0740 1 -- start start 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.530+0000 7f898a3a0740 1 Processor -- start 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.530+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee55d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f898a3a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4cee59070 con 0x55d4cee4b5d0 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f898a3a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d4ced6c060 con 0x55d4cee4cb40 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f898a3a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4cee591f0 con 0x55d4cee56370 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee55d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f898a3a0740 10 client.0 _ll_register_callbacks cb 0x55d4ceabf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee55d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47502/0 (socket says 172.21.15.64:47502) 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 -- 172.21.15.64:0/2545609199 learned_addr learned my addr 172.21.15.64:0/2545609199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985562640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 -- 172.21.15.64:0/2545609199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 msgr2=0x55d4cee535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 -- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 msgr2=0x55d4cee52ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee52ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 -- 172.21.15.64:0/2545609199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ceb223c0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985562640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f8985d63640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee55d40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f897c004810 tx=0x7f897c006e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d5a1c12b8a95c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f89697fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.531+0000 7f89697fa640 1 -- 172.21.15.64:0/2545609199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c017050 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f89697fa640 1 -- 172.21.15.64:0/2545609199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f897c01c650 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897c013470 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f89697fa640 1 -- 172.21.15.64:0/2545609199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f897c013970 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f898a3a0740 10 client.15903 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4cee64ae0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f89697fa640 1 -- 172.21.15.64:0/2545609199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c013470 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.532+0000 7f89697fa640 1 -- 172.21.15.64:0/2545609199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f897c02f950 con 0x55d4cee4b5d0 2024-09-19T05:08:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f898a3a0740 10 client.15903 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f898a3a0740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f898a3a0740 20 client.15903 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f898a3a0740 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4cee65e30 0x55d4cee68220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f898a3a0740 4 client.15903 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4cedbe860 con 0x55d4cee65e30 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f8985562640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4cee65e30 0x55d4cee68220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f89697fa640 10 client.15903 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.533+0000 7f8985562640 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4cee65e30 0x55d4cee68220 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1ed3e49d57ccab server_cookie=e93b9ba26be2e5d9 in_seq=0 out_seq=0 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f89697fa640 1 -- 172.21.15.64:0/2545609199 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4cedbe860 con 0x55d4cee65e30 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f89697fa640 10 client.15903 handle_command_reply: tid=0 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 client.15903 shutdown 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 msgr2=0x55d4cee55d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee55d40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f897c004810 tx=0x7f897c006e80 comp rx=0 tx=0).stop 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 shutdown_connections 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4cee65e30 0x55d4cee68220 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.534+0000 7f898a3a0740 1 --2- 172.21.15.64:0/2545609199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4b5d0 0x55d4cee55d40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.535+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 >> 172.21.15.64:0/2545609199 conn(0x55d4cee14310 msgr2=0x55d4cee4d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.535+0000 7f89697fa640 0 client.15903 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.535+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 shutdown_connections 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.535+0000 7f898a3a0740 1 -- 172.21.15.64:0/2545609199 wait complete. 2024-09-19T05:08:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.535+0000 7f898a3a0740 20 client.15903 trim_cache size 0 max 16384 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 Processor -- start 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 -- start start 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4b5d0 0x55d4cee62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee47700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee56370 0x55d4cee47c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89801b2420 con 0x55d4cee56370 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89801afcf0 con 0x55d4cee4cb40 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f898a3a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89801adb40 con 0x55d4cee4b5d0 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f8986564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee56370 0x55d4cee47c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f8986564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee56370 0x55d4cee47c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47518/0 (socket says 172.21.15.64:47518) 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f8986564640 1 -- 172.21.15.64:0/1511395130 learned_addr learned my addr 172.21.15.64:0/1511395130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f8985562640 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.549+0000 7f8985d63640 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4b5d0 0x55d4cee62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f8985562640 1 -- 172.21.15.64:0/1511395130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4b5d0 msgr2=0x55d4cee62310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f8985562640 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4b5d0 0x55d4cee62310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f8985562640 1 -- 172.21.15.64:0/1511395130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee56370 msgr2=0x55d4cee47c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f8985562640 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee56370 0x55d4cee47c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f8985562640 1 -- 172.21.15.64:0/1511395130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4cee53950 con 0x55d4cee4cb40 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f8985562640 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee47700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8970009230 tx=0x7f897003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d82c6e633bba54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:25.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f896b7fe640 1 -- 172.21.15.64:0/1511395130 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970056070 con 0x55d4cee4cb40 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f896b7fe640 1 -- 172.21.15.64:0/1511395130 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89700035f0 con 0x55d4cee4cb40 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.550+0000 7f896b7fe640 1 -- 172.21.15.64:0/1511395130 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970035040 con 0x55d4cee4cb40 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 -- 172.21.15.64:0/1511395130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 msgr2=0x55d4cee47700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee47700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8970009230 tx=0x7f897003f000 comp rx=0 tx=0).stop 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 -- 172.21.15.64:0/1511395130 shutdown_connections 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee56370 0x55d4cee47c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4cb40 0x55d4cee47700 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 --2- 172.21.15.64:0/1511395130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee4b5d0 0x55d4cee62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 -- 172.21.15.64:0/1511395130 >> 172.21.15.64:0/1511395130 conn(0x55d4cee14310 msgr2=0x55d4cee518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 -- 172.21.15.64:0/1511395130 shutdown_connections 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.551+0000 7f898a3a0740 1 -- 172.21.15.64:0/1511395130 wait complete. 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 -- start start 2024-09-19T05:08:25.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 Processor -- start 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4b5d0 0x55d4cee50c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 0x55d4cee5e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee5ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89801ae1b0 con 0x55d4cee4cb40 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89801af620 con 0x55d4cee4b5d0 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89801ae880 con 0x55d4cee56370 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f898a3a0740 10 client.0 _ll_register_callbacks cb 0x7f897c0098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f8985562640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 0x55d4cee5e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f8985562640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 0x55d4cee5e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47528/0 (socket says 172.21.15.64:47528) 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f8985d63640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4b5d0 0x55d4cee50c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.552+0000 7f8985562640 1 -- 172.21.15.64:0/836718736 learned_addr learned my addr 172.21.15.64:0/836718736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8986564640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee5ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985d63640 1 -- 172.21.15.64:0/836718736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 msgr2=0x55d4cee5ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985d63640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee5ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985d63640 1 -- 172.21.15.64:0/836718736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 msgr2=0x55d4cee5e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985d63640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 0x55d4cee5e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985d63640 1 -- 172.21.15.64:0/836718736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4cee4f260 con 0x55d4cee4b5d0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985562640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 0x55d4cee5e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f8985d63640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4b5d0 0x55d4cee50c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f897c0099c0 tx=0x7f897c02fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42e5d77af3de0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f89697fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f89697fa640 1 -- 172.21.15.64:0/836718736 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c034030 con 0x55d4cee4b5d0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89700035f0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.553+0000 7f89697fa640 1 -- 172.21.15.64:0/836718736 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f897c013bf0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f89697fa640 1 -- 172.21.15.64:0/836718736 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f897c017040 con 0x55d4cee4b5d0 2024-09-19T05:08:25.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 10 client.25592 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4ceecfab0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f89697fa640 1 -- 172.21.15.64:0/836718736 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89700035f0 con 0x55d4cee4b5d0 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f89697fa640 1 -- 172.21.15.64:0/836718736 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f897c01c450 con 0x55d4cee4b5d0 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 10 client.25592 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 20 client.25592 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4ceed0160 0x55d4ceed2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 4 client.25592 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4cedb8d70 con 0x55d4ceed0160 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.554+0000 7f8985562640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4ceed0160 0x55d4ceed2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f8985562640 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4ceed0160 0x55d4ceed2580 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3e6c50fb7a1738 server_cookie=756f344fd9ebcfd4 in_seq=0 out_seq=0 2024-09-19T05:08:25.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f89697fa640 10 client.25592 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f89697fa640 1 -- 172.21.15.64:0/836718736 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4cedb8d70 con 0x55d4ceed0160 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f89697fa640 10 client.25592 handle_command_reply: tid=0 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 client.25592 shutdown 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4b5d0 msgr2=0x55d4cee50c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4b5d0 0x55d4cee50c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f897c0099c0 tx=0x7f897c02fc30 comp rx=0 tx=0).stop 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 shutdown_connections 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4cee56370 0x55d4cee5ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d4ceed0160 0x55d4ceed2580 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f898a3a0740 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4cee4cb40 0x55d4cee5e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.555+0000 7f89697fa640 0 client.25592 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:25.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f898a3a0740 1 --2- 172.21.15.64:0/836718736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4cee4b5d0 0x55d4cee50c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 >> 172.21.15.64:0/836718736 conn(0x55d4cee14310 msgr2=0x55d4cee51840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:25.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 shutdown_connections 2024-09-19T05:08:25.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f898a3a0740 1 -- 172.21.15.64:0/836718736 wait complete. 2024-09-19T05:08:25.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f898a3a0740 20 client.25592 trim_cache size 0 max 16384 2024-09-19T05:08:25.567 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f896c0a33b0 msgr2=0x7f896c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f896c0a33b0 0x7f896c0a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8970031440 tx=0x7f8970005850 comp rx=0 tx=0).stop 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 msgr2=0x7f898019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 0x7f898019efe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f897800eaa0 tx=0x7f897800eea0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 shutdown_connections 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f896c0a33b0 0x7f896c0a5870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f898010e4d0 0x7f898019efe0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8980101360 0x7f898019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 --2- 172.21.15.64:0/948732541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89801009b0 0x7f898019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:25.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.556+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 >> 172.21.15.64:0/948732541 conn(0x7f89800fc7d0 msgr2=0x7f89800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:25.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.557+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 shutdown_connections 2024-09-19T05:08:25.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:25.557+0000 7f8987fc7640 1 -- 172.21.15.64:0/948732541 wait complete. 2024-09-19T05:08:25.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:25 smithi064 ceph-mon[30960]: pgmap v78: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:08:25.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:25 smithi103 ceph-mon[34446]: pgmap v78: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:08:26.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:25 smithi174 ceph-mon[34488]: pgmap v78: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:08:27.390 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:08:27.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:08:27.679 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:27.803 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:27 smithi064 ceph-mon[30960]: pgmap v79: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:08:27.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:27 smithi103 ceph-mon[34446]: pgmap v79: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:08:28.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:27 smithi174 ceph-mon[34488]: pgmap v79: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:08:28.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 -- 172.21.15.64:0/2660551645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 msgr2=0x7fc94810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 --2- 172.21.15.64:0/2660551645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94810ec40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc940009cd0 tx=0x7fc94002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 -- 172.21.15.64:0/2660551645 shutdown_connections 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 --2- 172.21.15.64:0/2660551645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94810ec40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 --2- 172.21.15.64:0/2660551645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9481036e0 0x7fc948107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 --2- 172.21.15.64:0/2660551645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc948102d30 0x7fc948103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 -- 172.21.15.64:0/2660551645 >> 172.21.15.64:0/2660551645 conn(0x7fc9480fc7d0 msgr2=0x7fc9480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 -- 172.21.15.64:0/2660551645 shutdown_connections 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.187+0000 7fc94eb27640 1 -- 172.21.15.64:0/2660551645 wait complete. 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 Processor -- start 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 -- start start 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948102d30 0x7fc9481984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9481036e0 0x7fc948198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc948111f50 con 0x7fc9481082a0 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc948111dd0 con 0x7fc948102d30 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94d0c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94d0c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47552/0 (socket says 172.21.15.64:47552) 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94d0c4640 1 -- 172.21.15.64:0/1294225764 learned_addr learned my addr 172.21.15.64:0/1294225764 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481120d0 con 0x7fc9481036e0 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.188+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948102d30 0x7fc9481984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc947fff640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9481036e0 0x7fc948198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94d0c4640 1 -- 172.21.15.64:0/1294225764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9481036e0 msgr2=0x7fc948198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94d0c4640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9481036e0 0x7fc948198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94d0c4640 1 -- 172.21.15.64:0/1294225764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948102d30 msgr2=0x7fc9481984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94d0c4640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948102d30 0x7fc9481984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94d0c4640 1 -- 172.21.15.64:0/1294225764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94819d4a0 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94d0c4640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94819cdc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc940005cc0 tx=0x7fc940003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc945ffb640 1 -- 172.21.15.64:0/1294225764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940004450 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94819d730 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc945ffb640 1 -- 172.21.15.64:0/1294225764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc940031c90 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.189+0000 7fc945ffb640 1 -- 172.21.15.64:0/1294225764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94003b400 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.190+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9481a5070 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.192+0000 7fc950f00740 1 Processor -- start 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.192+0000 7fc945ffb640 1 -- 172.21.15.64:0/1294225764 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc940004680 con 0x7fc9481082a0 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.192+0000 7fc950f00740 1 -- start start 2024-09-19T05:08:28.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc950f00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc950f00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d900800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc950f00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d903130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc950f00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56465d85b280 con 0x56465d900d40 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc950f00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56465d802b10 con 0x56465d8f6480 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc950f00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56465d857240 con 0x56465d8f5b60 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d903130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d903130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47558/0 (socket says 172.21.15.64:47558) 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 -- 172.21.15.64:0/3223648881 learned_addr learned my addr 172.21.15.64:0/3223648881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d900800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc945ffb640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9380a3430 0x7fc9380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 -- 172.21.15.64:0/3223648881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 msgr2=0x56465d8f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 -- 172.21.15.64:0/3223648881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 msgr2=0x56465d900800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.193+0000 7fc947fff640 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d900800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc947fff640 1 -- 172.21.15.64:0/3223648881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465d80f1f0 con 0x56465d900d40 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d900800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc94d0c4640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9380a3430 0x7fc9380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc947fff640 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d903130 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc9480697d0 tx=0x7fc9340127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8b382e0cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc945ffb640 1 -- 172.21.15.64:0/1294225764 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fc94002e030 con 0x7fc9481082a0 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc94d0c4640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9380a3430 0x7fc9380a58f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9400b9530 tx=0x7fc9400c1040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc932ffd640 1 -- 172.21.15.64:0/3223648881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934004020 con 0x56465d900d40 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc932ffd640 1 -- 172.21.15.64:0/3223648881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9340054d0 con 0x56465d900d40 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.194+0000 7fc932ffd640 1 -- 172.21.15.64:0/3223648881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934004d40 con 0x56465d900d40 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 -- 172.21.15.64:0/3223648881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 msgr2=0x56465d903130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d903130 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc9480697d0 tx=0x7fc9340127c0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 -- 172.21.15.64:0/3223648881 shutdown_connections 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d903130 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d900800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 --2- 172.21.15.64:0/3223648881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 -- 172.21.15.64:0/3223648881 >> 172.21.15.64:0/3223648881 conn(0x56465d8f20f0 msgr2=0x56465d8f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 -- 172.21.15.64:0/3223648881 shutdown_connections 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 -- 172.21.15.64:0/3223648881 wait complete. 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.195+0000 7fc950f00740 1 -- start start 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 Processor -- start 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d8fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d8fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d900d40 0x56465d8fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56465d903960 con 0x56465d8f5b60 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56465d815f10 con 0x56465d8f6480 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56465d903ae0 con 0x56465d900d40 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d8fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc950f00740 10 client.0 _ll_register_callbacks cb 0x56465d7dea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d8fdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47562/0 (socket says 172.21.15.64:47562) 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc947fff640 1 -- 172.21.15.64:0/2897475227 learned_addr learned my addr 172.21.15.64:0/2897475227 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc947fff640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d900d40 0x56465d8fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.196+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d8fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc947fff640 1 -- 172.21.15.64:0/2897475227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d900d40 msgr2=0x56465d8fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc947fff640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d900d40 0x56465d8fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc947fff640 1 -- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 msgr2=0x56465d8fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc947fff640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d8fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc947fff640 1 -- 172.21.15.64:0/2897475227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465d5cc6a0 con 0x56465d8f5b60 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc947fff640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d8fdc90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc934015040 tx=0x7fc934005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e377f50824c7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc927fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc927fff640 1 -- 172.21.15.64:0/2897475227 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934004cc0 con 0x56465d8f5b60 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9340054d0 con 0x56465d8f5b60 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc927fff640 1 -- 172.21.15.64:0/2897475227 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fc934010450 con 0x56465d8f5b60 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc927fff640 1 -- 172.21.15.64:0/2897475227 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc934004e60 con 0x56465d8f5b60 2024-09-19T05:08:28.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc950f00740 10 client.15921 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56465d9139f0 con 0x56465d8f5b60 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.197+0000 7fc927fff640 1 -- 172.21.15.64:0/2897475227 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9340054d0 con 0x56465d8f5b60 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc927fff640 1 -- 172.21.15.64:0/2897475227 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc934019070 con 0x56465d8f5b60 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc950f00740 10 client.15921 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc950f00740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc950f00740 20 client.15921 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc950f00740 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d914ee0 0x56465d9172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc950f00740 4 client.15921 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56465d8687f0 con 0x56465d914ee0 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.198+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d914ee0 0x56465d9172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.199+0000 7fc927fff640 10 client.15921 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:28.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.199+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d914ee0 0x56465d9172d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46aa5d939b68d9 server_cookie=8d3abc1ae13e8d37 in_seq=0 out_seq=0 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.199+0000 7fc927fff640 1 -- 172.21.15.64:0/2897475227 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56465d8687f0 con 0x56465d914ee0 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.199+0000 7fc927fff640 10 client.15921 handle_command_reply: tid=0 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.199+0000 7fc950f00740 1 client.15921 shutdown 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 msgr2=0x56465d8fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d8fdc90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc934015040 tx=0x7fc934005fb0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 shutdown_connections 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d900d40 0x56465d8fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d8fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d914ee0 0x56465d9172d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 --2- 172.21.15.64:0/2897475227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d8fdc90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 >> 172.21.15.64:0/2897475227 conn(0x56465d8f20f0 msgr2=0x56465d8f7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc927fff640 0 client.15921 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:28.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 shutdown_connections 2024-09-19T05:08:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 1 -- 172.21.15.64:0/2897475227 wait complete. 2024-09-19T05:08:28.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.200+0000 7fc950f00740 20 client.15921 trim_cache size 0 max 16384 2024-09-19T05:08:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.221+0000 7fc950f00740 1 Processor -- start 2024-09-19T05:08:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.221+0000 7fc950f00740 1 -- start start 2024-09-19T05:08:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.221+0000 7fc950f00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.221+0000 7fc950f00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d91c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc950f00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d91ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc950f00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481adec0 con 0x56465d900d40 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc950f00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481ab790 con 0x56465d8f6480 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc950f00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481a95e0 con 0x56465d8f5b60 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d91ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d91ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47576/0 (socket says 172.21.15.64:47576) 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 -- 172.21.15.64:0/2876611959 learned_addr learned my addr 172.21.15.64:0/2876611959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d91c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 -- 172.21.15.64:0/2876611959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 msgr2=0x56465d8ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 -- 172.21.15.64:0/2876611959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 msgr2=0x56465d91c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d91c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 -- 172.21.15.64:0/2876611959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465d900560 con 0x56465d900d40 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc947fff640 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d91ce40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc934012ea0 tx=0x7fc9340127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202ca464a50796da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.222+0000 7fc9327fc640 1 -- 172.21.15.64:0/2876611959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934005660 con 0x56465d900d40 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc9327fc640 1 -- 172.21.15.64:0/2876611959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc93401e070 con 0x56465d900d40 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc9327fc640 1 -- 172.21.15.64:0/2876611959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9340035f0 con 0x56465d900d40 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 -- 172.21.15.64:0/2876611959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 msgr2=0x56465d91ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d91ce40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc934012ea0 tx=0x7fc9340127c0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 -- 172.21.15.64:0/2876611959 shutdown_connections 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d900d40 0x56465d91ce40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d8f6480 0x56465d91c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 --2- 172.21.15.64:0/2876611959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f5b60 0x56465d8ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.223+0000 7fc950f00740 1 -- 172.21.15.64:0/2876611959 >> 172.21.15.64:0/2876611959 conn(0x56465d8f20f0 msgr2=0x56465d8ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:28.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.224+0000 7fc950f00740 1 -- 172.21.15.64:0/2876611959 shutdown_connections 2024-09-19T05:08:28.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.224+0000 7fc950f00740 1 -- 172.21.15.64:0/2876611959 wait complete. 2024-09-19T05:08:28.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.224+0000 7fc950f00740 1 -- start start 2024-09-19T05:08:28.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.224+0000 7fc950f00740 1 Processor -- start 2024-09-19T05:08:28.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d909100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f6480 0x56465d909640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d900d40 0x56465d909b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481a9c50 con 0x56465d8f5b60 2024-09-19T05:08:28.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481ab0c0 con 0x56465d900d40 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9481aa320 con 0x56465d8f6480 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc950f00740 10 client.0 _ll_register_callbacks cb 0x7fc93400bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d909100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d909100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47580/0 (socket says 172.21.15.64:47580) 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 -- 172.21.15.64:0/3910661055 learned_addr learned my addr 172.21.15.64:0/3910661055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d900d40 0x56465d909b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 -- 172.21.15.64:0/3910661055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f6480 msgr2=0x56465d909640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f6480 0x56465d909640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 -- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d900d40 msgr2=0x56465d909b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d900d40 0x56465d909b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.225+0000 7fc947fff640 1 -- 172.21.15.64:0/3910661055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465d97ead0 con 0x56465d8f5b60 2024-09-19T05:08:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc947fff640 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d909100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc934006010 tx=0x7fc934005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb5bf08c34b9426 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc927fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc927fff640 1 -- 172.21.15.64:0/3910661055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934040040 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc927fff640 1 -- 172.21.15.64:0/3910661055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fc934004020 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93401e070 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc927fff640 1 -- 172.21.15.64:0/3910661055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc934004240 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc950f00740 10 client.15927 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.226+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56465d97fdd0 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc927fff640 1 -- 172.21.15.64:0/3910661055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc93401e070 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc927fff640 1 -- 172.21.15.64:0/3910661055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc934038050 con 0x56465d8f5b60 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc950f00740 10 client.15927 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc950f00740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc950f00740 20 client.15927 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:28.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc950f00740 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d9801f0 0x56465d9825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc950f00740 4 client.15927 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56465d862d00 con 0x56465d9801f0 2024-09-19T05:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.227+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d9801f0 0x56465d9825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.228+0000 7fc94c8c3640 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d9801f0 0x56465d9825e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d56c1ba60609be server_cookie=5b61e3e081e924ba in_seq=0 out_seq=0 2024-09-19T05:08:28.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.228+0000 7fc927fff640 10 client.15927 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:28.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc927fff640 1 -- 172.21.15.64:0/3910661055 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56465d862d00 con 0x56465d9801f0 2024-09-19T05:08:28.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc927fff640 10 client.15927 handle_command_reply: tid=0 2024-09-19T05:08:28.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 client.15927 shutdown 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 msgr2=0x56465d909100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d909100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc934006010 tx=0x7fc934005970 comp rx=0 tx=0).stop 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 shutdown_connections 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56465d900d40 0x56465d909b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56465d8f6480 0x56465d909640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56465d9801f0 0x56465d9825e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 --2- 172.21.15.64:0/3910661055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56465d8f5b60 0x56465d909100 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 >> 172.21.15.64:0/3910661055 conn(0x56465d8f20f0 msgr2=0x56465d910860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:28.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.229+0000 7fc927fff640 0 client.15927 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.230+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 shutdown_connections 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.230+0000 7fc950f00740 1 -- 172.21.15.64:0/3910661055 wait complete. 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.230+0000 7fc950f00740 20 client.15927 trim_cache size 0 max 16384 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "a314b164-b122-4566-97cc-b43c42d35629", 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:08:28.235 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.230+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9380a3430 msgr2=0x7fc9380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.230+0000 7fc94eb27640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9380a3430 0x7fc9380a58f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9400b9530 tx=0x7fc9400c1040 comp rx=0 tx=0).stop 2024-09-19T05:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 msgr2=0x7fc94819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94819cdc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc940005cc0 tx=0x7fc940003c20 comp rx=0 tx=0).stop 2024-09-19T05:08:28.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 shutdown_connections 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9380a3430 0x7fc9380a58f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9481082a0 0x7fc94819cdc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9481036e0 0x7fc948198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 --2- 172.21.15.64:0/1294225764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948102d30 0x7fc9481984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 >> 172.21.15.64:0/1294225764 conn(0x7fc9480fc7d0 msgr2=0x7fc948105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 shutdown_connections 2024-09-19T05:08:28.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:28.231+0000 7fc94eb27640 1 -- 172.21.15.64:0/1294225764 wait complete. 2024-09-19T05:08:28.826 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:28 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:08:28.826 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:28 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:28.826 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:28 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:08:28.827 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:28 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:28.827 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:28 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:08:28.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:28 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:08:28.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:28 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:28.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:28 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:08:28.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:28 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:28.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:28 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:08:29.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:28 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:08:29.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:28 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:29.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:28 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:08:29.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:28 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:29.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:28 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:08:29.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:08:29.313 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 -- 172.21.15.64:0/1540882644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4101ad0 msgr2=0x7f04e4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1540882644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4101ad0 0x7f04e4101f30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009930 tx=0x7f04d002a420 comp rx=0 tx=0).stop 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 -- 172.21.15.64:0/1540882644 shutdown_connections 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1540882644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4102470 0x7f04e410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1540882644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4101ad0 0x7f04e4101f30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1540882644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e4107b70 0x7f04e4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 -- 172.21.15.64:0/1540882644 >> 172.21.15.64:0/1540882644 conn(0x7f04e40fd7d0 msgr2=0x7f04e40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 -- 172.21.15.64:0/1540882644 shutdown_connections 2024-09-19T05:08:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.731+0000 7f04eadc4640 1 -- 172.21.15.64:0/1540882644 wait complete. 2024-09-19T05:08:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 Processor -- start 2024-09-19T05:08:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 -- start start 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4101ad0 0x7f04e41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 0x7f04e4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e4107b70 0x7f04e419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4111dd0 con 0x7f04e4102470 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4111c50 con 0x7f04e4101ad0 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04eadc4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4111f50 con 0x7f04e4107b70 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 0x7f04e4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 0x7f04e4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47606/0 (socket says 172.21.15.64:47606) 2024-09-19T05:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04e3fff640 1 -- 172.21.15.64:0/1994474074 learned_addr learned my addr 172.21.15.64:0/1994474074 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04e8b60640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4101ad0 0x7f04e41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.732+0000 7f04e9361640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e4107b70 0x7f04e419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e3fff640 1 -- 172.21.15.64:0/1994474074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e4107b70 msgr2=0x7f04e419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e4107b70 0x7f04e419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e3fff640 1 -- 172.21.15.64:0/1994474074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4101ad0 msgr2=0x7f04e41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4101ad0 0x7f04e41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e3fff640 1 -- 172.21.15.64:0/1994474074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e419d2b0 con 0x7f04e4102470 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e8b60640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4101ad0 0x7f04e41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 0x7f04e4198800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005940 tx=0x7f04d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e1ffb640 1 -- 172.21.15.64:0/1994474074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d003f030 con 0x7f04e4102470 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e1ffb640 1 -- 172.21.15.64:0/1994474074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d00318a0 con 0x7f04e4102470 2024-09-19T05:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04e1ffb640 1 -- 172.21.15.64:0/1994474074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0035070 con 0x7f04e4102470 2024-09-19T05:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e419d540 con 0x7f04e4102470 2024-09-19T05:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.733+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a4e30 con 0x7f04e4102470 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04e1ffb640 1 -- 172.21.15.64:0/1994474074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f04d0042030 con 0x7f04e4102470 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 Processor -- start 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 -- start start 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d40bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 0x561d8d416770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d8d3702f0 con 0x561d8d416cb0 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561d8d318270 con 0x561d8d40baf0 2024-09-19T05:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04edc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d8d36c2b0 con 0x561d8d40c410 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04e8b60640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 0x561d8d416770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04e8b60640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 0x561d8d416770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:45976/0 (socket says 172.21.15.64:45976) 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04e8b60640 1 -- 172.21.15.64:0/578390576 learned_addr learned my addr 172.21.15.64:0/578390576 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.737+0000 7f04e9361640 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d40bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e1ffb640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04d40a3460 0x7f04d40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e3fff640 1 -- 172.21.15.64:0/578390576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 msgr2=0x561d8d416770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e3fff640 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 0x561d8d416770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e3fff640 1 -- 172.21.15.64:0/578390576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 msgr2=0x561d8d40bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e3fff640 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d40bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e3fff640 1 -- 172.21.15.64:0/578390576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8d317340 con 0x561d8d416cb0 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e8b60640 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 0x561d8d416770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e1ffb640 1 -- 172.21.15.64:0/1994474074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f04d00f0410 con 0x7f04e4102470 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e9361640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04d40a3460 0x7f04d40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.738+0000 7f04e3fff640 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4190a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f04d00edb60 tx=0x7f04d00bc410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e6061e93a01bed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.739+0000 7f04ceffd640 1 -- 172.21.15.64:0/578390576 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0049070 con 0x561d8d416cb0 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.739+0000 7f04ceffd640 1 -- 172.21.15.64:0/578390576 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d0003e20 con 0x561d8d416cb0 2024-09-19T05:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.739+0000 7f04ceffd640 1 -- 172.21.15.64:0/578390576 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00c0aa0 con 0x561d8d416cb0 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04e9361640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04d40a3460 0x7f04d40a5920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005fd0 tx=0x7f04dc004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 -- 172.21.15.64:0/578390576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 msgr2=0x561d8d4190a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4190a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f04d00edb60 tx=0x7f04d00bc410 comp rx=0 tx=0).stop 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 -- 172.21.15.64:0/578390576 shutdown_connections 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4190a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40c410 0x561d8d416770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 --2- 172.21.15.64:0/578390576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d40bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 -- 172.21.15.64:0/578390576 >> 172.21.15.64:0/578390576 conn(0x561d8d408060 msgr2=0x561d8d408440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 -- 172.21.15.64:0/578390576 shutdown_connections 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 -- 172.21.15.64:0/578390576 wait complete. 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 -- start start 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.740+0000 7f04edc39740 1 Processor -- start 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d413b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40c410 0x561d8d4140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561d8d419640 con 0x561d8d416cb0 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561d8d2f9290 con 0x561d8d40c410 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d8d4197c0 con 0x561d8d40baf0 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e9361640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04edc39740 10 client.0 _ll_register_callbacks cb 0x561d8d32a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d413b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e9361640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4145e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47624/0 (socket says 172.21.15.64:47624) 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d413b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:45986/0 (socket says 172.21.15.64:45986) 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e9361640 1 -- 172.21.15.64:0/2098086441 learned_addr learned my addr 172.21.15.64:0/2098086441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e3fff640 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40c410 0x561d8d4140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 -- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40c410 msgr2=0x561d8d4140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40c410 0x561d8d4140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 -- 172.21.15.64:0/2098086441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 msgr2=0x561d8d4145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.746 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:29 smithi064 ceph-mon[30960]: pgmap v80: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.741+0000 7f04e8b60640 1 -- 172.21.15.64:0/2098086441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8d083210 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04e8b60640 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d413b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f04d80116c0 tx=0x7f04d8011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5376142b0c7531a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04c3fff640 1 -- 172.21.15.64:0/2098086441 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8010c60 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561d8d37da20 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04c3fff640 1 -- 172.21.15.64:0/2098086441 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f04d8010e00 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04c3fff640 1 -- 172.21.15.64:0/2098086441 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04d8004030 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04edc39740 10 client.25387 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.742+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d8d14f620 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04c3fff640 1 -- 172.21.15.64:0/2098086441 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561d8d37da20 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04c3fff640 1 -- 172.21.15.64:0/2098086441 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04d801a410 con 0x561d8d40baf0 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04edc39740 10 client.25387 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04edc39740 10 client.25387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04edc39740 20 client.25387 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04edc39740 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d42acf0 0x561d8d42d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04edc39740 4 client.25387 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d8d377d70 con 0x561d8d42acf0 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.743+0000 7f04e3fff640 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d42acf0 0x561d8d42d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.744+0000 7f04c3fff640 10 client.25387 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.744+0000 7f04e3fff640 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d42acf0 0x561d8d42d180 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e376ae3b7b9ce9 server_cookie=d70c653c8166a610 in_seq=0 out_seq=0 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.744+0000 7f04c3fff640 1 -- 172.21.15.64:0/2098086441 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d8d377d70 con 0x561d8d42acf0 2024-09-19T05:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.744+0000 7f04c3fff640 10 client.25387 handle_command_reply: tid=0 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.744+0000 7f04edc39740 1 client.25387 shutdown 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 msgr2=0x561d8d413b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d413b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f04d80116c0 tx=0x7f04d8011b80 comp rx=0 tx=0).stop 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 shutdown_connections 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d42acf0 0x561d8d42d180 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d4145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40c410 0x561d8d4140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 --2- 172.21.15.64:0/2098086441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d413b60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04c3fff640 0 client.25387 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 >> 172.21.15.64:0/2098086441 conn(0x561d8d408060 msgr2=0x561d8d40ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 shutdown_connections 2024-09-19T05:08:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 1 -- 172.21.15.64:0/2098086441 wait complete. 2024-09-19T05:08:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.745+0000 7f04edc39740 20 client.25387 trim_cache size 0 max 16384 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 Processor -- start 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 -- start start 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d414410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 0x561d8d432a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d416cb0 0x561d8d432fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41adeb0 con 0x561d8d40c410 2024-09-19T05:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41ab780 con 0x561d8d416cb0 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.769+0000 7f04edc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41a95d0 con 0x561d8d40baf0 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 0x561d8d432a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 0x561d8d432a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47638/0 (socket says 172.21.15.64:47638) 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 -- 172.21.15.64:0/3674781832 learned_addr learned my addr 172.21.15.64:0/3674781832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e8b60640 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d414410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e9361640 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d416cb0 0x561d8d432fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 -- 172.21.15.64:0/3674781832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 msgr2=0x561d8d414410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d414410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 -- 172.21.15.64:0/3674781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d416cb0 msgr2=0x561d8d432fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d416cb0 0x561d8d432fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 -- 172.21.15.64:0/3674781832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8d407e90 con 0x561d8d40c410 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e9361640 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d416cb0 0x561d8d432fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04e3fff640 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 0x561d8d432a90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f04d00bc730 tx=0x7f04d00f3f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937e3adeba9f844c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04ce7fc640 1 -- 172.21.15.64:0/3674781832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0049040 con 0x561d8d40c410 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04ce7fc640 1 -- 172.21.15.64:0/3674781832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d00c0aa0 con 0x561d8d40c410 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.770+0000 7f04ce7fc640 1 -- 172.21.15.64:0/3674781832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00edf70 con 0x561d8d40c410 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 -- 172.21.15.64:0/3674781832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 msgr2=0x561d8d432a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 0x561d8d432a90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f04d00bc730 tx=0x7f04d00f3f60 comp rx=0 tx=0).stop 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 -- 172.21.15.64:0/3674781832 shutdown_connections 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d416cb0 0x561d8d432fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d40c410 0x561d8d432a90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 --2- 172.21.15.64:0/3674781832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d40baf0 0x561d8d414410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 -- 172.21.15.64:0/3674781832 >> 172.21.15.64:0/3674781832 conn(0x561d8d408060 msgr2=0x561d8d415be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 -- 172.21.15.64:0/3674781832 shutdown_connections 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.771+0000 7f04edc39740 1 -- 172.21.15.64:0/3674781832 wait complete. 2024-09-19T05:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 -- start start 2024-09-19T05:08:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 Processor -- start 2024-09-19T05:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d41a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d41ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d41b1c0 0x561d8d494a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41a9c40 con 0x561d8d416cb0 2024-09-19T05:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41ab0b0 con 0x561d8d40baf0 2024-09-19T05:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41aa310 con 0x561d8d41b1c0 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04edc39740 10 client.0 _ll_register_callbacks cb 0x7f04d800bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d41ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04e8b60640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d41a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04e8b60640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d41a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:49438/0 (socket says 172.21.15.64:49438) 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04e9361640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d41b1c0 0x561d8d494a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04e8b60640 1 -- 172.21.15.64:0/1403603517 learned_addr learned my addr 172.21.15.64:0/1403603517 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.772+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d41ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47654/0 (socket says 172.21.15.64:47654) 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e3fff640 1 -- 172.21.15.64:0/1403603517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d41b1c0 msgr2=0x561d8d494a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d41b1c0 0x561d8d494a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e3fff640 1 -- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 msgr2=0x561d8d41a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d41a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e3fff640 1 -- 172.21.15.64:0/1403603517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8d494fa0 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e8b60640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d41a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e9361640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d41b1c0 0x561d8d494a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04e3fff640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d41ac80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x561d8d42d3a0 tx=0x7f04d00bc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805d64d5e59ab56d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04c3fff640 1 -- 172.21.15.64:0/1403603517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00d4020 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04c3fff640 1 -- 172.21.15.64:0/1403603517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f04d00edde0 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d00c0aa0 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04c3fff640 1 -- 172.21.15.64:0/1403603517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04d0049040 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04edc39740 10 client.15945 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.773+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d8d496120 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04c3fff640 1 -- 172.21.15.64:0/1403603517 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d00c0aa0 con 0x561d8d416cb0 2024-09-19T05:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04c3fff640 1 -- 172.21.15.64:0/1403603517 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04d00bca00 con 0x561d8d416cb0 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04edc39740 10 client.15945 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04edc39740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04edc39740 20 client.15945 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04edc39740 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d496370 0x561d8d498760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04edc39740 4 client.15945 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d8d2f4900 con 0x561d8d496370 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.774+0000 7f04e8b60640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d496370 0x561d8d498760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.775+0000 7f04e8b60640 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d496370 0x561d8d498760 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4304714ea5453788 server_cookie=b1da389094090ee4 in_seq=0 out_seq=0 2024-09-19T05:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.775+0000 7f04c3fff640 10 client.15945 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04c3fff640 1 -- 172.21.15.64:0/1403603517 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d8d2f4900 con 0x561d8d496370 2024-09-19T05:08:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04c3fff640 10 client.15945 handle_command_reply: tid=0 2024-09-19T05:08:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 client.15945 shutdown 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 msgr2=0x561d8d41ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d41ac80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x561d8d42d3a0 tx=0x7f04d00bc7a0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 shutdown_connections 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d8d41b1c0 0x561d8d494a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561d8d496370 0x561d8d498760 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561d8d416cb0 0x561d8d41ac80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04c3fff640 0 client.15945 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 --2- 172.21.15.64:0/1403603517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d8d40baf0 0x561d8d41a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 >> 172.21.15.64:0/1403603517 conn(0x561d8d408060 msgr2=0x561d8d4069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 shutdown_connections 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 1 -- 172.21.15.64:0/1403603517 wait complete. 2024-09-19T05:08:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.779+0000 7f04edc39740 20 client.15945 trim_cache size 0 max 16384 2024-09-19T05:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:08:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04d40a3460 msgr2=0x7f04d40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04d40a3460 0x7f04d40a5920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005fd0 tx=0x7f04dc004ab0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 msgr2=0x7f04e4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 0x7f04e4198800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005940 tx=0x7f04d00042f0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 shutdown_connections 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04d40a3460 0x7f04d40a5920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e4107b70 0x7f04e419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4102470 0x7f04e4198800 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 --2- 172.21.15.64:0/1994474074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04e4101ad0 0x7f04e41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 >> 172.21.15.64:0/1994474074 conn(0x7f04e40fd7d0 msgr2=0x7f04e40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 shutdown_connections 2024-09-19T05:08:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:29.780+0000 7f04eadc4640 1 -- 172.21.15.64:0/1994474074 wait complete. 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 -- 172.21.15.64:0/2066062557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff15010c2f0 msgr2=0x7ff15010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 --2- 172.21.15.64:0/2066062557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff15010c2f0 0x7ff15010e790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff148009ca0 tx=0x7ff14802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 -- 172.21.15.64:0/2066062557 shutdown_connections 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 --2- 172.21.15.64:0/2066062557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff15010c2f0 0x7ff15010e790 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 --2- 172.21.15.64:0/2066062557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1501037a0 0x7ff15010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 --2- 172.21.15.64:0/2066062557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff150102df0 0x7ff1501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 -- 172.21.15.64:0/2066062557 >> 172.21.15.64:0/2066062557 conn(0x7ff1500fc7d0 msgr2=0x7ff1500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 -- 172.21.15.64:0/2066062557 shutdown_connections 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.848+0000 7ff1573d1640 1 -- 172.21.15.64:0/2066062557 wait complete. 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 Processor -- start 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 -- start start 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 0x7ff1501984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1501037a0 0x7ff1501989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff15010c2f0 0x7ff15019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff150111fa0 con 0x7ff150102df0 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff150111e20 con 0x7ff15010c2f0 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff1573d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff150112120 con 0x7ff1501037a0 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.849+0000 7ff15516d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 0x7ff1501984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 0x7ff1501984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47672/0 (socket says 172.21.15.64:47672) 2024-09-19T05:08:29.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 -- 172.21.15.64:0/3271578104 learned_addr learned my addr 172.21.15.64:0/3271578104 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15596e640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff15010c2f0 0x7ff15019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 -- 172.21.15.64:0/3271578104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1501037a0 msgr2=0x7ff1501989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1501037a0 0x7ff1501989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 -- 172.21.15.64:0/3271578104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff15010c2f0 msgr2=0x7ff15019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff15010c2f0 0x7ff15019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 -- 172.21.15.64:0/3271578104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15019d460 con 0x7ff150102df0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff15516d640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 0x7ff1501984b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00d950 tx=0x7ff14c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff1467fc640 1 -- 172.21.15.64:0/3271578104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c017030 con 0x7ff150102df0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff1467fc640 1 -- 172.21.15.64:0/3271578104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff14c004020 con 0x7ff150102df0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff1467fc640 1 -- 172.21.15.64:0/3271578104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0047a0 con 0x7ff150102df0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15019d750 con 0x7ff150102df0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.850+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff150100980 con 0x7ff150102df0 2024-09-19T05:08:29.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.853+0000 7ff1467fc640 1 -- 172.21.15.64:0/3271578104 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff14c0041d0 con 0x7ff150102df0 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.853+0000 7ff1597aa740 1 Processor -- start 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 -- start start 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef5184a0 0x55d1ef518880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef523140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef525a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1ef47e2e0 con 0x55d1ef523680 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1ef425ce0 con 0x55d1ef518dc0 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1597aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1ef47a2a0 con 0x55d1ef5184a0 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1467fc640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff13c0a3430 0x7ff13c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef525a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15516d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef523140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15596e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef5184a0 0x55d1ef518880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef525a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47680/0 (socket says 172.21.15.64:47680) 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 -- 172.21.15.64:0/89624657 learned_addr learned my addr 172.21.15.64:0/89624657 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15596e640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff13c0a3430 0x7ff13c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff1467fc640 1 -- 172.21.15.64:0/3271578104 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff14c0cf770 con 0x7ff150102df0 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 -- 172.21.15.64:0/89624657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef5184a0 msgr2=0x55d1ef518880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef5184a0 0x55d1ef518880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 -- 172.21.15.64:0/89624657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 msgr2=0x55d1ef523140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.854+0000 7ff15496c640 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef523140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff15496c640 1 -- 172.21.15.64:0/89624657 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ef40a440 con 0x55d1ef523680 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff15516d640 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef523140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff15496c640 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef525a70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff140005940 tx=0x7ff14000ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04b44a5f9e58c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff15596e640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff13c0a3430 0x7ff13c0a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff148004af0 tx=0x7ff148004a60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff13b7fe640 1 -- 172.21.15.64:0/89624657 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004220 con 0x55d1ef523680 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff13b7fe640 1 -- 172.21.15.64:0/89624657 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff140015e20 con 0x55d1ef523680 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.855+0000 7ff13b7fe640 1 -- 172.21.15.64:0/89624657 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140003c50 con 0x55d1ef523680 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 -- 172.21.15.64:0/89624657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 msgr2=0x55d1ef525a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef525a70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff140005940 tx=0x7ff14000ae20 comp rx=0 tx=0).stop 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 -- 172.21.15.64:0/89624657 shutdown_connections 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef525a70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef523140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 --2- 172.21.15.64:0/89624657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef5184a0 0x55d1ef518880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 -- 172.21.15.64:0/89624657 >> 172.21.15.64:0/89624657 conn(0x55d1ef514a30 msgr2=0x55d1ef514e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 -- 172.21.15.64:0/89624657 shutdown_connections 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 -- 172.21.15.64:0/89624657 wait complete. 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 -- start start 2024-09-19T05:08:29.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.856+0000 7ff1597aa740 1 Processor -- start 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef522f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef520220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef523680 0x55d1ef520760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1ef526250 con 0x55d1ef5184a0 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1ef439050 con 0x55d1ef518dc0 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1ef5263d0 con 0x55d1ef523680 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef522f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff1597aa740 10 client.0 _ll_register_callbacks cb 0x55d1ef479160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef522f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47692/0 (socket says 172.21.15.64:47692) 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 -- 172.21.15.64:0/157837944 learned_addr learned my addr 172.21.15.64:0/157837944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15496c640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef520220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 -- 172.21.15.64:0/157837944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef523680 msgr2=0x55d1ef520760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef523680 0x55d1ef520760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 -- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 msgr2=0x55d1ef520220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef520220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15516d640 1 -- 172.21.15.64:0/157837944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ef1ef3c0 con 0x55d1ef5184a0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.857+0000 7ff15496c640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef520220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff15516d640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef522f70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff14c06d7a0 tx=0x7ff14c06d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64100e932452c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff138ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff138ff9640 1 -- 172.21.15.64:0/157837944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c09fe00 con 0x55d1ef5184a0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff140015e20 con 0x55d1ef5184a0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff138ff9640 1 -- 172.21.15.64:0/157837944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff14c0980c0 con 0x55d1ef5184a0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff138ff9640 1 -- 172.21.15.64:0/157837944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff14c0a46c0 con 0x55d1ef5184a0 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff1597aa740 10 client.15957 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:29.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1ef531c10 con 0x55d1ef5184a0 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff138ff9640 1 -- 172.21.15.64:0/157837944 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff140015e20 con 0x55d1ef5184a0 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.858+0000 7ff138ff9640 1 -- 172.21.15.64:0/157837944 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff14c0a4230 con 0x55d1ef5184a0 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff1597aa740 10 client.15957 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff1597aa740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff1597aa740 20 client.15957 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff1597aa740 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef533560 0x55d1ef5359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff1597aa740 4 client.15957 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ef48b850 con 0x55d1ef533560 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.859+0000 7ff15496c640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef533560 0x55d1ef5359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff138ff9640 10 client.15957 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff15496c640 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef533560 0x55d1ef5359f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9caad941b691ff53 server_cookie=8881c781de9ed17 in_seq=0 out_seq=0 2024-09-19T05:08:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff138ff9640 1 -- 172.21.15.64:0/157837944 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1ef48b850 con 0x55d1ef533560 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff138ff9640 10 client.15957 handle_command_reply: tid=0 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 client.15957 shutdown 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 msgr2=0x55d1ef522f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef522f70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff14c06d7a0 tx=0x7ff14c06d840 comp rx=0 tx=0).stop 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 shutdown_connections 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef523680 0x55d1ef520760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef518dc0 0x55d1ef520220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef533560 0x55d1ef5359f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 --2- 172.21.15.64:0/157837944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef522f70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.860+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 >> 172.21.15.64:0/157837944 conn(0x55d1ef514a30 msgr2=0x55d1ef51a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.861+0000 7ff138ff9640 0 client.15957 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.861+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 shutdown_connections 2024-09-19T05:08:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.861+0000 7ff1597aa740 1 -- 172.21.15.64:0/157837944 wait complete. 2024-09-19T05:08:29.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.861+0000 7ff1597aa740 20 client.15957 trim_cache size 0 max 16384 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 Processor -- start 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 -- start start 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef52f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef520410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef523680 0x55d1ef51b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1501addf0 con 0x55d1ef5184a0 2024-09-19T05:08:29.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1501ab6c0 con 0x55d1ef523680 2024-09-19T05:08:29.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff1597aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1501a9510 con 0x55d1ef518dc0 2024-09-19T05:08:29.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff15516d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef52f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.880+0000 7ff15516d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef52f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47708/0 (socket says 172.21.15.64:47708) 2024-09-19T05:08:29.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 -- 172.21.15.64:0/3763883856 learned_addr learned my addr 172.21.15.64:0/3763883856 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15596e640 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef523680 0x55d1ef51b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15496c640 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef520410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 -- 172.21.15.64:0/3763883856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 msgr2=0x55d1ef520410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef520410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 -- 172.21.15.64:0/3763883856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef523680 msgr2=0x55d1ef51b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef523680 0x55d1ef51b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 -- 172.21.15.64:0/3763883856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ef51f1d0 con 0x55d1ef5184a0 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15496c640 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef520410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff15516d640 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef52f4b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0cd4c0 tx=0x7ff14c09bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741dfb146827d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff13affd640 1 -- 172.21.15.64:0/3763883856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c09c150 con 0x55d1ef5184a0 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff13affd640 1 -- 172.21.15.64:0/3763883856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff14c09fe00 con 0x55d1ef5184a0 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.881+0000 7ff13affd640 1 -- 172.21.15.64:0/3763883856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0977a0 con 0x55d1ef5184a0 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 -- 172.21.15.64:0/3763883856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 msgr2=0x55d1ef52f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef52f4b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0cd4c0 tx=0x7ff14c09bbd0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 -- 172.21.15.64:0/3763883856 shutdown_connections 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef523680 0x55d1ef51b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef520410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 --2- 172.21.15.64:0/3763883856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef5184a0 0x55d1ef52f4b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 -- 172.21.15.64:0/3763883856 >> 172.21.15.64:0/3763883856 conn(0x55d1ef514a30 msgr2=0x55d1ef51b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 -- 172.21.15.64:0/3763883856 shutdown_connections 2024-09-19T05:08:29.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.882+0000 7ff1597aa740 1 -- 172.21.15.64:0/3763883856 wait complete. 2024-09-19T05:08:29.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 -- start start 2024-09-19T05:08:29.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 Processor -- start 2024-09-19T05:08:29.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef5184a0 0x55d1ef5266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef526c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef59c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1501a9b80 con 0x55d1ef523680 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1501aaff0 con 0x55d1ef5184a0 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.883+0000 7ff1597aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1501aa250 con 0x55d1ef518dc0 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef59c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef59c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47722/0 (socket says 172.21.15.64:47722) 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 -- 172.21.15.64:0/1492807720 learned_addr learned my addr 172.21.15.64:0/1492807720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15516d640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef5184a0 0x55d1ef5266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15496c640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef526c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 -- 172.21.15.64:0/1492807720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 msgr2=0x55d1ef526c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef526c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 -- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef5184a0 msgr2=0x55d1ef5266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef5184a0 0x55d1ef5266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 -- 172.21.15.64:0/1492807720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ef5274c0 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff15596e640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef59c2c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff14802ade0 tx=0x7ff14802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd916e3b37751f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff138ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff1597aa740 10 client.0 _ll_register_callbacks cb 0x7ff14c09b3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14c09fe00 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff138ff9640 1 -- 172.21.15.64:0/1492807720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148052070 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.884+0000 7ff138ff9640 1 -- 172.21.15.64:0/1492807720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff148004040 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff138ff9640 1 -- 172.21.15.64:0/1492807720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff14805e050 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 10 client.15963 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55d1ef59d7c0 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff138ff9640 1 -- 172.21.15.64:0/1492807720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff14804c040 con 0x55d1ef523680 2024-09-19T05:08:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 20 client.15963 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef59dde0 0x55d1ef5a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 4 client.15963 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.885+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1ef520af0 con 0x55d1ef59dde0 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff15516d640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef59dde0 0x55d1ef5a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff15516d640 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef59dde0 0x55d1ef5a0280 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cefaf9ff427d2582 server_cookie=d4641cd19fb2f368 in_seq=0 out_seq=0 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff138ff9640 10 client.15963 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff138ff9640 1 -- 172.21.15.64:0/1492807720 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d1ef520af0 con 0x55d1ef59dde0 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff138ff9640 10 client.15963 handle_command_reply: tid=0 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff1597aa740 1 client.15963 shutdown 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 msgr2=0x55d1ef59c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.886+0000 7ff1597aa740 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef59c2c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff14802ade0 tx=0x7ff14802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 shutdown_connections 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d1ef59dde0 0x55d1ef5a0280 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1ef523680 0x55d1ef59c2c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1ef518dc0 0x55d1ef526c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 --2- 172.21.15.64:0/1492807720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1ef5184a0 0x55d1ef5266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 >> 172.21.15.64:0/1492807720 conn(0x55d1ef514a30 msgr2=0x55d1ef513a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff138ff9640 0 client.15963 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 shutdown_connections 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 1 -- 172.21.15.64:0/1492807720 wait complete. 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.887+0000 7ff1597aa740 20 client.15963 trim_cache size 0 max 16384 2024-09-19T05:08:29.893 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:08:29.894 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:08:29.894 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:08:29.894 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:08:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff13c0a3430 msgr2=0x7ff13c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff13c0a3430 0x7ff13c0a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff148004af0 tx=0x7ff148004a60 comp rx=0 tx=0).stop 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 msgr2=0x7ff1501984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 0x7ff1501984b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00d950 tx=0x7ff14c00de10 comp rx=0 tx=0).stop 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 shutdown_connections 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff13c0a3430 0x7ff13c0a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff15010c2f0 0x7ff15019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1501037a0 0x7ff1501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 --2- 172.21.15.64:0/3271578104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff150102df0 0x7ff1501984b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 >> 172.21.15.64:0/3271578104 conn(0x7ff1500fc7d0 msgr2=0x7ff1500fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 shutdown_connections 2024-09-19T05:08:29.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:29.888+0000 7ff1573d1640 1 -- 172.21.15.64:0/3271578104 wait complete. 2024-09-19T05:08:29.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:29 smithi103 ceph-mon[34446]: pgmap v80: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:08:30.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:29 smithi174 ceph-mon[34488]: pgmap v80: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T05:08:30.743 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:08:30.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:08:31.020 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.545+0000 7f1b120bc640 1 -- 172.21.15.64:0/1764697432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 msgr2=0x7f1b0c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.545+0000 7f1b120bc640 1 --2- 172.21.15.64:0/1764697432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c106e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009930 tx=0x7f1afc02a420 comp rx=0 tx=0).stop 2024-09-19T05:08:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 -- 172.21.15.64:0/1764697432 shutdown_connections 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 --2- 172.21.15.64:0/1764697432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b0c101410 0x7f1b0c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 --2- 172.21.15.64:0/1764697432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c100a70 0x7f1b0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 --2- 172.21.15.64:0/1764697432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c106e50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 -- 172.21.15.64:0/1764697432 >> 172.21.15.64:0/1764697432 conn(0x7f1b0c0fc790 msgr2=0x7f1b0c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 -- 172.21.15.64:0/1764697432 shutdown_connections 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 -- 172.21.15.64:0/1764697432 wait complete. 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.546+0000 7f1b120bc640 1 Processor -- start 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 -- start start 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b0c100a70 0x7f1b0c1079e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c101410 0x7f1b0c107f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c108460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c110d30 con 0x7f1b0c106a70 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c110bb0 con 0x7f1b0c101410 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b120bc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c110eb0 con 0x7f1b0c100a70 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c108460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c108460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47742/0 (socket says 172.21.15.64:47742) 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b0c100a70 0x7f1b0c1079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 -- 172.21.15.64:0/797681087 learned_addr learned my addr 172.21.15.64:0/797681087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0affd640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c101410 0x7f1b0c107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 -- 172.21.15.64:0/797681087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b0c100a70 msgr2=0x7f1b0c1079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b0c100a70 0x7f1b0c1079e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 -- 172.21.15.64:0/797681087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c101410 msgr2=0x7f1b0c107f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c101410 0x7f1b0c107f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.547+0000 7f1b0bfff640 1 -- 172.21.15.64:0/797681087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c1a6890 con 0x7f1b0c106a70 2024-09-19T05:08:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.548+0000 7f1b0bfff640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c108460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400cad0 tx=0x7f1b0400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.550+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400ae20 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.550+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b04005020 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.550+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400f6b0 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.550+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c1a6b80 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.550+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c1a70c0 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.552+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c106e50 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.553+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1b040051d0 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.553+0000 7f1b08ff9640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1af40a3430 0x7f1af40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.554+0000 7f1b0b7fe640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1af40a3430 0x7f1af40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.554+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1b040cfc30 con 0x7f1b0c106a70 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.554+0000 7f1b0b7fe640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1af40a3430 0x7f1af40a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1afc031440 tx=0x7f1afc005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:31.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.558+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b04014030 con 0x7f1b0c106a70 2024-09-19T05:08:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.804+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b0c1a7350 con 0x7f1b0c106a70 2024-09-19T05:08:31.806 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:31 smithi064 ceph-mon[30960]: pgmap v81: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:08:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.805+0000 7f1b08ff9640 1 -- 172.21.15.64:0/797681087 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f1b0409bfc0 con 0x7f1b0c106a70 2024-09-19T05:08:31.807 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:08:31.807 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:08:31.807 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1af40a3430 msgr2=0x7f1af40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1af40a3430 0x7f1af40a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1afc031440 tx=0x7f1afc005a20 comp rx=0 tx=0).stop 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 msgr2=0x7f1b0c108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c108460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400cad0 tx=0x7f1b0400cf90 comp rx=0 tx=0).stop 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 shutdown_connections 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1af40a3430 0x7f1af40a58f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b0c106a70 0x7f1b0c108460 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c101410 0x7f1b0c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 --2- 172.21.15.64:0/797681087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b0c100a70 0x7f1b0c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:31.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 >> 172.21.15.64:0/797681087 conn(0x7f1b0c0fc790 msgr2=0x7f1b0c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:31.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 shutdown_connections 2024-09-19T05:08:31.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:31.808+0000 7f1b120bc640 1 -- 172.21.15.64:0/797681087 wait complete. 2024-09-19T05:08:31.830 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:31 smithi174 ceph-mon[34488]: pgmap v81: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:08:31.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:31 smithi103 ceph-mon[34446]: pgmap v81: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:08:32.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:08:32.845 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:32.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:32 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/797681087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:32.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:32 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/797681087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:33.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:32 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/797681087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/3403096743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 msgr2=0x7f1bf41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/3403096743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 0x7f1bf41029b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009900 tx=0x7f1be402a480 comp rx=0 tx=0).stop 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/3403096743 shutdown_connections 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/3403096743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4102ef0 0x7f1bf410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/3403096743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 0x7f1bf41029b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/3403096743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4100e20 0x7f1bf4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/3403096743 >> 172.21.15.64:0/3403096743 conn(0x7f1bf40fc7d0 msgr2=0x7f1bf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/3403096743 shutdown_connections 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.375+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/3403096743 wait complete. 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 Processor -- start 2024-09-19T05:08:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 -- start start 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 0x7f1bf41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4110e30 con 0x7f1bf4102570 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4110cb0 con 0x7f1bf4102ef0 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bfb4f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4110fb0 con 0x7f1bf4100e20 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bf9290640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bf9290640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:46110/0 (socket says 172.21.15.64:46110) 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bf9290640 1 -- 172.21.15.64:0/4102854722 learned_addr learned my addr 172.21.15.64:0/4102854722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.376+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 0x7f1bf41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf9290640 1 -- 172.21.15.64:0/4102854722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102ef0 msgr2=0x7f1bf419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf9290640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf9290640 1 -- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 msgr2=0x7f1bf41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf9290640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 0x7f1bf41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf9290640 1 -- 172.21.15.64:0/4102854722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf419d310 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bf9290640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4198360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c920 tx=0x7f1bf000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bea7fc640 1 -- 172.21.15.64:0/4102854722 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0007c70 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bea7fc640 1 -- 172.21.15.64:0/4102854722 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bf0007e10 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf419d600 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.377+0000 7f1bea7fc640 1 -- 172.21.15.64:0/4102854722 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0018820 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.378+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf41a4ee0 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.380+0000 7f1bfd8cd740 1 Processor -- start 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.380+0000 7f1bfd8cd740 1 -- start start 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.380+0000 7f1bea7fc640 1 -- 172.21.15.64:0/4102854722 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1bf00189c0 con 0x7f1bf4100e20 2024-09-19T05:08:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 0x5591437fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437fd5d0 0x5591437fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x55914380eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591437632f0 con 0x5591437feb40 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55914370acf0 con 0x55914380c780 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55914375f2b0 con 0x5591437fd5d0 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 0x5591437fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 0x5591437fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47776/0 (socket says 172.21.15.64:47776) 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3519321421 learned_addr learned my addr 172.21.15.64:0/3519321421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9290640 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437fd5d0 0x5591437fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x55914380eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3519321421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437fd5d0 msgr2=0x5591437fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437fd5d0 0x5591437fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3519321421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 msgr2=0x55914380eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x55914380eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.381+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3519321421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591436ef3a0 con 0x5591437feb40 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bea7fc640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1be00a33b0 0x7f1be00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 0x5591437fd090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1bec010370 tx=0x7f1bec010770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ef405e911b49b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bea7fc640 1 -- 172.21.15.64:0/4102854722 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1bf00cfda0 con 0x7f1bf4100e20 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bf9290640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1be00a33b0 0x7f1be00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/3519321421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec016070 con 0x5591437feb40 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/3519321421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bec004d20 con 0x5591437feb40 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.382+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/3519321421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec012780 con 0x5591437feb40 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3519321421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 msgr2=0x5591437fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 0x5591437fd090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1bec010370 tx=0x7f1bec010770 comp rx=0 tx=0).stop 2024-09-19T05:08:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bf9290640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1be00a33b0 0x7f1be00a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00cd5a0 tx=0x7f1bf00d38e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3519321421 shutdown_connections 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x55914380eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437fd5d0 0x5591437fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3519321421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437feb40 0x5591437fd090 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3519321421 >> 172.21.15.64:0/3519321421 conn(0x5591437c6310 msgr2=0x5591437f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3519321421 shutdown_connections 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3519321421 wait complete. 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.383+0000 7f1bfd8cd740 1 -- start start 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 Processor -- start 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437fd5d0 0x55914380c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x559143809480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 0x559143809c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55914380f480 con 0x55914380c780 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55914371e060 con 0x5591437fd5d0 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55914380f600 con 0x5591437feb40 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bfd8cd740 10 client.0 _ll_register_callbacks cb 0x5591434718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 0x559143809c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 0x559143809c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47788/0 (socket says 172.21.15.64:47788) 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bf9a91640 1 -- 172.21.15.64:0/1479520121 learned_addr learned my addr 172.21.15.64:0/1479520121 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x559143809480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.384+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437fd5d0 0x55914380c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.385+0000 7f1bf9a91640 1 -- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 msgr2=0x559143809480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.385+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x559143809480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.385+0000 7f1bf9a91640 1 -- 172.21.15.64:0/1479520121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437fd5d0 msgr2=0x55914380c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.385+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437fd5d0 0x55914380c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.385+0000 7f1bf9a91640 1 -- 172.21.15.64:0/1479520121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591434d43c0 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.385+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x559143809480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 0x559143809c10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1bec005b00 tx=0x7f1bec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ed27571bb2a328 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bdcff9640 1 -- 172.21.15.64:0/1479520121 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec01e020 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bdcff9640 1 -- 172.21.15.64:0/1479520121 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1bec012bb0 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bec004d20 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bdcff9640 1 -- 172.21.15.64:0/1479520121 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bec02f050 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bfd8cd740 10 client.15981 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.386+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55914381af90 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bdcff9640 1 -- 172.21.15.64:0/1479520121 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bec025040 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bdcff9640 1 -- 172.21.15.64:0/1479520121 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1bec00f040 con 0x55914380c780 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bfd8cd740 10 client.15981 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bfd8cd740 10 client.15981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:08:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bfd8cd740 20 client.15981 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55914381c340 0x55914381e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bfd8cd740 4 client.15981 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1bec004d20 con 0x55914381c340 2024-09-19T05:08:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.387+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55914381c340 0x55914381e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.388+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55914381c340 0x55914381e730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdecd93399a03ba2 server_cookie=e236faf78654ec38 in_seq=0 out_seq=0 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.388+0000 7f1bdcff9640 10 client.15981 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.388+0000 7f1bdcff9640 1 -- 172.21.15.64:0/1479520121 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1bec004d20 con 0x55914381c340 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.388+0000 7f1bdcff9640 10 client.15981 handle_command_reply: tid=0 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 client.15981 shutdown 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 msgr2=0x559143809c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 0x559143809c10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1bec005b00 tx=0x7f1bec009bb0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 shutdown_connections 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55914381c340 0x55914381e730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55914380c780 0x559143809c10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x559143809480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bdcff9640 0 client.15981 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/1479520121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437fd5d0 0x55914380c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 >> 172.21.15.64:0/1479520121 conn(0x5591437c6310 msgr2=0x5591437ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 shutdown_connections 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/1479520121 wait complete. 2024-09-19T05:08:33.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.389+0000 7f1bfd8cd740 20 client.15981 trim_cache size 0 max 16384 2024-09-19T05:08:33.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 Processor -- start 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 -- start start 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x5591437f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x5591437fe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x5591438084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ade30 con 0x5591437fd5d0 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ab700 con 0x55914380c780 2024-09-19T05:08:33.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.404+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41a9550 con 0x5591437feb40 2024-09-19T05:08:33.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x5591437f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x5591437f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47798/0 (socket says 172.21.15.64:47798) 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3125498348 learned_addr learned my addr 172.21.15.64:0/3125498348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x5591437fe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3125498348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 msgr2=0x5591437fe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x5591437fe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3125498348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 msgr2=0x5591438084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x5591438084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 -- 172.21.15.64:0/3125498348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914380c540 con 0x5591437fd5d0 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x5591437f9d70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1bec02a7c0 tx=0x7f1bec02a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9986c3ae571944f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bdeffd640 1 -- 172.21.15.64:0/3125498348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec016070 con 0x5591437fd5d0 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bdeffd640 1 -- 172.21.15.64:0/3125498348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bec010c70 con 0x5591437fd5d0 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.405+0000 7f1bdeffd640 1 -- 172.21.15.64:0/3125498348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec02a960 con 0x5591437fd5d0 2024-09-19T05:08:33.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3125498348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 msgr2=0x5591437f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x5591437f9d70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1bec02a7c0 tx=0x7f1bec02a7f0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3125498348 shutdown_connections 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55914380c780 0x5591438084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591437feb40 0x5591437fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/3125498348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x5591437f9d70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3125498348 >> 172.21.15.64:0/3125498348 conn(0x5591437c6310 msgr2=0x559143807d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3125498348 shutdown_connections 2024-09-19T05:08:33.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.406+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/3125498348 wait complete. 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 -- start start 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 Processor -- start 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x559143802650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437feb40 0x559143802b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55914380c780 0x5591438030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41a9bc0 con 0x5591437fd5d0 2024-09-19T05:08:33.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ab030 con 0x5591437feb40 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41aa290 con 0x55914380c780 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bfd8cd740 10 client.0 _ll_register_callbacks cb 0x7f1bf00d37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x559143802650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bf9a91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x559143802650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47812/0 (socket says 172.21.15.64:47812) 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bf9a91640 1 -- 172.21.15.64:0/415152018 learned_addr learned my addr 172.21.15.64:0/415152018 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437feb40 0x559143802b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.407+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55914380c780 0x5591438030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf9a91640 1 -- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55914380c780 msgr2=0x5591438030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55914380c780 0x5591438030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf9a91640 1 -- 172.21.15.64:0/415152018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437feb40 msgr2=0x559143802b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:33.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437feb40 0x559143802b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf9a91640 1 -- 172.21.15.64:0/415152018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591438207e0 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437feb40 0x559143802b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bf9a91640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x559143802650 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009770 tx=0x7f1bec012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62731f96de72d23b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bdcff9640 1 -- 172.21.15.64:0/415152018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec016070 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bdcff9640 1 -- 172.21.15.64:0/415152018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1bec02bda0 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bec010c70 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bdcff9640 1 -- 172.21.15.64:0/415152018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bec039040 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bfd8cd740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.408+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559143821aa0 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bdcff9640 1 -- 172.21.15.64:0/415152018 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bec010c70 con 0x5591437fd5d0 2024-09-19T05:08:33.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bdcff9640 1 -- 172.21.15.64:0/415152018 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1bec0044b0 con 0x5591437fd5d0 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bfd8cd740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bfd8cd740 10 client.15987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bfd8cd740 20 client.15987 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5591438221d0 0x55914388d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bfd8cd740 4 client.15987 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559143770860 con 0x5591438221d0 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.409+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5591438221d0 0x55914388d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bdcff9640 10 client.15987 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bf8a8f640 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5591438221d0 0x55914388d470 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ff077e3c8ddbcea server_cookie=e70594c122bf81e5 in_seq=0 out_seq=0 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bdcff9640 1 -- 172.21.15.64:0/415152018 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559143770860 con 0x5591438221d0 2024-09-19T05:08:33.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bdcff9640 10 client.15987 handle_command_reply: tid=0 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 client.15987 shutdown 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 msgr2=0x559143802650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x559143802650 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009770 tx=0x7f1bec012bb0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 shutdown_connections 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5591438221d0 0x55914388d470 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55914380c780 0x5591438030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5591437feb40 0x559143802b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 --2- 172.21.15.64:0/415152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591437fd5d0 0x559143802650 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 >> 172.21.15.64:0/415152018 conn(0x5591437c6310 msgr2=0x5591438180b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.410+0000 7f1bdcff9640 0 client.15987 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 shutdown_connections 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfd8cd740 1 -- 172.21.15.64:0/415152018 wait complete. 2024-09-19T05:08:33.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfd8cd740 20 client.15987 trim_cache size 0 max 16384 2024-09-19T05:08:33.417 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1be00a33b0 msgr2=0x7f1be00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1be00a33b0 0x7f1be00a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00cd5a0 tx=0x7f1bf00d38e0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 msgr2=0x7f1bf4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4198360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c920 tx=0x7f1bf000cde0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 shutdown_connections 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1be00a33b0 0x7f1be00a5870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.411+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1bf4102ef0 0x7f1bf419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.412+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bf4102570 0x7f1bf41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.412+0000 7f1bfb4f4640 1 --2- 172.21.15.64:0/4102854722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bf4100e20 0x7f1bf4198360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.412+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 >> 172.21.15.64:0/4102854722 conn(0x7f1bf40fc7d0 msgr2=0x7f1bf406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.412+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 shutdown_connections 2024-09-19T05:08:33.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:33.412+0000 7f1bfb4f4640 1 -- 172.21.15.64:0/4102854722 wait complete. 2024-09-19T05:08:33.870 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:33 smithi103 ceph-mon[34446]: pgmap v82: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:08:33.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:33 smithi064 ceph-mon[30960]: pgmap v82: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:08:34.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:33 smithi174 ceph-mon[34488]: pgmap v82: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:08:34.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:08:34.442 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:34.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:34 smithi064 ceph-mon[30960]: pgmap v83: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-19T05:08:34.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 -- 172.21.15.64:0/641173781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 msgr2=0x7f5c201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 --2- 172.21.15.64:0/641173781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c201029b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009900 tx=0x7f5c1402a480 comp rx=0 tx=0).stop 2024-09-19T05:08:34.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 -- 172.21.15.64:0/641173781 shutdown_connections 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 --2- 172.21.15.64:0/641173781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20102ef0 0x7f5c2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 --2- 172.21.15.64:0/641173781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c201029b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 --2- 172.21.15.64:0/641173781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c20100e20 0x7f5c20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 -- 172.21.15.64:0/641173781 >> 172.21.15.64:0/641173781 conn(0x7f5c200fc7d0 msgr2=0x7f5c200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 -- 172.21.15.64:0/641173781 shutdown_connections 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.950+0000 7f5c24e60640 1 -- 172.21.15.64:0/641173781 wait complete. 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 Processor -- start 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 -- start start 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20100e20 0x7f5c20198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c20198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c20102ef0 0x7f5c2019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20110e30 con 0x7f5c20102570 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20110cb0 con 0x7f5c20100e20 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c24e60640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20110fb0 con 0x7f5c20102ef0 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c1dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c20198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c1dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c20198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42152/0 (socket says 172.21.15.64:42152) 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c1dd9b640 1 -- 172.21.15.64:0/1807925137 learned_addr learned my addr 172.21.15.64:0/1807925137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.951+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20100e20 0x7f5c20198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c20102ef0 0x7f5c2019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1807925137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c20102ef0 msgr2=0x7f5c2019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c20102ef0 0x7f5c2019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 msgr2=0x7f5c20198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c20198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1807925137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2019d3d0 con 0x7f5c20100e20 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20100e20 0x7f5c20198420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c970 tx=0x7f5c1000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c0f7fe640 1 -- 172.21.15.64:0/1807925137 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10007bb0 con 0x7f5c20100e20 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2019d6c0 con 0x7f5c20100e20 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.952+0000 7f5c0f7fe640 1 -- 172.21.15.64:0/1807925137 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c10007d50 con 0x7f5c20100e20 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.953+0000 7f5c0f7fe640 1 -- 172.21.15.64:0/1807925137 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c10004e40 con 0x7f5c20100e20 2024-09-19T05:08:34.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.953+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201a4fa0 con 0x7f5c20100e20 2024-09-19T05:08:34.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.955+0000 7f5c0f7fe640 1 -- 172.21.15.64:0/1807925137 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5c10005020 con 0x7f5c20100e20 2024-09-19T05:08:34.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c0f7fe640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5c080a33b0 0x7f5c080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 Processor -- start 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5c080a33b0 0x7f5c080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c0f7fe640 1 -- 172.21.15.64:0/1807925137 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5c100cf980 con 0x7f5c20100e20 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 -- start start 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 0x55c0d393f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d393fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d3950b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0d38a52f0 con 0x55c0d3940b40 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0d384ccf0 con 0x55c0d393f5d0 2024-09-19T05:08:34.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.956+0000 7f5c27239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0d38a12b0 con 0x55c0d394e780 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 0x55c0d393f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5c080a33b0 0x7f5c080a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c14005ec0 tx=0x7f5c14005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 0x55c0d393f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42164/0 (socket says 172.21.15.64:42164) 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 -- 172.21.15.64:0/732476599 learned_addr learned my addr 172.21.15.64:0/732476599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d393fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d3950b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 -- 172.21.15.64:0/732476599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 msgr2=0x55c0d3950b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d3950b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 -- 172.21.15.64:0/732476599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 msgr2=0x55c0d393fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d393fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 -- 172.21.15.64:0/732476599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d38313a0 con 0x55c0d3940b40 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d3950b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 0x55c0d393f090 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000cfd0 tx=0x7f5c1009bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7ad9aa2ce89f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c0cff9640 1 -- 172.21.15.64:0/732476599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c100b4e60 con 0x55c0d3940b40 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c0cff9640 1 -- 172.21.15.64:0/732476599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c100b1020 con 0x55c0d3940b40 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.957+0000 7f5c0cff9640 1 -- 172.21.15.64:0/732476599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1001d040 con 0x55c0d3940b40 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 -- 172.21.15.64:0/732476599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 msgr2=0x55c0d393f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 0x55c0d393f090 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000cfd0 tx=0x7f5c1009bd10 comp rx=0 tx=0).stop 2024-09-19T05:08:34.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 -- 172.21.15.64:0/732476599 shutdown_connections 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d3950b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d393fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 --2- 172.21.15.64:0/732476599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d3940b40 0x55c0d393f090 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.958+0000 7f5c27239740 1 -- 172.21.15.64:0/732476599 >> 172.21.15.64:0/732476599 conn(0x55c0d3908310 msgr2=0x55c0d393bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 -- 172.21.15.64:0/732476599 shutdown_connections 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 -- 172.21.15.64:0/732476599 wait complete. 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 -- start start 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 Processor -- start 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d394e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d3940b40 0x55c0d394b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d394b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0d3951480 con 0x55c0d393f5d0 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0d3860060 con 0x55c0d3940b40 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0d3951600 con 0x55c0d394e780 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c27239740 10 client.0 _ll_register_callbacks cb 0x55c0d35b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.959+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d394e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d394e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42168/0 (socket says 172.21.15.64:42168) 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1697841163 learned_addr learned my addr 172.21.15.64:0/1697841163 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d3940b40 0x55c0d394b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d394b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 msgr2=0x55c0d394b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d394b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1697841163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d3940b40 msgr2=0x55c0d394b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d3940b40 0x55c0d394b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 -- 172.21.15.64:0/1697841163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d36163c0 con 0x55c0d393f5d0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d394e0c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5c100cfca0 tx=0x7f5c100a4960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2d4dc05fe2a9b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c05ffb640 1 -- 172.21.15.64:0/1697841163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c100b8070 con 0x55c0d393f5d0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c05ffb640 1 -- 172.21.15.64:0/1697841163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5c10098380 con 0x55c0d393f5d0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.960+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c100b1020 con 0x55c0d393f5d0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c05ffb640 1 -- 172.21.15.64:0/1697841163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5c100b4e60 con 0x55c0d393f5d0 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 10 client.15999 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:34.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c0d395ce70 con 0x55c0d393f5d0 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c05ffb640 1 -- 172.21.15.64:0/1697841163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c100b1020 con 0x55c0d393f5d0 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c05ffb640 1 -- 172.21.15.64:0/1697841163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5c1009abd0 con 0x55c0d393f5d0 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 10 client.15999 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 10 client.15999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 20 client.15999 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d395e1c0 0x55c0d39605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 4 client.15999 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d38b2860 con 0x55c0d395e1c0 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.961+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d395e1c0 0x55c0d39605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.962+0000 7f5c05ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:34.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.962+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d395e1c0 0x55c0d39605b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af85f026c69e2182 server_cookie=4d5e31811090a83f in_seq=0 out_seq=0 2024-09-19T05:08:34.971 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:34 smithi103 ceph-mon[34446]: pgmap v83: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c05ffb640 1 -- 172.21.15.64:0/1697841163 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0d38b2860 con 0x55c0d395e1c0 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c05ffb640 10 client.15999 handle_command_reply: tid=0 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 client.15999 shutdown 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 msgr2=0x55c0d394e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d394e0c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5c100cfca0 tx=0x7f5c100a4960 comp rx=0 tx=0).stop 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 shutdown_connections 2024-09-19T05:08:34.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d394e780 0x55c0d394b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d3940b40 0x55c0d394b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d395e1c0 0x55c0d39605b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 --2- 172.21.15.64:0/1697841163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d394e0c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 >> 172.21.15.64:0/1697841163 conn(0x55c0d3908310 msgr2=0x55c0d3941990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c05ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 shutdown_connections 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 1 -- 172.21.15.64:0/1697841163 wait complete. 2024-09-19T05:08:34.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.963+0000 7f5c27239740 20 client.15999 trim_cache size 0 max 16384 2024-09-19T05:08:34.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 Processor -- start 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 -- start start 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d393b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 0x55c0d3963730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201adec0 con 0x55c0d393f5d0 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201ab790 con 0x55c0d394e780 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.984+0000 7f5c27239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201a95e0 con 0x55c0d3940b40 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 0x55c0d3963730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d393b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 0x55c0d3963730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:43176/0 (socket says 172.21.15.64:43176) 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:38752/0 (socket says 172.21.15.64:38752) 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/389273007 learned_addr learned my addr 172.21.15.64:0/389273007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/389273007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 msgr2=0x55c0d39631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/389273007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 msgr2=0x55c0d393b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d393b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/389273007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d3940300 con 0x55c0d394e780 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:34.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d393b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.985+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 0x55c0d3963730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5c18003b30 tx=0x7f5c18009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4665a326dbdf2b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.986+0000 7f5c07fff640 1 -- 172.21.15.64:0/389273007 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18004b20 con 0x55c0d394e780 2024-09-19T05:08:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.986+0000 7f5c07fff640 1 -- 172.21.15.64:0/389273007 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c18004cc0 con 0x55c0d394e780 2024-09-19T05:08:34.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.986+0000 7f5c07fff640 1 -- 172.21.15.64:0/389273007 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18010720 con 0x55c0d394e780 2024-09-19T05:08:34.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 -- 172.21.15.64:0/389273007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 msgr2=0x55c0d3963730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 0x55c0d3963730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5c18003b30 tx=0x7f5c18009f90 comp rx=0 tx=0).stop 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 -- 172.21.15.64:0/389273007 shutdown_connections 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d394e780 0x55c0d3963730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 --2- 172.21.15.64:0/389273007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d393f5d0 0x55c0d393b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 -- 172.21.15.64:0/389273007 >> 172.21.15.64:0/389273007 conn(0x55c0d3908310 msgr2=0x55c0d3949c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 -- 172.21.15.64:0/389273007 shutdown_connections 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 -- 172.21.15.64:0/389273007 wait complete. 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 -- start start 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.987+0000 7f5c27239740 1 Processor -- start 2024-09-19T05:08:34.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d3963360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 0x55c0d3960920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201a9c50 con 0x55c0d394e780 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201ab0c0 con 0x55c0d393f5d0 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201aa320 con 0x55c0d3940b40 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c27239740 10 client.0 _ll_register_callbacks cb 0x7f5c10097920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 0x55c0d3960920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 0x55c0d3960920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42194/0 (socket says 172.21.15.64:42194) 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/2786553124 learned_addr learned my addr 172.21.15.64:0/2786553124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1dd9b640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d3963360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 msgr2=0x55c0d39638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/2786553124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 msgr2=0x55c0d3963360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.988+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d3963360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c1ed9d640 1 -- 172.21.15.64:0/2786553124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d3960ee0 con 0x55c0d394e780 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c1ed9d640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 0x55c0d3960920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005680 tx=0x7f5c180056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb4f56ae5e51107 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c05ffb640 1 -- 172.21.15.64:0/2786553124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1802a070 con 0x55c0d394e780 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c18004cc0 con 0x55c0d394e780 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c05ffb640 1 -- 172.21.15.64:0/2786553124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5c18004a40 con 0x55c0d394e780 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c05ffb640 1 -- 172.21.15.64:0/2786553124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5c18009da0 con 0x55c0d394e780 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c27239740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c0d3945bf0 con 0x55c0d394e780 2024-09-19T05:08:34.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.989+0000 7f5c05ffb640 1 -- 172.21.15.64:0/2786553124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c18004cc0 con 0x55c0d394e780 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c05ffb640 1 -- 172.21.15.64:0/2786553124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5c1802d020 con 0x55c0d394e780 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c27239740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c27239740 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c27239740 20 client.16005 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c27239740 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d39cd2e0 0x55c0d39cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c27239740 4 client.16005 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0d38acd70 con 0x55c0d39cd2e0 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d39cd2e0 0x55c0d39cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c1e59c640 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d39cd2e0 0x55c0d39cf6d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8774f1f27f9f5e16 server_cookie=1d36a65168e827ad in_seq=0 out_seq=0 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.990+0000 7f5c05ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c05ffb640 1 -- 172.21.15.64:0/2786553124 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0d38acd70 con 0x55c0d39cd2e0 2024-09-19T05:08:34.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c05ffb640 10 client.16005 handle_command_reply: tid=0 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 client.16005 shutdown 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 msgr2=0x55c0d3960920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 0x55c0d3960920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005680 tx=0x7f5c180056b0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 shutdown_connections 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c0d39cd2e0 0x55c0d39cf6d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d394e780 0x55c0d3960920 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d3940b40 0x55c0d39638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 --2- 172.21.15.64:0/2786553124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0d393f5d0 0x55c0d3963360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 >> 172.21.15.64:0/2786553124 conn(0x55c0d3908310 msgr2=0x55c0d394a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c05ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 shutdown_connections 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 1 -- 172.21.15.64:0/2786553124 wait complete. 2024-09-19T05:08:34.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.991+0000 7f5c27239740 20 client.16005 trim_cache size 0 max 16384 2024-09-19T05:08:34.999 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:08:34.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5c080a33b0 msgr2=0x7f5c080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:34.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5c080a33b0 0x7f5c080a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c14005ec0 tx=0x7f5c14005a20 comp rx=0 tx=0).stop 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20100e20 msgr2=0x7f5c20198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20100e20 0x7f5c20198420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c970 tx=0x7f5c1000ce30 comp rx=0 tx=0).stop 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 shutdown_connections 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5c080a33b0 0x7f5c080a5870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c20102ef0 0x7f5c2019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c20102570 0x7f5c20198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 --2- 172.21.15.64:0/1807925137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20100e20 0x7f5c20198420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 >> 172.21.15.64:0/1807925137 conn(0x7f5c200fc7d0 msgr2=0x7f5c20069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 shutdown_connections 2024-09-19T05:08:35.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:34.992+0000 7f5c24e60640 1 -- 172.21.15.64:0/1807925137 wait complete. 2024-09-19T05:08:35.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:34 smithi174 ceph-mon[34488]: pgmap v83: 97 pgs: 97 active+clean; 4.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-09-19T05:08:35.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:08:36.067 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 -- 172.21.15.64:0/180030683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 msgr2=0x7f6ba8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 --2- 172.21.15.64:0/180030683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba8107b80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009930 tx=0x7f6b9802a420 comp rx=0 tx=0).stop 2024-09-19T05:08:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 -- 172.21.15.64:0/180030683 shutdown_connections 2024-09-19T05:08:36.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 --2- 172.21.15.64:0/180030683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 0x7f6ba810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 --2- 172.21.15.64:0/180030683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba8107b80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 --2- 172.21.15.64:0/180030683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8102d90 0x7f6ba8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.571+0000 7f6bafb28640 1 -- 172.21.15.64:0/180030683 >> 172.21.15.64:0/180030683 conn(0x7f6ba80fc7d0 msgr2=0x7f6ba80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.572+0000 7f6bafb28640 1 -- 172.21.15.64:0/180030683 shutdown_connections 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.572+0000 7f6bafb28640 1 -- 172.21.15.64:0/180030683 wait complete. 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.572+0000 7f6bafb28640 1 Processor -- start 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.572+0000 7f6bafb28640 1 -- start start 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bafb28640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8102d90 0x7f6ba8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bafb28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bafb28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 0x7f6ba819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bafb28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8111f40 con 0x7f6ba8103740 2024-09-19T05:08:36.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bafb28640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8111dc0 con 0x7f6ba8108290 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bafb28640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81120c0 con 0x7f6ba8102d90 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42206/0 (socket says 172.21.15.64:42206) 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 -- 172.21.15.64:0/1502325784 learned_addr learned my addr 172.21.15.64:0/1502325784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 0x7f6ba819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 -- 172.21.15.64:0/1502325784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8102d90 msgr2=0x7f6ba8198470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8102d90 0x7f6ba8198470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 -- 172.21.15.64:0/1502325784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 msgr2=0x7f6ba819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 0x7f6ba819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 -- 172.21.15.64:0/1502325784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba819d420 con 0x7f6ba8103740 2024-09-19T05:08:36.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 0x7f6ba819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.573+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba81989b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6b98031440 tx=0x7f6b98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.574+0000 7f6b9effd640 1 -- 172.21.15.64:0/1502325784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9803f030 con 0x7f6ba8103740 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.574+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba819d6b0 con 0x7f6ba8103740 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.574+0000 7f6b9effd640 1 -- 172.21.15.64:0/1502325784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b98003aa0 con 0x7f6ba8103740 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.574+0000 7f6b9effd640 1 -- 172.21.15.64:0/1502325784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98035070 con 0x7f6ba8103740 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.574+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a4ff0 con 0x7f6ba8103740 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.576+0000 7f6b9effd640 1 -- 172.21.15.64:0/1502325784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6b98042030 con 0x7f6ba8103740 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 Processor -- start 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 -- start start 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3aab40 0x55558f3a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 0x55558f3b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55558f30f2f0 con 0x55558f3b4370 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55558f2b6cf0 con 0x55558f3a95d0 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55558f30b2b0 con 0x55558f3aab40 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 0x55558f3b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6b9effd640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b940a33b0 0x7f6b940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 0x55558f3b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42208/0 (socket says 172.21.15.64:42208) 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bad0c3640 1 -- 172.21.15.64:0/2794110208 learned_addr learned my addr 172.21.15.64:0/2794110208 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.577+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6b9effd640 1 -- 172.21.15.64:0/1502325784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6b980eff10 con 0x7f6ba8103740 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3aab40 0x55558f3a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b940a33b0 0x7f6b940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad8c4640 1 -- 172.21.15.64:0/2794110208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3aab40 msgr2=0x55558f3a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3aab40 0x55558f3a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad8c4640 1 -- 172.21.15.64:0/2794110208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 msgr2=0x55558f3b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 0x55558f3b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad8c4640 1 -- 172.21.15.64:0/2794110208 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558f29b3a0 con 0x55558f3a95d0 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 0x55558f3b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a9a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b9d0 tx=0x7f6ba400be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b121e7ab90fe15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b940a33b0 0x7f6b940a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00124b0 tx=0x7f6ba0005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.578+0000 7f6b93fff640 1 -- 172.21.15.64:0/2794110208 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4016070 con 0x55558f3a95d0 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6b93fff640 1 -- 172.21.15.64:0/2794110208 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ba40054d0 con 0x55558f3a95d0 2024-09-19T05:08:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6b93fff640 1 -- 172.21.15.64:0/2794110208 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4010460 con 0x55558f3a95d0 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2794110208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 msgr2=0x55558f3a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a9a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b9d0 tx=0x7f6ba400be90 comp rx=0 tx=0).stop 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2794110208 shutdown_connections 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3b4370 0x55558f3b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a9a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2794110208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3aab40 0x55558f3a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.579+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2794110208 >> 172.21.15.64:0/2794110208 conn(0x55558f372310 msgr2=0x55558f3a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.580+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2794110208 shutdown_connections 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.580+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2794110208 wait complete. 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.580+0000 7f6bb1f01740 1 -- start start 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.580+0000 7f6bb1f01740 1 Processor -- start 2024-09-19T05:08:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3a95d0 0x55558f3b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3b4370 0x55558f3b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55558f3b7070 con 0x55558f3aab40 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55558f2ca060 con 0x55558f3b4370 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55558f3b71f0 con 0x55558f3a95d0 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bb1f01740 10 client.0 _ll_register_callbacks cb 0x55558f01d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42210/0 (socket says 172.21.15.64:42210) 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 -- 172.21.15.64:0/924791023 learned_addr learned my addr 172.21.15.64:0/924791023 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3a95d0 0x55558f3b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3b4370 0x55558f3b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 -- 172.21.15.64:0/924791023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3a95d0 msgr2=0x55558f3b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3a95d0 0x55558f3b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 -- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3b4370 msgr2=0x55558f3b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3b4370 0x55558f3b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 -- 172.21.15.64:0/924791023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558f0803c0 con 0x55558f3aab40 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3b0f90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b8bc0 tx=0x7f6b980bbe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc0c9ffc11bd7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.581+0000 7f6b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6b917fa640 1 -- 172.21.15.64:0/924791023 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98049070 con 0x55558f3aab40 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6b917fa640 1 -- 172.21.15.64:0/924791023 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6b980b8860 con 0x55558f3aab40 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba40054d0 con 0x55558f3aab40 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6b917fa640 1 -- 172.21.15.64:0/924791023 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b980dae60 con 0x55558f3aab40 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 10 client.16017 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55558f3c2a80 con 0x55558f3aab40 2024-09-19T05:08:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6b917fa640 1 -- 172.21.15.64:0/924791023 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ba40054d0 con 0x55558f3aab40 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6b917fa640 1 -- 172.21.15.64:0/924791023 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b9804c060 con 0x55558f3aab40 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 10 client.16017 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 20 client.16017 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f3c3f70 0x55558f3c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 4 client.16017 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.582+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55558f31c860 con 0x55558f3c3f70 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.583+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f3c3f70 0x55558f3c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.583+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f3c3f70 0x55558f3c6380 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da773652a7459d93 server_cookie=3a0deb62e77a7277 in_seq=0 out_seq=0 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.583+0000 7f6b917fa640 10 client.16017 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6b917fa640 1 -- 172.21.15.64:0/924791023 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55558f31c860 con 0x55558f3c3f70 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6b917fa640 10 client.16017 handle_command_reply: tid=0 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 client.16017 shutdown 2024-09-19T05:08:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 msgr2=0x55558f3b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3b0f90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b8bc0 tx=0x7f6b980bbe60 comp rx=0 tx=0).stop 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 shutdown_connections 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3b4370 0x55558f3b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f3c3f70 0x55558f3c6380 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3b0f90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/924791023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3a95d0 0x55558f3b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6b917fa640 0 client.16017 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 >> 172.21.15.64:0/924791023 conn(0x55558f372310 msgr2=0x55558f3ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 shutdown_connections 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 1 -- 172.21.15.64:0/924791023 wait complete. 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.584+0000 7f6bb1f01740 20 client.16017 trim_cache size 0 max 16384 2024-09-19T05:08:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 Processor -- start 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 -- start start 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b0160 con 0x55558f3aab40 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ada30 con 0x55558f3a95d0 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ab880 con 0x55558f3b4370 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bad0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3c0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42216/0 (socket says 172.21.15.64:42216) 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.599+0000 7f6bad0c3640 1 -- 172.21.15.64:0/2879638489 learned_addr learned my addr 172.21.15.64:0/2879638489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad0c3640 1 -- 172.21.15.64:0/2879638489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 msgr2=0x55558f3b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad0c3640 1 -- 172.21.15.64:0/2879638489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 msgr2=0x55558f3a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad0c3640 1 -- 172.21.15.64:0/2879638489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558f3ac120 con 0x55558f3aab40 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3c0310 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6b980ed710 tx=0x7f6b98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8765f1e74e437833 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:36.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6b937fe640 1 -- 172.21.15.64:0/2879638489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98049070 con 0x55558f3aab40 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6b937fe640 1 -- 172.21.15.64:0/2879638489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b980c4e60 con 0x55558f3aab40 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.600+0000 7f6b937fe640 1 -- 172.21.15.64:0/2879638489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980c0100 con 0x55558f3aab40 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2879638489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 msgr2=0x55558f3c0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3c0310 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6b980ed710 tx=0x7f6b98000bc0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2879638489 shutdown_connections 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3aab40 0x55558f3c0310 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/2879638489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3a95d0 0x55558f3a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2879638489 >> 172.21.15.64:0/2879638489 conn(0x55558f372310 msgr2=0x55558f3a5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:36.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2879638489 shutdown_connections 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.601+0000 7f6bb1f01740 1 -- 172.21.15.64:0/2879638489 wait complete. 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 -- start start 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 Processor -- start 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 0x55558f3adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3aab40 0x55558f3ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81abef0 con 0x55558f3a95d0 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ad360 con 0x55558f3aab40 2024-09-19T05:08:36.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ac5c0 con 0x55558f3b4370 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bb1f01740 10 client.0 _ll_register_callbacks cb 0x7f6b98009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bad8c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 0x55558f3adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bad8c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 0x55558f3adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42228/0 (socket says 172.21.15.64:42228) 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bad8c4640 1 -- 172.21.15.64:0/3715169097 learned_addr learned my addr 172.21.15.64:0/3715169097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3aab40 0x55558f3ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.602+0000 7f6bae0c5640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bad8c4640 1 -- 172.21.15.64:0/3715169097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 msgr2=0x55558f3aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bad8c4640 1 -- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3aab40 msgr2=0x55558f3ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3aab40 0x55558f3ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bad8c4640 1 -- 172.21.15.64:0/3715169097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558f42dc30 con 0x55558f3a95d0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bad8c4640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 0x55558f3adfa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b9d0 tx=0x7f6ba4004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4380623c358f875 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6b917fa640 1 -- 172.21.15.64:0/3715169097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba402b020 con 0x55558f3a95d0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b980c4e60 con 0x55558f3a95d0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6b917fa640 1 -- 172.21.15.64:0/3715169097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6ba400ddf0 con 0x55558f3a95d0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6b917fa640 1 -- 172.21.15.64:0/3715169097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ba4036050 con 0x55558f3a95d0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bb1f01740 10 client.16023 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.603+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55558f42eef0 con 0x55558f3a95d0 2024-09-19T05:08:36.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6b917fa640 1 -- 172.21.15.64:0/3715169097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b980c4e60 con 0x55558f3a95d0 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6b917fa640 1 -- 172.21.15.64:0/3715169097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ba4003bd0 con 0x55558f3a95d0 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bb1f01740 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bb1f01740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bb1f01740 20 client.16023 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f42f390 0x55558f4317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bb1f01740 4 client.16023 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55558f316d70 con 0x55558f42f390 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.604+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f42f390 0x55558f4317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bad0c3640 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f42f390 0x55558f4317b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb3d537721bd42 server_cookie=804d7696ba159a27 in_seq=0 out_seq=0 2024-09-19T05:08:36.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6b917fa640 10 client.16023 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:36.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6b917fa640 1 -- 172.21.15.64:0/3715169097 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55558f316d70 con 0x55558f42f390 2024-09-19T05:08:36.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6b917fa640 10 client.16023 handle_command_reply: tid=0 2024-09-19T05:08:36.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 client.16023 shutdown 2024-09-19T05:08:36.756 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 msgr2=0x55558f3adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 0x55558f3adfa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b9d0 tx=0x7f6ba4004840 comp rx=0 tx=0).stop 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 shutdown_connections 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558f3b4370 0x55558f3aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558f3aab40 0x55558f3ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55558f42f390 0x55558f4317b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 --2- 172.21.15.64:0/3715169097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558f3a95d0 0x55558f3adfa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6b917fa640 0 client.16023 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 >> 172.21.15.64:0/3715169097 conn(0x55558f372310 msgr2=0x55558f3af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 shutdown_connections 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.605+0000 7f6bb1f01740 1 -- 172.21.15.64:0/3715169097 wait complete. 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bb1f01740 20 client.16023 trim_cache size 0 max 16384 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b940a33b0 msgr2=0x7f6b940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b940a33b0 0x7f6b940a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00124b0 tx=0x7f6ba0005f90 comp rx=0 tx=0).stop 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 msgr2=0x7f6ba81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba81989b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6b98031440 tx=0x7f6b98004270 comp rx=0 tx=0).stop 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 shutdown_connections 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b940a33b0 0x7f6b940a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8108290 0x7f6ba819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba8103740 0x7f6ba81989b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 --2- 172.21.15.64:0/1502325784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8102d90 0x7f6ba8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.606+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 >> 172.21.15.64:0/1502325784 conn(0x7f6ba80fc7d0 msgr2=0x7f6ba8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.607+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 shutdown_connections 2024-09-19T05:08:36.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:36.607+0000 7f6bafb28640 1 -- 172.21.15.64:0/1502325784 wait complete. 2024-09-19T05:08:37.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:36 smithi064 ceph-mon[30960]: pgmap v84: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 133 KiB/s wr, 9 op/s 2024-09-19T05:08:37.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:36 smithi103 ceph-mon[34446]: pgmap v84: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 133 KiB/s wr, 9 op/s 2024-09-19T05:08:37.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:36 smithi174 ceph-mon[34488]: pgmap v84: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 133 KiB/s wr, 9 op/s 2024-09-19T05:08:38.540 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:08:38.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:08:38.824 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:38.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:38 smithi064 ceph-mon[30960]: pgmap v85: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 133 KiB/s wr, 6 op/s 2024-09-19T05:08:39.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:38 smithi103 ceph-mon[34446]: pgmap v85: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 133 KiB/s wr, 6 op/s 2024-09-19T05:08:39.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:38 smithi174 ceph-mon[34488]: pgmap v85: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 133 KiB/s wr, 6 op/s 2024-09-19T05:08:39.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 -- 172.21.15.64:0/4109384353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 msgr2=0x7fa63410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 --2- 172.21.15.64:0/4109384353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa63410ea90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009ca0 tx=0x7fa62c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 -- 172.21.15.64:0/4109384353 shutdown_connections 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 --2- 172.21.15.64:0/4109384353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa63410ea90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 --2- 172.21.15.64:0/4109384353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa634102790 0x7fa6341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 --2- 172.21.15.64:0/4109384353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634101de0 0x7fa6341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 -- 172.21.15.64:0/4109384353 >> 172.21.15.64:0/4109384353 conn(0x7fa6340fd8c0 msgr2=0x7fa6340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 -- 172.21.15.64:0/4109384353 shutdown_connections 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 -- 172.21.15.64:0/4109384353 wait complete. 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.324+0000 7fa639c65640 1 Processor -- start 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 -- start start 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634101de0 0x7fa6340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa634102790 0x7fa634074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa634074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa634111e20 con 0x7fa634106f10 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa634111ca0 con 0x7fa634102790 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa639c65640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa634111fa0 con 0x7fa634101de0 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa633fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa634074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa633fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa634074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42238/0 (socket says 172.21.15.64:42238) 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa633fff640 1 -- 172.21.15.64:0/1037736570 learned_addr learned my addr 172.21.15.64:0/1037736570 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa6337fe640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634101de0 0x7fa6340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa632ffd640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa634102790 0x7fa634074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa633fff640 1 -- 172.21.15.64:0/1037736570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634101de0 msgr2=0x7fa6340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa633fff640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634101de0 0x7fa6340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.325+0000 7fa633fff640 1 -- 172.21.15.64:0/1037736570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa634102790 msgr2=0x7fa634074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa633fff640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa634102790 0x7fa634074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa633fff640 1 -- 172.21.15.64:0/1037736570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341a6800 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa633fff640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa634074f70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009dc0 tx=0x7fa62c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa630ff9640 1 -- 172.21.15.64:0/1037736570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c003bf0 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa630ff9640 1 -- 172.21.15.64:0/1037736570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa62c003d90 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a6a90 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa630ff9640 1 -- 172.21.15.64:0/1037736570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c030b80 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.326+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a6fd0 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.329+0000 7fa630ff9640 1 -- 172.21.15.64:0/1037736570 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa62c004040 con 0x7fa634106f10 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.329+0000 7fa63c03e740 1 Processor -- start 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.329+0000 7fa63c03e740 1 -- start start 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.329+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e0097f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.329+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e00a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 0x55c7e00a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa63c03e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c7dfffd280 con 0x55c7e00a2d40 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa63c03e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7dffa4b10 con 0x55c7e0097b60 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa63c03e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7dfff9240 con 0x55c7e0098480 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 0x55c7e00a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 0x55c7e00a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42240/0 (socket says 172.21.15.64:42240) 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 -- 172.21.15.64:0/1085681875 learned_addr learned my addr 172.21.15.64:0/1085681875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e0097f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa630ff9640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa61c0a33b0 0x7fa61c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa6337fe640 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e00a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 -- 172.21.15.64:0/1085681875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 msgr2=0x55c7e00a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e00a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 -- 172.21.15.64:0/1085681875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 msgr2=0x55c7e0097f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e0097f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 -- 172.21.15.64:0/1085681875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7dffb11f0 con 0x55c7e00a2d40 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa630ff9640 1 -- 172.21.15.64:0/1037736570 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa62c0f1200 con 0x7fa634106f10 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa633fff640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa61c0a33b0 0x7fa61c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa632ffd640 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 0x55c7e00a5130 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa628009730 tx=0x7fa6280096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c3a7ea157c0009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.330+0000 7fa619ffb640 1 -- 172.21.15.64:0/1085681875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628004050 con 0x55c7e00a2d40 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa619ffb640 1 -- 172.21.15.64:0/1085681875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa62800fc70 con 0x55c7e00a2d40 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa619ffb640 1 -- 172.21.15.64:0/1085681875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280039f0 con 0x55c7e00a2d40 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 -- 172.21.15.64:0/1085681875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 msgr2=0x55c7e00a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa633fff640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa61c0a33b0 0x7fa61c0a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0eeb10 tx=0x7fa62c0095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 0x55c7e00a5130 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa628009730 tx=0x7fa6280096c0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 -- 172.21.15.64:0/1085681875 shutdown_connections 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e00a2d40 0x55c7e00a5130 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e00a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 --2- 172.21.15.64:0/1085681875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e0097f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 -- 172.21.15.64:0/1085681875 >> 172.21.15.64:0/1085681875 conn(0x55c7e00940f0 msgr2=0x55c7e00944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.331+0000 7fa63c03e740 1 -- 172.21.15.64:0/1085681875 shutdown_connections 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.332+0000 7fa63c03e740 1 -- 172.21.15.64:0/1085681875 wait complete. 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.332+0000 7fa63c03e740 1 -- start start 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.332+0000 7fa63c03e740 1 Processor -- start 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.332+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e009fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.332+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e00a2d40 0x55c7e00a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa63c03e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c7e00a5910 con 0x55c7e0098480 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa63c03e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7dffb7f10 con 0x55c7e0097b60 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa63c03e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7e00a5a90 con 0x55c7e00a2d40 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa63c03e740 10 client.0 _ll_register_callbacks cb 0x55c7dff80a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42252/0 (socket says 172.21.15.64:42252) 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 -- 172.21.15.64:0/3411015444 learned_addr learned my addr 172.21.15.64:0/3411015444 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e009fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e00a2d40 0x55c7e00a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 -- 172.21.15.64:0/3411015444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e00a2d40 msgr2=0x55c7e00a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e00a2d40 0x55c7e00a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 -- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 msgr2=0x55c7e009fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e009fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 -- 172.21.15.64:0/3411015444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7dfd6e6a0 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e00a2d40 0x55c7e00a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa6337fe640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00a0150 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b950 tx=0x7fa62400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4c50e8bb7a10d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa612ffd640 1 -- 172.21.15.64:0/3411015444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624003ea0 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa612ffd640 1 -- 172.21.15.64:0/3411015444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa62401b640 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.333+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62800fc70 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa612ffd640 1 -- 172.21.15.64:0/3411015444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa62401b840 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 10 client.16035 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7e00b15a0 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa612ffd640 1 -- 172.21.15.64:0/3411015444 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa62800fc70 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa612ffd640 1 -- 172.21.15.64:0/3411015444 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa624019020 con 0x55c7e0098480 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 10 client.16035 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:39.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 20 client.16035 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e00b29b0 0x55c7e00b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 4 client.16035 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7e000a7f0 con 0x55c7e00b29b0 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.334+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e00b29b0 0x55c7e00b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.335+0000 7fa612ffd640 10 client.16035 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.335+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e00b29b0 0x55c7e00b4e00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c53fbf5aa5e170 server_cookie=f6613577f679ab8c in_seq=0 out_seq=0 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa612ffd640 1 -- 172.21.15.64:0/3411015444 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7e000a7f0 con 0x55c7e00b29b0 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa612ffd640 10 client.16035 handle_command_reply: tid=0 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 client.16035 shutdown 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 msgr2=0x55c7e00a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00a0150 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b950 tx=0x7fa62400be10 comp rx=0 tx=0).stop 2024-09-19T05:08:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 shutdown_connections 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e00a2d40 0x55c7e00a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e00b29b0 0x55c7e00b4e00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa612ffd640 0 client.16035 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00a0150 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3411015444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e0097b60 0x55c7e009fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 >> 172.21.15.64:0/3411015444 conn(0x55c7e00940f0 msgr2=0x55c7e0093270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 shutdown_connections 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 1 -- 172.21.15.64:0/3411015444 wait complete. 2024-09-19T05:08:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.336+0000 7fa63c03e740 20 client.16035 trim_cache size 0 max 16384 2024-09-19T05:08:39.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.364+0000 7fa63c03e740 1 Processor -- start 2024-09-19T05:08:39.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.364+0000 7fa63c03e740 1 -- start start 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0097b60 0x55c7e00ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e0092a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa63c03e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b2290 con 0x55c7e0098480 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa63c03e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341afb60 con 0x55c7e00a2d40 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa63c03e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341ad9b0 con 0x55c7e0097b60 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42260/0 (socket says 172.21.15.64:42260) 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 -- 172.21.15.64:0/615505460 learned_addr learned my addr 172.21.15.64:0/615505460 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:39.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa632ffd640 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0097b60 0x55c7e00ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa632ffd640 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e0092a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 -- 172.21.15.64:0/615505460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0097b60 msgr2=0x55c7e00ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0097b60 0x55c7e00ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 -- 172.21.15.64:0/615505460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 msgr2=0x55c7e0092a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e0092a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa6337fe640 1 -- 172.21.15.64:0/615505460 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e008e680 con 0x55c7e0098480 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.365+0000 7fa632ffd640 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e0092a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.366+0000 7fa6337fe640 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00bac10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa624004300 tx=0x7fa624005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765a63bef72d53db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.366+0000 7fa6197fa640 1 -- 172.21.15.64:0/615505460 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62401b640 con 0x55c7e0098480 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.366+0000 7fa6197fa640 1 -- 172.21.15.64:0/615505460 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6240039c0 con 0x55c7e0098480 2024-09-19T05:08:39.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.366+0000 7fa6197fa640 1 -- 172.21.15.64:0/615505460 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624005290 con 0x55c7e0098480 2024-09-19T05:08:39.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 -- 172.21.15.64:0/615505460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 msgr2=0x55c7e00bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00bac10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa624004300 tx=0x7fa624005990 comp rx=0 tx=0).stop 2024-09-19T05:08:39.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 -- 172.21.15.64:0/615505460 shutdown_connections 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e0092a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0098480 0x55c7e00bac10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 --2- 172.21.15.64:0/615505460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0097b60 0x55c7e00ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 -- 172.21.15.64:0/615505460 >> 172.21.15.64:0/615505460 conn(0x55c7e00940f0 msgr2=0x55c7e00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 -- 172.21.15.64:0/615505460 shutdown_connections 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 -- 172.21.15.64:0/615505460 wait complete. 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 -- start start 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.367+0000 7fa63c03e740 1 Processor -- start 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 0x55c7e00ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e009cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e009d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341ae020 con 0x55c7e0097b60 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341af490 con 0x55c7e00a2d40 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341ae6f0 con 0x55c7e0098480 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa63c03e740 10 client.0 _ll_register_callbacks cb 0x7fa624004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 0x55c7e00ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 0x55c7e00ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42262/0 (socket says 172.21.15.64:42262) 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 -- 172.21.15.64:0/3517740406 learned_addr learned my addr 172.21.15.64:0/3517740406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 -- 172.21.15.64:0/3517740406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 msgr2=0x55c7e009cb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e009cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 -- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 msgr2=0x55c7e009d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e009d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 -- 172.21.15.64:0/3517740406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e011c6a0 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa632ffd640 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 0x55c7e00ab5f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa628009700 tx=0x7fa628009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9208f5884c3e344c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.368+0000 7fa612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa612ffd640 1 -- 172.21.15.64:0/3517740406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62802d020 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa612ffd640 1 -- 172.21.15.64:0/3517740406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa62800f990 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6240039c0 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa612ffd640 1 -- 172.21.15.64:0/3517740406 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa628038050 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 10 client.16041 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7e011d880 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa612ffd640 1 -- 172.21.15.64:0/3517740406 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6240039c0 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa612ffd640 1 -- 172.21.15.64:0/3517740406 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa628004980 con 0x55c7e0097b60 2024-09-19T05:08:39.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 20 client.16041 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e011dec0 0x55c7e01202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 4 client.16041 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.369+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c7e0004d00 con 0x55c7e011dec0 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.370+0000 7fa6337fe640 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e011dec0 0x55c7e01202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.370+0000 7fa612ffd640 10 client.16041 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.370+0000 7fa6337fe640 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e011dec0 0x55c7e01202e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c664b0dddf9fd484 server_cookie=ef379d289b82eeb8 in_seq=0 out_seq=0 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa612ffd640 1 -- 172.21.15.64:0/3517740406 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c7e0004d00 con 0x55c7e011dec0 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa612ffd640 10 client.16041 handle_command_reply: tid=0 2024-09-19T05:08:39.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 client.16041 shutdown 2024-09-19T05:08:39.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 msgr2=0x55c7e00ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 0x55c7e00ab5f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa628009700 tx=0x7fa628009bd0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 shutdown_connections 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7e00a2d40 0x55c7e009d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7e0098480 0x55c7e009cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c7e011dec0 0x55c7e01202e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 --2- 172.21.15.64:0/3517740406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7e0097b60 0x55c7e00ab5f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 >> 172.21.15.64:0/3517740406 conn(0x55c7e00940f0 msgr2=0x55c7e009fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.371+0000 7fa612ffd640 0 client.16041 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 shutdown_connections 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa63c03e740 1 -- 172.21.15.64:0/3517740406 wait complete. 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa63c03e740 20 client.16041 trim_cache size 0 max 16384 2024-09-19T05:08:39.378 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:08:39.379 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:08:39.379 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "bdb337a1-bb6e-438f-aad0-30933efdd7a1", 2024-09-19T05:08:39.379 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:08:39.379 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:08:39.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa61c0a33b0 msgr2=0x7fa61c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa61c0a33b0 0x7fa61c0a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0eeb10 tx=0x7fa62c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 msgr2=0x7fa634074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa634074f70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009dc0 tx=0x7fa62c032ff0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 shutdown_connections 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa61c0a33b0 0x7fa61c0a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa634106f10 0x7fa634074f70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa634102790 0x7fa634074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.372+0000 7fa639c65640 1 --2- 172.21.15.64:0/1037736570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634101de0 0x7fa6340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.373+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 >> 172.21.15.64:0/1037736570 conn(0x7fa6340fd8c0 msgr2=0x7fa6340fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.373+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 shutdown_connections 2024-09-19T05:08:39.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:39.373+0000 7fa639c65640 1 -- 172.21.15.64:0/1037736570 wait complete. 2024-09-19T05:08:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:39 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:08:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:39 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:40.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:39 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:08:40.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:39 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:40.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:08:40.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:39 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:08:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:39 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:39 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:08:40.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:39 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:40.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:39 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:08:40.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:39 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:40.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:39 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:08:40.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:39 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:40.417 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 -- 172.21.15.64:0/649244402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50108300 msgr2=0x7fca5010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 --2- 172.21.15.64:0/649244402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50108300 0x7fca5010ec70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fca48009cd0 tx=0x7fca4802a8d0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 -- 172.21.15.64:0/649244402 shutdown_connections 2024-09-19T05:08:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 --2- 172.21.15.64:0/649244402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50108300 0x7fca5010ec70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 --2- 172.21.15.64:0/649244402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50103780 0x7fca50107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 --2- 172.21.15.64:0/649244402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca50102dd0 0x7fca501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 -- 172.21.15.64:0/649244402 >> 172.21.15.64:0/649244402 conn(0x7fca500fc7d0 msgr2=0x7fca500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 -- 172.21.15.64:0/649244402 shutdown_connections 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.936+0000 7fca56528640 1 -- 172.21.15.64:0/649244402 wait complete. 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.937+0000 7fca56528640 1 Processor -- start 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.937+0000 7fca56528640 1 -- start start 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.937+0000 7fca56528640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca50102dd0 0x7fca50100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.937+0000 7fca56528640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 0x7fca50100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.937+0000 7fca56528640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50108300 0x7fca501014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.937+0000 7fca56528640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111fc0 con 0x7fca50103780 2024-09-19T05:08:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca56528640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca50111e40 con 0x7fca50108300 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca56528640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca50112140 con 0x7fca50102dd0 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 0x7fca50100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 0x7fca50100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42280/0 (socket says 172.21.15.64:42280) 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 -- 172.21.15.64:0/1671826905 learned_addr learned my addr 172.21.15.64:0/1671826905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4ffff640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca50102dd0 0x7fca50100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca54ac5640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50108300 0x7fca501014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 -- 172.21.15.64:0/1671826905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca50102dd0 msgr2=0x7fca50100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca50102dd0 0x7fca50100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 -- 172.21.15.64:0/1671826905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50108300 msgr2=0x7fca501014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50108300 0x7fca501014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 -- 172.21.15.64:0/1671826905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca501a6a40 con 0x7fca50103780 2024-09-19T05:08:40.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 0x7fca50100f70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca4400d950 tx=0x7fca4400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4d7fa640 1 -- 172.21.15.64:0/1671826905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44017070 con 0x7fca50103780 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4d7fa640 1 -- 172.21.15.64:0/1671826905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44009c30 con 0x7fca50103780 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca4d7fa640 1 -- 172.21.15.64:0/1671826905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440055e0 con 0x7fca50103780 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.938+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca501a6d30 con 0x7fca50103780 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.939+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a7210 con 0x7fca50103780 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.941+0000 7fca4d7fa640 1 -- 172.21.15.64:0/1671826905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fca44004020 con 0x7fca50103780 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 Processor -- start 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 -- start start 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca4d7fa640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca380a33b0 0x7fca380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x562136825880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x562136830140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136832a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56213678b2e0 con 0x562136830680 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562136732ce0 con 0x5621368254a0 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca58901740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621367872a0 con 0x562136825dc0 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca54ac5640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca380a33b0 0x7fca380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca54ac5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136832a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca54ac5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136832a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42294/0 (socket says 172.21.15.64:42294) 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca54ac5640 1 -- 172.21.15.64:0/3819959420 learned_addr learned my addr 172.21.15.64:0/3819959420 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca4d7fa640 1 -- 172.21.15.64:0/1671826905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fca440cf960 con 0x7fca50103780 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x562136830140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.942+0000 7fca4ffff640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x562136825880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4f7fe640 1 -- 172.21.15.64:0/3819959420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 msgr2=0x562136825880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x562136825880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4f7fe640 1 -- 172.21.15.64:0/3819959420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 msgr2=0x562136832a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136832a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4f7fe640 1 -- 172.21.15.64:0/3819959420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562136717440 con 0x562136825dc0 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca54ac5640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136832a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4ffff640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x562136825880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca54ac5640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca380a33b0 0x7fca380a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fca48009cd0 tx=0x7fca48009880 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x562136830140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca440cd0b0 tx=0x7fca4409bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f767314b5211b23c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca367fc640 1 -- 172.21.15.64:0/3819959420 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440c8160 con 0x562136825dc0 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca367fc640 1 -- 172.21.15.64:0/3819959420 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44009de0 con 0x562136825dc0 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.943+0000 7fca367fc640 1 -- 172.21.15.64:0/3819959420 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44004310 con 0x562136825dc0 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 -- 172.21.15.64:0/3819959420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 msgr2=0x562136830140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x562136830140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca440cd0b0 tx=0x7fca4409bb40 comp rx=0 tx=0).stop 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 -- 172.21.15.64:0/3819959420 shutdown_connections 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136832a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x562136830140 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 --2- 172.21.15.64:0/3819959420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x562136825880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 -- 172.21.15.64:0/3819959420 >> 172.21.15.64:0/3819959420 conn(0x562136821a30 msgr2=0x562136821e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 -- 172.21.15.64:0/3819959420 shutdown_connections 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 -- 172.21.15.64:0/3819959420 wait complete. 2024-09-19T05:08:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.944+0000 7fca58901740 1 -- start start 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 Processor -- start 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213682ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x56213682d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x56213682d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562136833250 con 0x5621368254a0 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562136746050 con 0x562136825dc0 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621368333d0 con 0x562136830680 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca58901740 10 client.0 _ll_register_callbacks cb 0x562136786160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:40.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213682ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213682ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42302/0 (socket says 172.21.15.64:42302) 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 -- 172.21.15.64:0/2692490061 learned_addr learned my addr 172.21.15.64:0/2692490061 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x56213682d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca54ac5640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x56213682d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 -- 172.21.15.64:0/2692490061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 msgr2=0x56213682d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x56213682d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 -- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 msgr2=0x56213682d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x56213682d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.945+0000 7fca4ffff640 1 -- 172.21.15.64:0/2692490061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621364fc3c0 con 0x5621368254a0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca4ffff640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213682ffa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fca400115e0 tx=0x7fca40011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45a30c0c9096c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2692490061 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4000f750 con 0x5621368254a0 2024-09-19T05:08:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2692490061 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fca4000f8f0 con 0x5621368254a0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca44009de0 con 0x5621368254a0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2692490061 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca4000faf0 con 0x5621368254a0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca58901740 10 client.16053 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56213683ec80 con 0x5621368254a0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.946+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2692490061 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca44009de0 con 0x5621368254a0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2692490061 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca4001d400 con 0x5621368254a0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca58901740 10 client.16053 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca58901740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca58901740 20 client.16053 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca58901740 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562136840140 0x562136842530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca58901740 4 client.16053 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562136798850 con 0x562136840140 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.947+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562136840140 0x562136842530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.948+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562136840140 0x562136842530 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9487ce6b6de51179 server_cookie=f4621e6559ee86a7 in_seq=0 out_seq=0 2024-09-19T05:08:40.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.948+0000 7fca2f7fe640 10 client.16053 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.948+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2692490061 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562136798850 con 0x562136840140 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.948+0000 7fca2f7fe640 10 client.16053 handle_command_reply: tid=0 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.948+0000 7fca58901740 1 client.16053 shutdown 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 msgr2=0x56213682ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213682ffa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fca400115e0 tx=0x7fca40011aa0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 shutdown_connections 2024-09-19T05:08:40.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x56213682d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x56213682d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562136840140 0x562136842530 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 --2- 172.21.15.64:0/2692490061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213682ffa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 >> 172.21.15.64:0/2692490061 conn(0x562136821a30 msgr2=0x562136827760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca2f7fe640 0 client.16053 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 shutdown_connections 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 1 -- 172.21.15.64:0/2692490061 wait complete. 2024-09-19T05:08:40.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.949+0000 7fca58901740 20 client.16053 trim_cache size 0 max 16384 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 Processor -- start 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 -- start start 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213683c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x562136827600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x562136847e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca501b2350 con 0x5621368254a0 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca501afc20 con 0x562136825dc0 2024-09-19T05:08:40.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.977+0000 7fca58901740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca501ada70 con 0x562136830680 2024-09-19T05:08:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213683c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213683c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42318/0 (socket says 172.21.15.64:42318) 2024-09-19T05:08:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 -- 172.21.15.64:0/1679361649 learned_addr learned my addr 172.21.15.64:0/1679361649 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4f7fe640 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x562136827600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca54ac5640 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x562136847e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 -- 172.21.15.64:0/1679361649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 msgr2=0x562136847e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x562136847e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 -- 172.21.15.64:0/1679361649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 msgr2=0x562136827600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x562136827600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 -- 172.21.15.64:0/1679361649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621368302d0 con 0x5621368254a0 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca54ac5640 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x562136847e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca4ffff640 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213683c4b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca40009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468c142e12fbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca35ffb640 1 -- 172.21.15.64:0/1679361649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4002a070 con 0x5621368254a0 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.978+0000 7fca35ffb640 1 -- 172.21.15.64:0/1679361649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca4001d400 con 0x5621368254a0 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca35ffb640 1 -- 172.21.15.64:0/1679361649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4002e900 con 0x5621368254a0 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 -- 172.21.15.64:0/1679361649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 msgr2=0x56213683c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213683c4b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca40009f50 comp rx=0 tx=0).stop 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 -- 172.21.15.64:0/1679361649 shutdown_connections 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136830680 0x562136847e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562136825dc0 0x562136827600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 --2- 172.21.15.64:0/1679361649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621368254a0 0x56213683c4b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 -- 172.21.15.64:0/1679361649 >> 172.21.15.64:0/1679361649 conn(0x562136821a30 msgr2=0x562136826ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:40.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 -- 172.21.15.64:0/1679361649 shutdown_connections 2024-09-19T05:08:40.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.979+0000 7fca58901740 1 -- 172.21.15.64:0/1679361649 wait complete. 2024-09-19T05:08:40.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 -- start start 2024-09-19T05:08:40.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 Processor -- start 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x5621368344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x5621368349f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136834f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca501ae0e0 con 0x562136830680 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca501af550 con 0x5621368254a0 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca501ae7b0 con 0x562136825dc0 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca58901740 10 client.0 _ll_register_callbacks cb 0x7fca400095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca54ac5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136834f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca54ac5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136834f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42328/0 (socket says 172.21.15.64:42328) 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.980+0000 7fca54ac5640 1 -- 172.21.15.64:0/2783466934 learned_addr learned my addr 172.21.15.64:0/2783466934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca4ffff640 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x5621368344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca54ac5640 1 -- 172.21.15.64:0/2783466934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 msgr2=0x5621368349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca54ac5640 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x5621368349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca54ac5640 1 -- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 msgr2=0x5621368344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca54ac5640 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x5621368344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca54ac5640 1 -- 172.21.15.64:0/2783466934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621368a9f30 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca54ac5640 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136834f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x562136822db0 tx=0x7fca48048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a479908cc01e6403 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2783466934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480315e0 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2783466934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fca48004040 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.981+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4001d400 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2783466934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca48004240 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 10 client.16059 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621368ab1e0 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2783466934 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca48041040 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2783466934 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca48030b90 con 0x562136830680 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 10 client.16059 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:40.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 20 client.16059 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621368ab960 0x5621368adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 4 client.16059 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fca4001d400 con 0x5621368ab960 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.982+0000 7fca4ffff640 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621368ab960 0x5621368adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca4ffff640 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621368ab960 0x5621368adda0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f1576ce5816622 server_cookie=2ac0f1cd17827b69 in_seq=0 out_seq=0 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca2f7fe640 10 client.16059 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca2f7fe640 1 -- 172.21.15.64:0/2783466934 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fca4001d400 con 0x5621368ab960 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca2f7fe640 10 client.16059 handle_command_reply: tid=0 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 client.16059 shutdown 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 msgr2=0x562136834f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136834f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x562136822db0 tx=0x7fca48048000 comp rx=0 tx=0).stop 2024-09-19T05:08:40.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 shutdown_connections 2024-09-19T05:08:40.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621368ab960 0x5621368adda0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562136830680 0x562136834f30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562136825dc0 0x5621368349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 --2- 172.21.15.64:0/2783466934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621368254a0 0x5621368344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.983+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 >> 172.21.15.64:0/2783466934 conn(0x562136821a30 msgr2=0x562136827710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:40.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca2f7fe640 0 client.16059 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:40.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 shutdown_connections 2024-09-19T05:08:40.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca58901740 1 -- 172.21.15.64:0/2783466934 wait complete. 2024-09-19T05:08:40.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca58901740 20 client.16059 trim_cache size 0 max 16384 2024-09-19T05:08:40.992 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: "bdb337a1-bb6e-438f-aad0-30933efdd7a1": { 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "bdb337a1-bb6e-438f-aad0-30933efdd7a1", 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:08:40.993 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:08:40.994 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:08:40.994 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:08:40.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca380a33b0 msgr2=0x7fca380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca380a33b0 0x7fca380a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fca48009cd0 tx=0x7fca48009880 comp rx=0 tx=0).stop 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 msgr2=0x7fca50100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 0x7fca50100f70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca4400d950 tx=0x7fca4400dd50 comp rx=0 tx=0).stop 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 shutdown_connections 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca380a33b0 0x7fca380a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca50108300 0x7fca501014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca50103780 0x7fca50100f70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 --2- 172.21.15.64:0/1671826905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca50102dd0 0x7fca50100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 >> 172.21.15.64:0/1671826905 conn(0x7fca500fc7d0 msgr2=0x7fca50105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.984+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 shutdown_connections 2024-09-19T05:08:40.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:08:40.985+0000 7fca56528640 1 -- 172.21.15.64:0/1671826905 wait complete. 2024-09-19T05:08:41.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:40 smithi064 ceph-mon[30960]: pgmap v86: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 133 KiB/s wr, 7 op/s 2024-09-19T05:08:41.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:40 smithi103 ceph-mon[34446]: pgmap v86: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 133 KiB/s wr, 7 op/s 2024-09-19T05:08:41.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:40 smithi174 ceph-mon[34488]: pgmap v86: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 133 KiB/s wr, 7 op/s 2024-09-19T05:08:41.707 INFO:tasks.cephfs.filesystem:scrub status for tag:bdb337a1-bb6e-438f-aad0-30933efdd7a1 - {'path': '/', 'tag': 'bdb337a1-bb6e-438f-aad0-30933efdd7a1', 'options': 'recursive,force'} 2024-09-19T05:08:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 -- 172.21.15.64:0/3165932322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc101350 msgr2=0x7f76dc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 --2- 172.21.15.64:0/3165932322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc101350 0x7f76dc10d880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009cd0 tx=0x7f76d402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 -- 172.21.15.64:0/3165932322 shutdown_connections 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 --2- 172.21.15.64:0/3165932322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc101350 0x7f76dc10d880 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 --2- 172.21.15.64:0/3165932322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc1009b0 0x7f76dc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 --2- 172.21.15.64:0/3165932322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76dc1069c0 0x7f76dc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 -- 172.21.15.64:0/3165932322 >> 172.21.15.64:0/3165932322 conn(0x7f76dc077990 msgr2=0x7f76dc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 -- 172.21.15.64:0/3165932322 shutdown_connections 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.875+0000 7f76e1318640 1 -- 172.21.15.64:0/3165932322 wait complete. 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.876+0000 7f76e1318640 1 Processor -- start 2024-09-19T05:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.876+0000 7f76e1318640 1 -- start start 2024-09-19T05:08:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.876+0000 7f76e1318640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 0x7f76dc0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76e1318640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc101350 0x7f76dc0fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76e1318640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76dc1069c0 0x7f76dc0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76e1318640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc110cb0 con 0x7f76dc1009b0 2024-09-19T05:08:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76e1318640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc110b30 con 0x7f76dc1069c0 2024-09-19T05:08:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76e1318640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc110e30 con 0x7f76dc101350 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 0x7f76dc0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 0x7f76dc0fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42348/0 (socket says 172.21.15.64:42348) 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 -- 172.21.15.64:0/1640025259 learned_addr learned my addr 172.21.15.64:0/1640025259 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76db7fe640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76dc1069c0 0x7f76dc0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76da7fc640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc101350 0x7f76dc0fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 -- 172.21.15.64:0/1640025259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc101350 msgr2=0x7f76dc0fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc101350 0x7f76dc0fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 -- 172.21.15.64:0/1640025259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76dc1069c0 msgr2=0x7f76dc0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76dc1069c0 0x7f76dc0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 -- 172.21.15.64:0/1640025259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc1a6660 con 0x7f76dc1009b0 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76da7fc640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc101350 0x7f76dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76daffd640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 0x7f76dc0fcde0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c920 tx=0x7f76cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76c3fff640 1 -- 172.21.15.64:0/1640025259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc007c70 con 0x7f76dc1009b0 2024-09-19T05:08:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc1a6950 con 0x7f76dc1009b0 2024-09-19T05:08:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.877+0000 7f76c3fff640 1 -- 172.21.15.64:0/1640025259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76cc007e10 con 0x7f76dc1009b0 2024-09-19T05:08:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.878+0000 7f76c3fff640 1 -- 172.21.15.64:0/1640025259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc018820 con 0x7f76dc1009b0 2024-09-19T05:08:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.878+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc1a6ea0 con 0x7f76dc1009b0 2024-09-19T05:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.880+0000 7f76c3fff640 1 -- 172.21.15.64:0/1640025259 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f76cc0189c0 con 0x7f76dc1009b0 2024-09-19T05:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.880+0000 7f76e40b0740 1 Processor -- start 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.880+0000 7f76e40b0740 1 -- start start 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76c3fff640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f76c40a3460 0x7f76c40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5baf0 0x5617b2c5bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c5c410 0x5617b2c66770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 0x5617b2c690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76e40b0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5617b2bc02f0 con 0x5617b2c66cb0 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76e40b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5617b2b68270 con 0x5617b2c5baf0 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76e40b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617b2bbc2b0 con 0x5617b2c5c410 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76daffd640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f76c40a3460 0x7f76c40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 0x5617b2c690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 0x5617b2c690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42360/0 (socket says 172.21.15.64:42360) 2024-09-19T05:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 -- 172.21.15.64:0/2364931359 learned_addr learned my addr 172.21.15.64:0/2364931359 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76da7fc640 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c5c410 0x5617b2c66770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76c3fff640 1 -- 172.21.15.64:0/1640025259 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f76cc0cfe20 con 0x7f76dc1009b0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5baf0 0x5617b2c5bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 -- 172.21.15.64:0/2364931359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c5c410 msgr2=0x5617b2c66770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c5c410 0x5617b2c66770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 -- 172.21.15.64:0/2364931359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5baf0 msgr2=0x5617b2c5bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5baf0 0x5617b2c5bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 -- 172.21.15.64:0/2364931359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b2b67340 con 0x5617b2c66cb0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 0x5617b2c690a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009cd0 tx=0x7f76d4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186036264785b549 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.881+0000 7f76daffd640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f76c40a3460 0x7f76c40a5920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0cdb70 tx=0x7f76cc0bc040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.882+0000 7f76c17fa640 1 -- 172.21.15.64:0/2364931359 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4031950 con 0x5617b2c66cb0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.882+0000 7f76c17fa640 1 -- 172.21.15.64:0/2364931359 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d4036030 con 0x5617b2c66cb0 2024-09-19T05:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.882+0000 7f76c17fa640 1 -- 172.21.15.64:0/2364931359 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4004500 con 0x5617b2c66cb0 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.882+0000 7f76e40b0740 1 -- 172.21.15.64:0/2364931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 msgr2=0x5617b2c690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.882+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 0x5617b2c690a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009cd0 tx=0x7f76d4005ef0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- 172.21.15.64:0/2364931359 shutdown_connections 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c66cb0 0x5617b2c690a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c5c410 0x5617b2c66770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2364931359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5baf0 0x5617b2c5bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- 172.21.15.64:0/2364931359 >> 172.21.15.64:0/2364931359 conn(0x5617b2c58060 msgr2=0x5617b2c58440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- 172.21.15.64:0/2364931359 shutdown_connections 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- 172.21.15.64:0/2364931359 wait complete. 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- start start 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 Processor -- start 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c64680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5617b2c69880 con 0x5617b2c5baf0 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5617b2b49290 con 0x5617b2c5c410 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617b2c69a00 con 0x5617b2c66cb0 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76e40b0740 10 client.0 _ll_register_callbacks cb 0x5617b2b7a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.883+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c63c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42362/0 (socket says 172.21.15.64:42362) 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 -- 172.21.15.64:0/326960751 learned_addr learned my addr 172.21.15.64:0/326960751 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76da7fc640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 -- 172.21.15.64:0/326960751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 msgr2=0x5617b2c64680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c64680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 -- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 msgr2=0x5617b2c64140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c64140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 -- 172.21.15.64:0/326960751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b28d3210 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76db7fe640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c63c00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006010 tx=0x7f76d40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccc5c150635221a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76beff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76beff5640 1 -- 172.21.15.64:0/326960751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4050020 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76beff5640 1 -- 172.21.15.64:0/326960751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f76d4004b40 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d4036030 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76beff5640 1 -- 172.21.15.64:0/326960751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76d405b050 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76e40b0740 10 client.16071 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.884+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617b299f620 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76beff5640 1 -- 172.21.15.64:0/326960751 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d4036030 con 0x5617b2c5baf0 2024-09-19T05:08:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76beff5640 1 -- 172.21.15.64:0/326960751 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76d4045070 con 0x5617b2c5baf0 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76e40b0740 10 client.16071 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76e40b0740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76e40b0740 20 client.16071 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76e40b0740 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2c76c70 0x5617b2c79060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76e40b0740 4 client.16071 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b2bcda20 con 0x5617b2c76c70 2024-09-19T05:08:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.885+0000 7f76da7fc640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2c76c70 0x5617b2c79060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.886+0000 7f76da7fc640 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2c76c70 0x5617b2c79060 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75d6aaf4e9b6e82 server_cookie=3a0a2837f9850705 in_seq=0 out_seq=0 2024-09-19T05:08:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.886+0000 7f76beff5640 10 client.16071 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.886+0000 7f76beff5640 1 -- 172.21.15.64:0/326960751 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617b2bcda20 con 0x5617b2c76c70 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.886+0000 7f76beff5640 10 client.16071 handle_command_reply: tid=0 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.886+0000 7f76e40b0740 1 client.16071 shutdown 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 msgr2=0x5617b2c63c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c63c00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006010 tx=0x7f76d40040e0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 shutdown_connections 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c64680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c64140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2c76c70 0x5617b2c79060 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 --2- 172.21.15.64:0/326960751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c63c00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 >> 172.21.15.64:0/326960751 conn(0x5617b2c58060 msgr2=0x5617b2c5dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76beff5640 0 client.16071 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 shutdown_connections 2024-09-19T05:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 1 -- 172.21.15.64:0/326960751 wait complete. 2024-09-19T05:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.887+0000 7f76e40b0740 20 client.16071 trim_cache size 0 max 16384 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 Processor -- start 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 -- start start 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c65b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1b2060 con 0x5617b2c5baf0 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1af930 con 0x5617b2c5c410 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76e40b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1ad780 con 0x5617b2c66cb0 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42372/0 (socket says 172.21.15.64:42372) 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.904+0000 7f76db7fe640 1 -- 172.21.15.64:0/3874140248 learned_addr learned my addr 172.21.15.64:0/3874140248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76da7fc640 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c65b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 -- 172.21.15.64:0/3874140248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 msgr2=0x5617b2c7e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 -- 172.21.15.64:0/3874140248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 msgr2=0x5617b2c65b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c65b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 -- 172.21.15.64:0/3874140248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b2c62b90 con 0x5617b2c5baf0 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76da7fc640 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c65b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76db7fe640 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c644b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f76d4002230 tx=0x7f76d402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3245ffdfda2c4d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76c0ff9640 1 -- 172.21.15.64:0/3874140248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4050040 con 0x5617b2c5baf0 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76c0ff9640 1 -- 172.21.15.64:0/3874140248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d4036050 con 0x5617b2c5baf0 2024-09-19T05:08:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.905+0000 7f76c0ff9640 1 -- 172.21.15.64:0/3874140248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4004600 con 0x5617b2c5baf0 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 -- 172.21.15.64:0/3874140248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 msgr2=0x5617b2c644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c644b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f76d4002230 tx=0x7f76d402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 -- 172.21.15.64:0/3874140248 shutdown_connections 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c65b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 --2- 172.21.15.64:0/3874140248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c644b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 -- 172.21.15.64:0/3874140248 >> 172.21.15.64:0/3874140248 conn(0x5617b2c58060 msgr2=0x5617b2c65460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 -- 172.21.15.64:0/3874140248 shutdown_connections 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 -- 172.21.15.64:0/3874140248 wait complete. 2024-09-19T05:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 -- start start 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.906+0000 7f76e40b0740 1 Processor -- start 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c7ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c7b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1addf0 con 0x5617b2c5baf0 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1af260 con 0x5617b2c5c410 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1ae4c0 con 0x5617b2c66cb0 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76e40b0740 10 client.0 _ll_register_callbacks cb 0x7f76d0008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c7ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c7ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42380/0 (socket says 172.21.15.64:42380) 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 -- 172.21.15.64:0/2210938952 learned_addr learned my addr 172.21.15.64:0/2210938952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76da7fc640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c7b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 -- 172.21.15.64:0/2210938952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 msgr2=0x5617b2c7ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 -- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 msgr2=0x5617b2c7b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c7b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 -- 172.21.15.64:0/2210938952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b2c5ea80 con 0x5617b2c5baf0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76da7fc640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c7b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76db7fe640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c7ef90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f76d4005fc0 tx=0x7f76d4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dd82f06a0c498c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.907+0000 7f76beff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76beff5640 1 -- 172.21.15.64:0/2210938952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4030ed0 con 0x5617b2c5baf0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76beff5640 1 -- 172.21.15.64:0/2210938952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f76d405f640 con 0x5617b2c5baf0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d4036050 con 0x5617b2c5baf0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76beff5640 1 -- 172.21.15.64:0/2210938952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76d4031660 con 0x5617b2c5baf0 2024-09-19T05:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 10 client.16077 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617b2ce0670 con 0x5617b2c5baf0 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76beff5640 1 -- 172.21.15.64:0/2210938952 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d4036050 con 0x5617b2c5baf0 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76beff5640 1 -- 172.21.15.64:0/2210938952 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76d4045040 con 0x5617b2c5baf0 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 10 client.16077 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 20 client.16077 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2ce0b70 0x5617b2ce2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 4 client.16077 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617b2bc7d70 con 0x5617b2ce0b70 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.908+0000 7f76da7fc640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2ce0b70 0x5617b2ce2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.909+0000 7f76beff5640 10 client.16077 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.909+0000 7f76da7fc640 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2ce0b70 0x5617b2ce2f90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12bdbb5981b7493f server_cookie=d23c4b5b2000c171 in_seq=0 out_seq=0 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76beff5640 1 -- 172.21.15.64:0/2210938952 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617b2bc7d70 con 0x5617b2ce0b70 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76beff5640 10 client.16077 handle_command_reply: tid=0 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 client.16077 shutdown 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 msgr2=0x5617b2c7ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c7ef90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f76d4005fc0 tx=0x7f76d4000f40 comp rx=0 tx=0).stop 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 shutdown_connections 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b2c66cb0 0x5617b2c7ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5617b2c5c410 0x5617b2c7b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617b2ce0b70 0x5617b2ce2f90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 --2- 172.21.15.64:0/2210938952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617b2c5baf0 0x5617b2c7ef90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 >> 172.21.15.64:0/2210938952 conn(0x5617b2c58060 msgr2=0x5617b2c65c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76beff5640 0 client.16077 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 shutdown_connections 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 1 -- 172.21.15.64:0/2210938952 wait complete. 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.913+0000 7f76e40b0740 20 client.16077 trim_cache size 0 max 16384 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:08:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f76c40a3460 msgr2=0x7f76c40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f76c40a3460 0x7f76c40a5920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0cdb70 tx=0x7f76cc0bc040 comp rx=0 tx=0).stop 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 msgr2=0x7f76dc0fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 0x7f76dc0fcde0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c920 tx=0x7f76cc00cde0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 shutdown_connections 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f76c40a3460 0x7f76c40a5920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76dc1069c0 0x7f76dc0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76dc101350 0x7f76dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 --2- 172.21.15.64:0/1640025259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc1009b0 0x7f76dc0fcde0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 >> 172.21.15.64:0/1640025259 conn(0x7f76dc077990 msgr2=0x7f76dc0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 shutdown_connections 2024-09-19T05:08:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:42.914+0000 7f76e1318640 1 -- 172.21.15.64:0/1640025259 wait complete. 2024-09-19T05:08:43.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:42 smithi064 ceph-mon[30960]: pgmap v87: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 133 KiB/s wr, 9 op/s 2024-09-19T05:08:43.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:42 smithi103 ceph-mon[34446]: pgmap v87: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 133 KiB/s wr, 9 op/s 2024-09-19T05:08:43.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:42 smithi174 ceph-mon[34488]: pgmap v87: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 133 KiB/s wr, 9 op/s 2024-09-19T05:08:44.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:43 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:08:44.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:43 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:08:44.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:43 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:08:45.078 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:44 smithi064 ceph-mon[30960]: pgmap v88: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 133 KiB/s wr, 6 op/s 2024-09-19T05:08:45.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:44 smithi103 ceph-mon[34446]: pgmap v88: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 133 KiB/s wr, 6 op/s 2024-09-19T05:08:45.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:44 smithi174 ceph-mon[34488]: pgmap v88: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 133 KiB/s wr, 6 op/s 2024-09-19T05:08:47.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:46 smithi064 ceph-mon[30960]: pgmap v89: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 134 KiB/s wr, 9 op/s 2024-09-19T05:08:47.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:46 smithi103 ceph-mon[34446]: pgmap v89: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 134 KiB/s wr, 9 op/s 2024-09-19T05:08:47.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:46 smithi174 ceph-mon[34488]: pgmap v89: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 134 KiB/s wr, 9 op/s 2024-09-19T05:08:49.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:48 smithi064 ceph-mon[30960]: pgmap v90: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T05:08:49.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:48 smithi103 ceph-mon[34446]: pgmap v90: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T05:08:49.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:48 smithi174 ceph-mon[34488]: pgmap v90: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T05:08:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:50 smithi064 ceph-mon[30960]: pgmap v91: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:51.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:50 smithi103 ceph-mon[34446]: pgmap v91: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:51.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:50 smithi174 ceph-mon[34488]: pgmap v91: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:53.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:52 smithi064 ceph-mon[30960]: pgmap v92: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:53.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:52 smithi103 ceph-mon[34446]: pgmap v92: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:53.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:52 smithi174 ceph-mon[34488]: pgmap v92: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:55.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:54 smithi064 ceph-mon[30960]: pgmap v93: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-19T05:08:55.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:54 smithi103 ceph-mon[34446]: pgmap v93: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-19T05:08:55.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:54 smithi174 ceph-mon[34488]: pgmap v93: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 -- 172.21.15.64:0/3770053083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01009b0 msgr2=0x7f27c0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 --2- 172.21.15.64:0/3770053083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01009b0 0x7f27c0100e10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009930 tx=0x7f27b002a420 comp rx=0 tx=0).stop 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 -- 172.21.15.64:0/3770053083 shutdown_connections 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 --2- 172.21.15.64:0/3770053083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27c0101350 0x7f27c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 --2- 172.21.15.64:0/3770053083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01009b0 0x7f27c0100e10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 --2- 172.21.15.64:0/3770053083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27c01069c0 0x7f27c0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 -- 172.21.15.64:0/3770053083 >> 172.21.15.64:0/3770053083 conn(0x7f27c0077990 msgr2=0x7f27c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 -- 172.21.15.64:0/3770053083 shutdown_connections 2024-09-19T05:08:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.017+0000 7f27c843c640 1 -- 172.21.15.64:0/3770053083 wait complete. 2024-09-19T05:08:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 Processor -- start 2024-09-19T05:08:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 -- start start 2024-09-19T05:08:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27c01009b0 0x7f27c0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27c0101350 0x7f27c0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 0x7f27c019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0110cb0 con 0x7f27c01069c0 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0110b30 con 0x7f27c0101350 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c843c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0110e30 con 0x7f27c01009b0 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 0x7f27c019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 0x7f27c019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59812/0 (socket says 172.21.15.64:59812) 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 -- 172.21.15.64:0/3114220779 learned_addr learned my addr 172.21.15.64:0/3114220779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27c01009b0 0x7f27c0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27c0101350 0x7f27c0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 -- 172.21.15.64:0/3114220779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27c01009b0 msgr2=0x7f27c0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27c01009b0 0x7f27c0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 -- 172.21.15.64:0/3114220779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27c0101350 msgr2=0x7f27c0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27c0101350 0x7f27c0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.018+0000 7f27c69d9640 1 -- 172.21.15.64:0/3114220779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c019d310 con 0x7f27c01069c0 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.019+0000 7f27c69d9640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 0x7f27c019cbe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27b800eb10 tx=0x7f27b800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.019+0000 7f27b77fe640 1 -- 172.21.15.64:0/3114220779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b800ce20 con 0x7f27c01069c0 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.019+0000 7f27b77fe640 1 -- 172.21.15.64:0/3114220779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27b8005020 con 0x7f27c01069c0 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.019+0000 7f27b77fe640 1 -- 172.21.15.64:0/3114220779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8010690 con 0x7f27c01069c0 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.019+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c019d600 con 0x7f27c01069c0 2024-09-19T05:08:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.019+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c01a4e90 con 0x7f27c01069c0 2024-09-19T05:08:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.021+0000 7f27b77fe640 1 -- 172.21.15.64:0/3114220779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f27b80051d0 con 0x7f27c01069c0 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27cb1d8740 1 Processor -- start 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27cb1d8740 1 -- start start 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27b77fe640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27ac0a3460 0x7f27ac0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea2af0 0x55e83cea2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea3410 0x55e83cead770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 0x55e83ceb00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.022+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e83ce072f0 con 0x55e83ceadcb0 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e83cdaf270 con 0x55e83cea2af0 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e83ce032b0 con 0x55e83cea3410 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27ac0a3460 0x7f27ac0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 0x55e83ceb00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 0x55e83ceb00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59816/0 (socket says 172.21.15.64:59816) 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 -- 172.21.15.64:0/1021485423 learned_addr learned my addr 172.21.15.64:0/1021485423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27b77fe640 1 -- 172.21.15.64:0/3114220779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27b80d02b0 con 0x7f27c01069c0 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c61d8640 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea2af0 0x55e83cea2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c59d7640 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea3410 0x55e83cead770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 -- 172.21.15.64:0/1021485423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea3410 msgr2=0x55e83cead770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea3410 0x55e83cead770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 -- 172.21.15.64:0/1021485423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea2af0 msgr2=0x55e83cea2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea2af0 0x55e83cea2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 -- 172.21.15.64:0/1021485423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e83cdae340 con 0x55e83ceadcb0 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27ac0a3460 0x7f27ac0a5920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009900 tx=0x7f27b0031f80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27c69d9640 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 0x55e83ceb00a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f27b8098f60 tx=0x7f27b809c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e6fc1a95c7c8be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27b4bfe640 1 -- 172.21.15.64:0/1021485423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b809cc90 con 0x55e83ceadcb0 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27b4bfe640 1 -- 172.21.15.64:0/1021485423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27b809c590 con 0x55e83ceadcb0 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.023+0000 7f27b4bfe640 1 -- 172.21.15.64:0/1021485423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80cdc80 con 0x55e83ceadcb0 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 -- 172.21.15.64:0/1021485423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 msgr2=0x55e83ceb00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 0x55e83ceb00a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f27b8098f60 tx=0x7f27b809c0d0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 -- 172.21.15.64:0/1021485423 shutdown_connections 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83ceadcb0 0x55e83ceb00a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea3410 0x55e83cead770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/1021485423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea2af0 0x55e83cea2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 -- 172.21.15.64:0/1021485423 >> 172.21.15.64:0/1021485423 conn(0x55e83ce9f060 msgr2=0x55e83ce9f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 -- 172.21.15.64:0/1021485423 shutdown_connections 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.024+0000 7f27cb1d8740 1 -- 172.21.15.64:0/1021485423 wait complete. 2024-09-19T05:08:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 -- start start 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 Processor -- start 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83ceaab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83ceab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83ceab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e83ceb0880 con 0x55e83cea3410 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e83cd90290 con 0x55e83ceadcb0 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e83ceb0a00 con 0x55e83cea2af0 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27cb1d8740 10 client.0 _ll_register_callbacks cb 0x55e83cdc1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27c59d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83ceab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27c59d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83ceab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59828/0 (socket says 172.21.15.64:59828) 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27c59d7640 1 -- 172.21.15.64:0/3548145852 learned_addr learned my addr 172.21.15.64:0/3548145852 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.025+0000 7f27c69d9640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83ceab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83ceaab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c59d7640 1 -- 172.21.15.64:0/3548145852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 msgr2=0x55e83ceaab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83ceaab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c59d7640 1 -- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 msgr2=0x55e83ceab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83ceab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c59d7640 1 -- 172.21.15.64:0/3548145852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83cb1a210 con 0x55e83cea3410 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83ceaab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c69d9640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83ceab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83ceab0a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f27b0031e70 tx=0x7f27b002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e8107a3786dbbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27a9ffb640 1 -- 172.21.15.64:0/3548145852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b004e020 con 0x55e83cea3410 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b809c590 con 0x55e83cea3410 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27a9ffb640 1 -- 172.21.15.64:0/3548145852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27b00037f0 con 0x55e83cea3410 2024-09-19T05:08:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27a9ffb640 1 -- 172.21.15.64:0/3548145852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27b0059050 con 0x55e83cea3410 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27cb1d8740 10 client.16089 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.026+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e83cbe6620 con 0x55e83cea3410 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27a9ffb640 1 -- 172.21.15.64:0/3548145852 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27b809c590 con 0x55e83cea3410 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27a9ffb640 1 -- 172.21.15.64:0/3548145852 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27b004aa60 con 0x55e83cea3410 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27cb1d8740 10 client.16089 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27cb1d8740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27cb1d8740 20 client.16089 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cec4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27cb1d8740 4 client.16089 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e83ce14a20 con 0x55e83cec2050 2024-09-19T05:08:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.027+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cec4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27a9ffb640 10 client.16089 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cec4460 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd496aedc8f38226 server_cookie=f83b4afb1885ee50 in_seq=0 out_seq=0 2024-09-19T05:08:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27a9ffb640 1 -- 172.21.15.64:0/3548145852 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e83ce14a20 con 0x55e83cec2050 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27a9ffb640 10 client.16089 handle_command_reply: tid=0 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 client.16089 shutdown 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 msgr2=0x55e83ceab0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83ceab0a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f27b0031e70 tx=0x7f27b002a420 comp rx=0 tx=0).stop 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 shutdown_connections 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83ceab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cec4460 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83ceab0a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.028+0000 7f27a9ffb640 0 client.16089 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.029+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3548145852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83ceaab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.029+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 >> 172.21.15.64:0/3548145852 conn(0x55e83ce9f060 msgr2=0x55e83cea4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.029+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 shutdown_connections 2024-09-19T05:08:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.029+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3548145852 wait complete. 2024-09-19T05:08:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.029+0000 7f27cb1d8740 20 client.16089 trim_cache size 0 max 16384 2024-09-19T05:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 Processor -- start 2024-09-19T05:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 -- start start 2024-09-19T05:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 0x55e83cea5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea3410 0x55e83cec9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83ceadcb0 0x55e83ceca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01adeb0 con 0x55e83cea2af0 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01ab780 con 0x55e83cea3410 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01a95f0 con 0x55e83ceadcb0 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 0x55e83cea5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 0x55e83cea5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59844/0 (socket says 172.21.15.64:59844) 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 -- 172.21.15.64:0/3496461619 learned_addr learned my addr 172.21.15.64:0/3496461619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c59d7640 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea3410 0x55e83cec9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c69d9640 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83ceadcb0 0x55e83ceca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 -- 172.21.15.64:0/3496461619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83ceadcb0 msgr2=0x55e83ceca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83ceadcb0 0x55e83ceca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 -- 172.21.15.64:0/3496461619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea3410 msgr2=0x55e83cec9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea3410 0x55e83cec9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 -- 172.21.15.64:0/3496461619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e83ce9ee90 con 0x55e83cea2af0 2024-09-19T05:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.045+0000 7f27c61d8640 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 0x55e83cea5560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009e60 tx=0x7f27bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b5c0a2f21e673f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.046+0000 7f27abfff640 1 -- 172.21.15.64:0/3496461619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc01d310 con 0x55e83cea2af0 2024-09-19T05:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.046+0000 7f27abfff640 1 -- 172.21.15.64:0/3496461619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27bc01d960 con 0x55e83cea2af0 2024-09-19T05:08:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.046+0000 7f27abfff640 1 -- 172.21.15.64:0/3496461619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc015900 con 0x55e83cea2af0 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.046+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3496461619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 msgr2=0x55e83cea5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.046+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 0x55e83cea5560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009e60 tx=0x7f27bc008040 comp rx=0 tx=0).stop 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3496461619 shutdown_connections 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83ceadcb0 0x55e83ceca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83cea3410 0x55e83cec9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/3496461619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea2af0 0x55e83cea5560 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3496461619 >> 172.21.15.64:0/3496461619 conn(0x55e83ce9f060 msgr2=0x55e83ceacbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3496461619 shutdown_connections 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- 172.21.15.64:0/3496461619 wait complete. 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- start start 2024-09-19T05:08:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 Processor -- start 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83cea7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83cea7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83cea84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01a9c40 con 0x55e83cea3410 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01ab0b0 con 0x55e83ceadcb0 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01aa310 con 0x55e83cea2af0 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.047+0000 7f27cb1d8740 10 client.0 _ll_register_callbacks cb 0x7f27b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83cea7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83cea7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59860/0 (socket says 172.21.15.64:59860) 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 -- 172.21.15.64:0/169873947 learned_addr learned my addr 172.21.15.64:0/169873947 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c61d8640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83cea7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c69d9640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83cea84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 -- 172.21.15.64:0/169873947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 msgr2=0x55e83cea7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83cea7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 -- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 msgr2=0x55e83cea84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83cea84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 -- 172.21.15.64:0/169873947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83cf2bd40 con 0x55e83cea3410 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27c59d7640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83cea7fb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f27b004add0 tx=0x7f27b003d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ec914f4ec12cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27a9ffb640 1 -- 172.21.15.64:0/169873947 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b004e040 con 0x55e83cea3410 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27a9ffb640 1 -- 172.21.15.64:0/169873947 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27b003dcc0 con 0x55e83cea3410 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27bc01d960 con 0x55e83cea3410 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.048+0000 7f27a9ffb640 1 -- 172.21.15.64:0/169873947 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27b0035040 con 0x55e83cea3410 2024-09-19T05:08:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 10 client.16095 fetch_fsmap learned FSMap version 25 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e83cf2cf90 con 0x55e83cea3410 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27a9ffb640 1 -- 172.21.15.64:0/169873947 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27bc01d960 con 0x55e83cea3410 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27a9ffb640 1 -- 172.21.15.64:0/169873947 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27b00035f0 con 0x55e83cea3410 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 10 client.16095 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 20 client.16095 populate_metadata read hostname 'smithi064' 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cf2f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 4 client.16095 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e83ce0ed70 con 0x55e83cec2050 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.049+0000 7f27c61d8640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cf2f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.050+0000 7f27a9ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.050+0000 7f27c61d8640 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cf2f710 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3648e662ff316de server_cookie=7211dc2a9cd52109 in_seq=0 out_seq=0 2024-09-19T05:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27a9ffb640 1 -- 172.21.15.64:0/169873947 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e83ce0ed70 con 0x55e83cec2050 2024-09-19T05:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27a9ffb640 10 client.16095 handle_command_reply: tid=0 2024-09-19T05:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 client.16095 shutdown 2024-09-19T05:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 msgr2=0x55e83cea7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83cea7fb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f27b004add0 tx=0x7f27b003d670 comp rx=0 tx=0).stop 2024-09-19T05:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 shutdown_connections 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e83ceadcb0 0x55e83cea84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e83cec2050 0x55e83cf2f710 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e83cea3410 0x55e83cea7fb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27a9ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 --2- 172.21.15.64:0/169873947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e83cea2af0 0x55e83cea7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 >> 172.21.15.64:0/169873947 conn(0x55e83ce9f060 msgr2=0x55e83ce9d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 shutdown_connections 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 1 -- 172.21.15.64:0/169873947 wait complete. 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.057+0000 7f27cb1d8740 20 client.16095 trim_cache size 0 max 16384 2024-09-19T05:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:08:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27ac0a3460 msgr2=0x7f27ac0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27ac0a3460 0x7f27ac0a5920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009900 tx=0x7f27b0031f80 comp rx=0 tx=0).stop 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 msgr2=0x7f27c019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 0x7f27c019cbe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27b800eb10 tx=0x7f27b800ef10 comp rx=0 tx=0).stop 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 shutdown_connections 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27ac0a3460 0x7f27ac0a5920 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27c01069c0 0x7f27c019cbe0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27c0101350 0x7f27c0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 --2- 172.21.15.64:0/3114220779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27c01009b0 0x7f27c0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 >> 172.21.15.64:0/3114220779 conn(0x7f27c0077990 msgr2=0x7f27c0069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 shutdown_connections 2024-09-19T05:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:08:56.058+0000 7f27c843c640 1 -- 172.21.15.64:0/3114220779 wait complete. 2024-09-19T05:08:56.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:56 smithi064 ceph-mon[30960]: pgmap v94: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:57.222 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:56 smithi103 ceph-mon[34446]: pgmap v94: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:57.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:56 smithi174 ceph-mon[34488]: pgmap v94: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-19T05:08:59.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:08:58 smithi064 ceph-mon[30960]: pgmap v95: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:08:59.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:08:58 smithi103 ceph-mon[34446]: pgmap v95: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:08:59.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:08:58 smithi174 ceph-mon[34488]: pgmap v95: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:09:01.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:00 smithi064 ceph-mon[30960]: pgmap v96: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:09:01.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:00 smithi103 ceph-mon[34446]: pgmap v96: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:09:01.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:00 smithi174 ceph-mon[34488]: pgmap v96: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:09:03.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:02 smithi064 ceph-mon[30960]: pgmap v97: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-19T05:09:03.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:02 smithi103 ceph-mon[34446]: pgmap v97: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-19T05:09:03.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:02 smithi174 ceph-mon[34488]: pgmap v97: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-19T05:09:05.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:04 smithi064 ceph-mon[30960]: pgmap v98: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-19T05:09:05.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:04 smithi103 ceph-mon[34446]: pgmap v98: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-19T05:09:05.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:04 smithi174 ceph-mon[34488]: pgmap v98: 97 pgs: 97 active+clean; 5.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-19T05:09:06.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:09:06.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:09:06.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:09:06.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:09:06.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:09:06.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:09:06.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:09:06.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:09:06.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:09:06.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:09:06.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:09:06.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:09:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:07 smithi064 ceph-mon[30960]: pgmap v99: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 197 KiB/s wr, 24 op/s 2024-09-19T05:09:07.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:07 smithi103 ceph-mon[34446]: pgmap v99: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 197 KiB/s wr, 24 op/s 2024-09-19T05:09:07.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:07 smithi174 ceph-mon[34488]: pgmap v99: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 197 KiB/s wr, 24 op/s 2024-09-19T05:09:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 -- 172.21.15.64:0/1699951458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 msgr2=0x7f6d4c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/1699951458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c100e10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009930 tx=0x7f6d4002a420 comp rx=0 tx=0).stop 2024-09-19T05:09:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 -- 172.21.15.64:0/1699951458 shutdown_connections 2024-09-19T05:09:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/1699951458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 0x7f6d4c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/1699951458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c100e10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/1699951458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 0x7f6d4c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 -- 172.21.15.64:0/1699951458 >> 172.21.15.64:0/1699951458 conn(0x7f6d4c077990 msgr2=0x7f6d4c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 -- 172.21.15.64:0/1699951458 shutdown_connections 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.150+0000 7f6d51ca5640 1 -- 172.21.15.64:0/1699951458 wait complete. 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.151+0000 7f6d51ca5640 1 Processor -- start 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.151+0000 7f6d51ca5640 1 -- start start 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.151+0000 7f6d51ca5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.151+0000 7f6d51ca5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 0x7f6d4c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.151+0000 7f6d51ca5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 0x7f6d4c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.151+0000 7f6d51ca5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c110cb0 con 0x7f6d4c1009b0 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d51ca5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c110b30 con 0x7f6d4c101350 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d51ca5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c110e30 con 0x7f6d4c1069c0 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38764/0 (socket says 172.21.15.64:38764) 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/891522477 learned_addr learned my addr 172.21.15.64:0/891522477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 0x7f6d4c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4affd640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 0x7f6d4c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/891522477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 msgr2=0x7f6d4c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 0x7f6d4c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/891522477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 msgr2=0x7f6d4c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 0x7f6d4c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/891522477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c19d300 con 0x7f6d4c1009b0 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 0x7f6d4c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4affd640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 0x7f6d4c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c198310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b4a0 tx=0x7f6d3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d48ff9640 1 -- 172.21.15.64:0/891522477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c014070 con 0x7f6d4c1009b0 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c19d5f0 con 0x7f6d4c1009b0 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d48ff9640 1 -- 172.21.15.64:0/891522477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c010ac0 con 0x7f6d4c1009b0 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.152+0000 7f6d48ff9640 1 -- 172.21.15.64:0/891522477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c017c30 con 0x7f6d4c1009b0 2024-09-19T05:09:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.153+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c1a4e80 con 0x7f6d4c1009b0 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d48ff9640 1 -- 172.21.15.64:0/891522477 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6d3c017070 con 0x7f6d4c1009b0 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 Processor -- start 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 -- start start 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee83db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f73ede82f0 con 0x55f73ee930c0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f73ed90270 con 0x55f73ee839d0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d54a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f73ede42b0 con 0x55f73ee842f0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee83db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38772/0 (socket says 172.21.15.64:38772) 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d4affd640 1 -- 172.21.15.64:0/3172396605 learned_addr learned my addr 172.21.15.64:0/3172396605 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.156+0000 7f6d48ff9640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6d340a3460 0x7f6d340a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d48ff9640 1 -- 172.21.15.64:0/891522477 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6d3c0cf380 con 0x7f6d4c1009b0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6d340a3460 0x7f6d340a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4affd640 1 -- 172.21.15.64:0/3172396605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 msgr2=0x55f73ee92b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4affd640 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee92b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4affd640 1 -- 172.21.15.64:0/3172396605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 msgr2=0x55f73ee83db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4affd640 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee83db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4affd640 1 -- 172.21.15.64:0/3172396605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73ed8f340 con 0x55f73ee930c0 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4affd640 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee954b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002a940 tx=0x7f6d40032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07f9dc792a41886 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d31ffb640 1 -- 172.21.15.64:0/3172396605 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40035030 con 0x55f73ee930c0 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6d340a3460 0x7f6d340a5920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d44005fd0 tx=0x7f6d44004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d31ffb640 1 -- 172.21.15.64:0/3172396605 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d40031760 con 0x55f73ee930c0 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.157+0000 7f6d31ffb640 1 -- 172.21.15.64:0/3172396605 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400046c0 con 0x55f73ee930c0 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 -- 172.21.15.64:0/3172396605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 msgr2=0x55f73ee954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee954b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002a940 tx=0x7f6d40032e70 comp rx=0 tx=0).stop 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 -- 172.21.15.64:0/3172396605 shutdown_connections 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee954b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 --2- 172.21.15.64:0/3172396605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee83db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 -- 172.21.15.64:0/3172396605 >> 172.21.15.64:0/3172396605 conn(0x55f73ee80060 msgr2=0x55f73ee80440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 -- 172.21.15.64:0/3172396605 shutdown_connections 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 -- 172.21.15.64:0/3172396605 wait complete. 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.158+0000 7f6d54a47740 1 -- start start 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 Processor -- start 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 0x55f73ee900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73ee905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee930c0 0x55f73ee90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f73ee95c90 con 0x55f73ee839d0 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f73ed71290 con 0x55f73ee842f0 2024-09-19T05:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f73ee95e10 con 0x55f73ee930c0 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d54a47740 10 client.0 _ll_register_callbacks cb 0x55f73eda2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 0x55f73ee900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 0x55f73ee900a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38786/0 (socket says 172.21.15.64:38786) 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/2878151894 learned_addr learned my addr 172.21.15.64:0/2878151894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4affd640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73ee905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/2878151894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee930c0 msgr2=0x55f73ee90b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee930c0 0x55f73ee90b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 msgr2=0x55f73ee905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73ee905e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4b7fe640 1 -- 172.21.15.64:0/2878151894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73eafb210 con 0x55f73ee839d0 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.159+0000 7f6d4affd640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73ee905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 0x55f73ee900a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0cd0d0 tx=0x7f6d3c09b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e132f3862c0e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d2affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d2affd640 1 -- 172.21.15.64:0/2878151894 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c01e050 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d2affd640 1 -- 172.21.15.64:0/2878151894 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6d3c097de0 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d40031760 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d2affd640 1 -- 172.21.15.64:0/2878151894 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c09fb20 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 10 client.16107 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f73ebc7620 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d2affd640 1 -- 172.21.15.64:0/2878151894 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d40031760 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d2affd640 1 -- 172.21.15.64:0/2878151894 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d3c0a3f50 con 0x55f73ee839d0 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 10 client.16107 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 20 client.16107 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73eea31b0 0x55f73eea55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 4 client.16107 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.160+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73edf5a20 con 0x55f73eea31b0 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.161+0000 7f6d4affd640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73eea31b0 0x55f73eea55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.161+0000 7f6d4affd640 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73eea31b0 0x55f73eea55a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f021c980b3fb5a3 server_cookie=8b6df2d5110d8b22 in_seq=0 out_seq=0 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.161+0000 7f6d2affd640 10 client.16107 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d2affd640 1 -- 172.21.15.64:0/2878151894 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f73edf5a20 con 0x55f73eea31b0 2024-09-19T05:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d2affd640 10 client.16107 handle_command_reply: tid=0 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 client.16107 shutdown 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 msgr2=0x55f73ee900a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 0x55f73ee900a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0cd0d0 tx=0x7f6d3c09b370 comp rx=0 tx=0).stop 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 shutdown_connections 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee930c0 0x55f73ee90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73ee905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73eea31b0 0x55f73eea55a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2878151894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee839d0 0x55f73ee900a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 >> 172.21.15.64:0/2878151894 conn(0x55f73ee80060 msgr2=0x55f73ee85d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d2affd640 0 client.16107 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 shutdown_connections 2024-09-19T05:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 1 -- 172.21.15.64:0/2878151894 wait complete. 2024-09-19T05:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.162+0000 7f6d54a47740 20 client.16107 trim_cache size 0 max 16384 2024-09-19T05:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.177+0000 7f6d54a47740 1 Processor -- start 2024-09-19T05:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.177+0000 7f6d54a47740 1 -- start start 2024-09-19T05:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee839d0 0x55f73ee90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73eeaadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73eeab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d54a47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1add30 con 0x55f73ee930c0 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d54a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1ab600 con 0x55f73ee842f0 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d54a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a9470 con 0x55f73ee839d0 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73eeab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73eeab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38798/0 (socket says 172.21.15.64:38798) 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 -- 172.21.15.64:0/1938915452 learned_addr learned my addr 172.21.15.64:0/1938915452 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee839d0 0x55f73ee90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4affd640 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73eeaadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 -- 172.21.15.64:0/1938915452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee839d0 msgr2=0x55f73ee90770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee839d0 0x55f73ee90770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 -- 172.21.15.64:0/1938915452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 msgr2=0x55f73eeaadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73eeaadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 -- 172.21.15.64:0/1938915452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73ee810a0 con 0x55f73ee930c0 2024-09-19T05:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73eeab310 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6d44012680 tx=0x7f6d440096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7681599c726d5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.178+0000 7f6d317fa640 1 -- 172.21.15.64:0/1938915452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44028480 con 0x55f73ee930c0 2024-09-19T05:09:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d317fa640 1 -- 172.21.15.64:0/1938915452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d44028ad0 con 0x55f73ee930c0 2024-09-19T05:09:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d317fa640 1 -- 172.21.15.64:0/1938915452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4401ebd0 con 0x55f73ee930c0 2024-09-19T05:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 -- 172.21.15.64:0/1938915452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 msgr2=0x55f73eeab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73eeab310 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6d44012680 tx=0x7f6d440096e0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 -- 172.21.15.64:0/1938915452 shutdown_connections 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73eeab310 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee842f0 0x55f73eeaadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 --2- 172.21.15.64:0/1938915452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee839d0 0x55f73ee90770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 -- 172.21.15.64:0/1938915452 >> 172.21.15.64:0/1938915452 conn(0x55f73ee80060 msgr2=0x55f73ee90060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 -- 172.21.15.64:0/1938915452 shutdown_connections 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.179+0000 7f6d54a47740 1 -- 172.21.15.64:0/1938915452 wait complete. 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 -- start start 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 Processor -- start 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee96d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a9ac0 con 0x55f73ee930c0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1aaf30 con 0x55f73ee839d0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1aa190 con 0x55f73ee842f0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d54a47740 10 client.0 _ll_register_callbacks cb 0x7f6d3c0174e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee9ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38808/0 (socket says 172.21.15.64:38808) 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d4bfff640 1 -- 172.21.15.64:0/2804995552 learned_addr learned my addr 172.21.15.64:0/2804995552 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.180+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4affd640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee96d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4bfff640 1 -- 172.21.15.64:0/2804995552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 msgr2=0x55f73ee96d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee96d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4bfff640 1 -- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 msgr2=0x55f73ee96810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee96810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4bfff640 1 -- 172.21.15.64:0/2804995552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73ee9b570 con 0x55f73ee930c0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d4bfff640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee9ae80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6d4401e500 tx=0x7f6d44009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d960686d2b55806 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d2affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d2affd640 1 -- 172.21.15.64:0/2804995552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44028680 con 0x55f73ee930c0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d2affd640 1 -- 172.21.15.64:0/2804995552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6d44028820 con 0x55f73ee930c0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d44028ad0 con 0x55f73ee930c0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d2affd640 1 -- 172.21.15.64:0/2804995552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d4401e7d0 con 0x55f73ee930c0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d54a47740 10 client.16113 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.181+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f73ef0d970 con 0x55f73ee930c0 2024-09-19T05:09:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d2affd640 1 -- 172.21.15.64:0/2804995552 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d44028ad0 con 0x55f73ee930c0 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d2affd640 1 -- 172.21.15.64:0/2804995552 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d44030030 con 0x55f73ee930c0 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d54a47740 10 client.16113 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d54a47740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d54a47740 20 client.16113 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73ef0dfb0 0x55f73ef103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d54a47740 4 client.16113 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f73edefd70 con 0x55f73ef0dfb0 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73ef0dfb0 0x55f73ef103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d2affd640 10 client.16113 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.182+0000 7f6d4b7fe640 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73ef0dfb0 0x55f73ef103d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1745371326dbf7 server_cookie=17a592088f9a810 in_seq=0 out_seq=0 2024-09-19T05:09:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.190+0000 7f6d2affd640 1 -- 172.21.15.64:0/2804995552 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f73edefd70 con 0x55f73ef0dfb0 2024-09-19T05:09:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.190+0000 7f6d2affd640 10 client.16113 handle_command_reply: tid=0 2024-09-19T05:09:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.190+0000 7f6d54a47740 1 client.16113 shutdown 2024-09-19T05:09:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 msgr2=0x55f73ee9ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee9ae80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6d4401e500 tx=0x7f6d44009e90 comp rx=0 tx=0).stop 2024-09-19T05:09:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 shutdown_connections 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f73ef0dfb0 0x55f73ef103d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f73ee930c0 0x55f73ee9ae80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f73ee842f0 0x55f73ee96d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 --2- 172.21.15.64:0/2804995552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f73ee839d0 0x55f73ee96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 >> 172.21.15.64:0/2804995552 conn(0x55f73ee80060 msgr2=0x55f73ee90860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d2affd640 0 client.16113 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 shutdown_connections 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 1 -- 172.21.15.64:0/2804995552 wait complete. 2024-09-19T05:09:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.191+0000 7f6d54a47740 20 client.16113 trim_cache size 0 max 16384 2024-09-19T05:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:09:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6d340a3460 msgr2=0x7f6d340a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6d340a3460 0x7f6d340a5920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d44005fd0 tx=0x7f6d44004ab0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 msgr2=0x7f6d4c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c198310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b4a0 tx=0x7f6d3c00b960 comp rx=0 tx=0).stop 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 shutdown_connections 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6d340a3460 0x7f6d340a5920 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d4c1069c0 0x7f6d4c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d4c101350 0x7f6d4c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 --2- 172.21.15.64:0/891522477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c1009b0 0x7f6d4c198310 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 >> 172.21.15.64:0/891522477 conn(0x7f6d4c077990 msgr2=0x7f6d4c069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 shutdown_connections 2024-09-19T05:09:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:09.192+0000 7f6d51ca5640 1 -- 172.21.15.64:0/891522477 wait complete. 2024-09-19T05:09:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:09 smithi064 ceph-mon[30960]: pgmap v100: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 197 KiB/s wr, 22 op/s 2024-09-19T05:09:09.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:09 smithi103 ceph-mon[34446]: pgmap v100: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 197 KiB/s wr, 22 op/s 2024-09-19T05:09:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:09 smithi174 ceph-mon[34488]: pgmap v100: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 197 KiB/s wr, 22 op/s 2024-09-19T05:09:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:11 smithi064 ceph-mon[30960]: pgmap v101: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 198 KiB/s wr, 24 op/s 2024-09-19T05:09:11.708 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:11.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:11 smithi103 ceph-mon[34446]: pgmap v101: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 198 KiB/s wr, 24 op/s 2024-09-19T05:09:11.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:11 smithi174 ceph-mon[34488]: pgmap v101: 97 pgs: 97 active+clean; 7.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 198 KiB/s wr, 24 op/s 2024-09-19T05:09:11.982 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:12.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.503+0000 7f3aa105c640 1 -- 172.21.15.64:0/627489081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102ef0 msgr2=0x7f3a9c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.503+0000 7f3aa105c640 1 --2- 172.21.15.64:0/627489081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c10da40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009ca0 tx=0x7f3a9402a950 comp rx=0 tx=0).stop 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 -- 172.21.15.64:0/627489081 shutdown_connections 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 --2- 172.21.15.64:0/627489081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c10da40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 --2- 172.21.15.64:0/627489081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a9c102570 0x7f3a9c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 --2- 172.21.15.64:0/627489081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a9c100e20 0x7f3a9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 -- 172.21.15.64:0/627489081 >> 172.21.15.64:0/627489081 conn(0x7f3a9c0fc7d0 msgr2=0x7f3a9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 -- 172.21.15.64:0/627489081 shutdown_connections 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.504+0000 7f3aa105c640 1 -- 172.21.15.64:0/627489081 wait complete. 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 Processor -- start 2024-09-19T05:09:12.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 -- start start 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a9c100e20 0x7f3a9c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 0x7f3a9c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c110e30 con 0x7f3a9c102570 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c110cb0 con 0x7f3a9c102ef0 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3aa105c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c110fb0 con 0x7f3a9c100e20 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.505+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 0x7f3a9c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a9b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 0x7f3a9c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38830/0 (socket says 172.21.15.64:38830) 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 -- 172.21.15.64:0/1431978786 learned_addr learned my addr 172.21.15.64:0/1431978786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a9ad9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a9c100e20 0x7f3a9c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 -- 172.21.15.64:0/1431978786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a9c100e20 msgr2=0x7f3a9c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a9c100e20 0x7f3a9c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 -- 172.21.15.64:0/1431978786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a9c102ef0 msgr2=0x7f3a9c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 -- 172.21.15.64:0/1431978786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c19d3b0 con 0x7f3a9c102570 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a93fff640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 0x7f3a9c198940 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400d950 tx=0x7f3a8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3a98d99640 1 -- 172.21.15.64:0/1431978786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84017070 con 0x7f3a9c102570 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.506+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c19d6a0 con 0x7f3a9c102570 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.507+0000 7f3a98d99640 1 -- 172.21.15.64:0/1431978786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a84009c30 con 0x7f3a9c102570 2024-09-19T05:09:12.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.507+0000 7f3a98d99640 1 -- 172.21.15.64:0/1431978786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400f620 con 0x7f3a9c102570 2024-09-19T05:09:12.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.507+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c1a4f80 con 0x7f3a9c102570 2024-09-19T05:09:12.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.510+0000 7f3a98d99640 1 -- 172.21.15.64:0/1431978786 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3a8400f7c0 con 0x7f3a9c102570 2024-09-19T05:09:12.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.510+0000 7f3aa3435740 1 Processor -- start 2024-09-19T05:09:12.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.510+0000 7f3aa3435740 1 -- start start 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x5603502747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350275130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x560350286110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3aa3435740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5603501d62e0 con 0x560350283d20 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3aa3435740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56035017dce0 con 0x560350275de0 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3aa3435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603501d22a0 con 0x560350274d10 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x560350286110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x560350286110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38844/0 (socket says 172.21.15.64:38844) 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 -- 172.21.15.64:0/2003135599 learned_addr learned my addr 172.21.15.64:0/2003135599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350275130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x5603502747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a98d99640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3a880a33b0 0x7f3a880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3a880a33b0 0x7f3a880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 -- 172.21.15.64:0/2003135599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 msgr2=0x560350275130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350275130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 -- 172.21.15.64:0/2003135599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 msgr2=0x5603502747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.511+0000 7f3a93fff640 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x5603502747d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a98d99640 1 -- 172.21.15.64:0/1431978786 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3a840cf890 con 0x7f3a9c102570 2024-09-19T05:09:12.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a93fff640 1 -- 172.21.15.64:0/2003135599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560350162440 con 0x560350283d20 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a93fff640 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x560350286110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3a840ccf50 tx=0x7f3a8400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7defd8c7a344cb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3a880a33b0 0x7f3a880a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a94004ad0 tx=0x7f3a9402a950 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a917fa640 1 -- 172.21.15.64:0/2003135599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84020070 con 0x560350283d20 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a917fa640 1 -- 172.21.15.64:0/2003135599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a84009de0 con 0x560350283d20 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.512+0000 7f3a917fa640 1 -- 172.21.15.64:0/2003135599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a47e0 con 0x560350283d20 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 -- 172.21.15.64:0/2003135599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 msgr2=0x560350286110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x560350286110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3a840ccf50 tx=0x7f3a8400bf60 comp rx=0 tx=0).stop 2024-09-19T05:09:12.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 -- 172.21.15.64:0/2003135599 shutdown_connections 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x560350286110 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350275130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2003135599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x5603502747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 -- 172.21.15.64:0/2003135599 >> 172.21.15.64:0/2003135599 conn(0x560350270e40 msgr2=0x560350271220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.513+0000 7f3aa3435740 1 -- 172.21.15.64:0/2003135599 shutdown_connections 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.514+0000 7f3aa3435740 1 -- 172.21.15.64:0/2003135599 wait complete. 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.514+0000 7f3aa3435740 1 -- start start 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.514+0000 7f3aa3435740 1 Processor -- start 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x56035026fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x56035026d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x56035026d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5603502869f0 con 0x560350283d20 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560350191050 con 0x560350275de0 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560350286b70 con 0x560350274d10 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3aa3435740 10 client.0 _ll_register_callbacks cb 0x5603501d1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x56035026d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x56035026d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38846/0 (socket says 172.21.15.64:38846) 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 -- 172.21.15.64:0/272048946 learned_addr learned my addr 172.21.15.64:0/272048946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x56035026fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 -- 172.21.15.64:0/272048946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 msgr2=0x56035026fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x56035026fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 -- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 msgr2=0x56035026d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x56035026d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 -- 172.21.15.64:0/272048946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034ff473c0 con 0x560350283d20 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x56035026d6c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009ca0 tx=0x7f3a940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2834e1702ea626ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.515+0000 7f3a827fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.516+0000 7f3a827fc640 1 -- 172.21.15.64:0/272048946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94043050 con 0x560350283d20 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.516+0000 7f3a827fc640 1 -- 172.21.15.64:0/272048946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3a94004510 con 0x560350283d20 2024-09-19T05:09:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.516+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a84009de0 con 0x560350283d20 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.516+0000 7f3a827fc640 1 -- 172.21.15.64:0/272048946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a94034040 con 0x560350283d20 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.516+0000 7f3aa3435740 10 client.16125 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.516+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560350292420 con 0x560350283d20 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3a827fc640 1 -- 172.21.15.64:0/272048946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a84009de0 con 0x560350283d20 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3a827fc640 1 -- 172.21.15.64:0/272048946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a94030440 con 0x560350283d20 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3aa3435740 10 client.16125 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3aa3435740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3aa3435740 20 client.16125 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3aa3435740 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350295d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3aa3435740 4 client.16125 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603501e3850 con 0x5603502938e0 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.517+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350295d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.518+0000 7f3a827fc640 10 client.16125 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.518+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350295d50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6fb8c829da02e1b server_cookie=cc4c9ab5c152fb03 in_seq=0 out_seq=0 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.518+0000 7f3a827fc640 1 -- 172.21.15.64:0/272048946 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603501e3850 con 0x5603502938e0 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.518+0000 7f3a827fc640 10 client.16125 handle_command_reply: tid=0 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.518+0000 7f3aa3435740 1 client.16125 shutdown 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 msgr2=0x56035026d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x56035026d6c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009ca0 tx=0x7f3a940095c0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 shutdown_connections 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350295d50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x56035026d6c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x56035026d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 --2- 172.21.15.64:0/272048946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x56035026fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3a827fc640 0 client.16125 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 >> 172.21.15.64:0/272048946 conn(0x560350270e40 msgr2=0x560350276b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 shutdown_connections 2024-09-19T05:09:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 1 -- 172.21.15.64:0/272048946 wait complete. 2024-09-19T05:09:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.519+0000 7f3aa3435740 20 client.16125 trim_cache size 0 max 16384 2024-09-19T05:09:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.539+0000 7f3aa3435740 1 Processor -- start 2024-09-19T05:09:12.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 -- start start 2024-09-19T05:09:12.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 0x56035028fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350275de0 0x56035026ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350283d20 0x56035026bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1adf10 con 0x560350274d10 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1ab7e0 con 0x560350283d20 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3aa3435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1a9650 con 0x560350275de0 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3a9ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 0x56035028fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3a9ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 0x56035028fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38852/0 (socket says 172.21.15.64:38852) 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3a9ad9d640 1 -- 172.21.15.64:0/3591193984 learned_addr learned my addr 172.21.15.64:0/3591193984 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350283d20 0x56035026bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.540+0000 7f3a93fff640 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350275de0 0x56035026ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a9ad9d640 1 -- 172.21.15.64:0/3591193984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350275de0 msgr2=0x56035026ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350275de0 0x56035026ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a9ad9d640 1 -- 172.21.15.64:0/3591193984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350283d20 msgr2=0x56035026bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350283d20 0x56035026bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a9ad9d640 1 -- 172.21.15.64:0/3591193984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035026d920 con 0x560350274d10 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 0x56035028fc90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00ba00 tx=0x7f3a8c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8db38777d6fbe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a90ff9640 1 -- 172.21.15.64:0/3591193984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c003cd0 con 0x560350274d10 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a90ff9640 1 -- 172.21.15.64:0/3591193984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a8c003e70 con 0x560350274d10 2024-09-19T05:09:12.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.541+0000 7f3a90ff9640 1 -- 172.21.15.64:0/3591193984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c011440 con 0x560350274d10 2024-09-19T05:09:12.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 -- 172.21.15.64:0/3591193984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 msgr2=0x56035028fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 0x56035028fc90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00ba00 tx=0x7f3a8c00b390 comp rx=0 tx=0).stop 2024-09-19T05:09:12.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 -- 172.21.15.64:0/3591193984 shutdown_connections 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350283d20 0x56035026bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350275de0 0x56035026ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 --2- 172.21.15.64:0/3591193984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350274d10 0x56035028fc90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 -- 172.21.15.64:0/3591193984 >> 172.21.15.64:0/3591193984 conn(0x560350270e40 msgr2=0x56035027fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 -- 172.21.15.64:0/3591193984 shutdown_connections 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.542+0000 7f3aa3435740 1 -- 172.21.15.64:0/3591193984 wait complete. 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 -- start start 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 Processor -- start 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350282c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x560350283170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x5603502836b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1a9ca0 con 0x560350283d20 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1ab110 con 0x560350275de0 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1aa370 con 0x560350274d10 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3aa3435740 10 client.0 _ll_register_callbacks cb 0x7f3a9402fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3a9b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x5603502836b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3a9b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x5603502836b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38866/0 (socket says 172.21.15.64:38866) 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3a9b59e640 1 -- 172.21.15.64:0/2171331826 learned_addr learned my addr 172.21.15.64:0/2171331826 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3a93fff640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x560350283170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.543+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350282c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a9b59e640 1 -- 172.21.15.64:0/2171331826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 msgr2=0x560350282c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350282c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a9b59e640 1 -- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 msgr2=0x560350283170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x560350283170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a9b59e640 1 -- 172.21.15.64:0/2171331826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603502fd750 con 0x560350283d20 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a9b59e640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x5603502836b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3a9402ff00 tx=0x7f3a94005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15624a69fa7e183a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a827fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.544+0000 7f3a827fc640 1 -- 172.21.15.64:0/2171331826 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94043070 con 0x560350283d20 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3a827fc640 1 -- 172.21.15.64:0/2171331826 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3a94004720 con 0x560350283d20 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8c003e70 con 0x560350283d20 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3a827fc640 1 -- 172.21.15.64:0/2171331826 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a9402f8c0 con 0x560350283d20 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 10 client.16131 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603502fea40 con 0x560350283d20 2024-09-19T05:09:12.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3a827fc640 1 -- 172.21.15.64:0/2171331826 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a8c003e70 con 0x560350283d20 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3a827fc640 1 -- 172.21.15.64:0/2171331826 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a94054440 con 0x560350283d20 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 10 client.16131 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 20 client.16131 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350301170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 4 client.16131 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.545+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603501ddd60 con 0x5603502938e0 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350301170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3a9ad9d640 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350301170 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3117a87b4f8188 server_cookie=a94938996724ec0 in_seq=0 out_seq=0 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3a827fc640 10 client.16131 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3a827fc640 1 -- 172.21.15.64:0/2171331826 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5603501ddd60 con 0x5603502938e0 2024-09-19T05:09:12.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3a827fc640 10 client.16131 handle_command_reply: tid=0 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3aa3435740 1 client.16131 shutdown 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 msgr2=0x5603502836b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.546+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x5603502836b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3a9402ff00 tx=0x7f3a94005d10 comp rx=0 tx=0).stop 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 shutdown_connections 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5603502938e0 0x560350301170 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560350283d20 0x5603502836b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560350275de0 0x560350283170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3a827fc640 0 client.16131 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 --2- 172.21.15.64:0/2171331826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560350274d10 0x560350282c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 >> 172.21.15.64:0/2171331826 conn(0x560350270e40 msgr2=0x56035026e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 shutdown_connections 2024-09-19T05:09:12.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 1 -- 172.21.15.64:0/2171331826 wait complete. 2024-09-19T05:09:12.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.547+0000 7f3aa3435740 20 client.16131 trim_cache size 0 max 16384 2024-09-19T05:09:12.553 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:12.553 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:09:12.553 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:09:12.553 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3a880a33b0 msgr2=0x7f3a880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3a880a33b0 0x7f3a880a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a94004ad0 tx=0x7f3a9402a950 comp rx=0 tx=0).stop 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 msgr2=0x7f3a9c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 0x7f3a9c198940 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400d950 tx=0x7f3a8400de10 comp rx=0 tx=0).stop 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 shutdown_connections 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3a880a33b0 0x7f3a880a5870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a9c102570 0x7f3a9c198940 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 --2- 172.21.15.64:0/1431978786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a9c100e20 0x7f3a9c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 >> 172.21.15.64:0/1431978786 conn(0x7f3a9c0fc7d0 msgr2=0x7f3a9c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:12.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 shutdown_connections 2024-09-19T05:09:12.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:12.548+0000 7f3aa105c640 1 -- 172.21.15.64:0/1431978786 wait complete. 2024-09-19T05:09:13.353 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:09:13.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:09:13.644 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:13 smithi064 ceph-mon[30960]: pgmap v102: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 199 KiB/s wr, 25 op/s 2024-09-19T05:09:13.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:13 smithi103 ceph-mon[34446]: pgmap v102: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 199 KiB/s wr, 25 op/s 2024-09-19T05:09:13.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:13 smithi174 ceph-mon[34488]: pgmap v102: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 199 KiB/s wr, 25 op/s 2024-09-19T05:09:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/1268460021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0106f10 msgr2=0x7fd9c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/1268460021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0106f10 0x7fd9c010ea90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009ca0 tx=0x7fd9b802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/1268460021 shutdown_connections 2024-09-19T05:09:14.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/1268460021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0106f10 0x7fd9c010ea90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/1268460021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c0102790 0x7fd9c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/1268460021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 0x7fd9c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/1268460021 >> 172.21.15.64:0/1268460021 conn(0x7fd9c00fd8e0 msgr2=0x7fd9c00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/1268460021 shutdown_connections 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.136+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/1268460021 wait complete. 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.137+0000 7fd9c7a2f640 1 Processor -- start 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.137+0000 7fd9c7a2f640 1 -- start start 2024-09-19T05:09:14.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c7a2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 0x7fd9c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:14.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c7a2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 0x7fd9c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:14.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c4fca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 0x7fd9c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:14.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c57cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 0x7fd9c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:14.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c4fca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 0x7fd9c0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38882/0 (socket says 172.21.15.64:38882) 2024-09-19T05:09:14.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c4fca640 1 -- 172.21.15.64:0/194170504 learned_addr learned my addr 172.21.15.64:0/194170504 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c0106f10 0x7fd9c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0111e20 con 0x7fd9c0102790 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0111ca0 con 0x7fd9c0101de0 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0111fa0 con 0x7fd9c0106f10 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.138+0000 7fd9c5fcc640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c0106f10 0x7fd9c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c4fca640 1 -- 172.21.15.64:0/194170504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c0106f10 msgr2=0x7fd9c0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:14.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c4fca640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c0106f10 0x7fd9c0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c4fca640 1 -- 172.21.15.64:0/194170504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 msgr2=0x7fd9c00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c4fca640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 0x7fd9c00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c4fca640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c01a67c0 con 0x7fd9c0102790 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c57cb640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 0x7fd9c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c4fca640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 0x7fd9c0074a30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000d770 tx=0x7fd9b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0017070 con 0x7fd9c0102790 2024-09-19T05:09:14.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9b00054d0 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0004cf0 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c01a6ab0 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.139+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c01a6fa0 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.140+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c01021c0 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.142+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd9b0004e90 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.142+0000 7fd9b67fc640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.142+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fd9b00cf6f0 con 0x7fd9c0102790 2024-09-19T05:09:14.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.142+0000 7fd9c57cb640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:14.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.143+0000 7fd9c57cb640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00bfd0 tx=0x7fd9bc00b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:14.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.147+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9b009c0d0 con 0x7fd9c0102790 2024-09-19T05:09:14.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.433+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9c010a120 con 0x7fd9c0102790 2024-09-19T05:09:14.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.434+0000 7fd9b67fc640 1 -- 172.21.15.64:0/194170504 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fd9b009b9d0 con 0x7fd9c0102790 2024-09-19T05:09:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:09:14.438 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9ac0a33b0 msgr2=0x7fd9ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00bfd0 tx=0x7fd9bc00b390 comp rx=0 tx=0).stop 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 msgr2=0x7fd9c0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 0x7fd9c0074a30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000d770 tx=0x7fd9b000dc30 comp rx=0 tx=0).stop 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 shutdown_connections 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c0106f10 0x7fd9c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c0102790 0x7fd9c0074a30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 --2- 172.21.15.64:0/194170504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9c0101de0 0x7fd9c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:14.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 >> 172.21.15.64:0/194170504 conn(0x7fd9c00fd8e0 msgr2=0x7fd9c00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:14.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 shutdown_connections 2024-09-19T05:09:14.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:14.437+0000 7fd9c7a2f640 1 -- 172.21.15.64:0/194170504 wait complete. 2024-09-19T05:09:15.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:09:15.354 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:15 smithi064 ceph-mon[30960]: pgmap v103: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 198 KiB/s wr, 22 op/s 2024-09-19T05:09:15.354 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:15 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/194170504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:15.413 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:15.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:15 smithi103 ceph-mon[34446]: pgmap v103: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 198 KiB/s wr, 22 op/s 2024-09-19T05:09:15.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:15 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/194170504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:15.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:15 smithi174 ceph-mon[34488]: pgmap v103: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 198 KiB/s wr, 22 op/s 2024-09-19T05:09:15.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:15 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/194170504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:15.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 -- 172.21.15.64:0/79681345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37641009b0 msgr2=0x7f3764100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 --2- 172.21.15.64:0/79681345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37641009b0 0x7f3764100d90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3760009900 tx=0x7f376002a410 comp rx=0 tx=0).stop 2024-09-19T05:09:15.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 -- 172.21.15.64:0/79681345 shutdown_connections 2024-09-19T05:09:15.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 --2- 172.21.15.64:0/79681345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376410e4d0 0x7f37641108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 --2- 172.21.15.64:0/79681345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101360 0x7f376410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 --2- 172.21.15.64:0/79681345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37641009b0 0x7f3764100d90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 -- 172.21.15.64:0/79681345 >> 172.21.15.64:0/79681345 conn(0x7f37640fc7d0 msgr2=0x7f37640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 -- 172.21.15.64:0/79681345 shutdown_connections 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 -- 172.21.15.64:0/79681345 wait complete. 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.906+0000 7f376b4b5640 1 Processor -- start 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 -- start start 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37641009b0 0x7f376419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101360 0x7f376419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 0x7f376419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37641140c0 con 0x7f376410e4d0 2024-09-19T05:09:15.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3764113f40 con 0x7f37641009b0 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f376b4b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3764114240 con 0x7f3764101360 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 0x7f376419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 0x7f376419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48648/0 (socket says 172.21.15.64:48648) 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 -- 172.21.15.64:0/2303007314 learned_addr learned my addr 172.21.15.64:0/2303007314 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3768a50640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101360 0x7f376419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769251640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37641009b0 0x7f376419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 -- 172.21.15.64:0/2303007314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101360 msgr2=0x7f376419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101360 0x7f376419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 -- 172.21.15.64:0/2303007314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37641009b0 msgr2=0x7f376419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37641009b0 0x7f376419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.907+0000 7f3769a52640 1 -- 172.21.15.64:0/2303007314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419f610 con 0x7f376410e4d0 2024-09-19T05:09:15.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.908+0000 7f3769a52640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 0x7f376419ef30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f375c00eaa0 tx=0x7f375c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:15.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.908+0000 7f375a7fc640 1 -- 172.21.15.64:0/2303007314 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c00cdb0 con 0x7f376410e4d0 2024-09-19T05:09:15.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.908+0000 7f375a7fc640 1 -- 172.21.15.64:0/2303007314 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375c005020 con 0x7f376410e4d0 2024-09-19T05:09:15.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.908+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419f900 con 0x7f376410e4d0 2024-09-19T05:09:15.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.908+0000 7f375a7fc640 1 -- 172.21.15.64:0/2303007314 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0116d0 con 0x7f376410e4d0 2024-09-19T05:09:15.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.908+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a71e0 con 0x7f376410e4d0 2024-09-19T05:09:15.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.911+0000 7f375a7fc640 1 -- 172.21.15.64:0/2303007314 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f375c0051d0 con 0x7f376410e4d0 2024-09-19T05:09:15.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.911+0000 7f376d88e740 1 Processor -- start 2024-09-19T05:09:15.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 -- start start 2024-09-19T05:09:15.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c20b40 0x55db16c1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c1f5d0 0x55db16c1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55db16b852f0 con 0x55db16c2a370 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55db16b2ccf0 con 0x55db16c1f5d0 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f376d88e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db16b812b0 con 0x55db16c20b40 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f3768a50640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f3768a50640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c2c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48660/0 (socket says 172.21.15.64:48660) 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f3768a50640 1 -- 172.21.15.64:0/1336627719 learned_addr learned my addr 172.21.15.64:0/1336627719 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f3768a50640 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c20b40 0x55db16c1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.912+0000 7f3769251640 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c1f5d0 0x55db16c1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3768a50640 1 -- 172.21.15.64:0/1336627719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c20b40 msgr2=0x55db16c1f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3768a50640 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c20b40 0x55db16c1f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3768a50640 1 -- 172.21.15.64:0/1336627719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c1f5d0 msgr2=0x55db16c1fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3768a50640 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c1f5d0 0x55db16c1fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3768a50640 1 -- 172.21.15.64:0/1336627719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db16b113a0 con 0x55db16c2a370 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3769251640 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c1f5d0 0x55db16c1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f375a7fc640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f37500a3430 0x7f37500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3769251640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f37500a3430 0x7f37500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f3768a50640 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c2c760 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f37540127c0 tx=0x7f375400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf25a418c40ff2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f375a7fc640 1 -- 172.21.15.64:0/2303007314 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f375c0cfd60 con 0x7f376410e4d0 2024-09-19T05:09:15.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f374f7fe640 1 -- 172.21.15.64:0/1336627719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754012e00 con 0x55db16c2a370 2024-09-19T05:09:15.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f374f7fe640 1 -- 172.21.15.64:0/1336627719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754010c20 con 0x55db16c2a370 2024-09-19T05:09:15.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.913+0000 7f374f7fe640 1 -- 172.21.15.64:0/1336627719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375401ea70 con 0x55db16c2a370 2024-09-19T05:09:15.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 -- 172.21.15.64:0/1336627719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 msgr2=0x55db16c2c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c2c760 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f37540127c0 tx=0x7f375400b7c0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f3769251640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f37500a3430 0x7f37500a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3760031440 tx=0x7f3760005890 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:15.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 -- 172.21.15.64:0/1336627719 shutdown_connections 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c2c760 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c1f5d0 0x55db16c1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 --2- 172.21.15.64:0/1336627719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c20b40 0x55db16c1f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 -- 172.21.15.64:0/1336627719 >> 172.21.15.64:0/1336627719 conn(0x55db16be8310 msgr2=0x55db16c1bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 -- 172.21.15.64:0/1336627719 shutdown_connections 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.914+0000 7f376d88e740 1 -- 172.21.15.64:0/1336627719 wait complete. 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 -- start start 2024-09-19T05:09:15.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 Processor -- start 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 0x55db16c29d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c20b40 0x55db16c26ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c2a370 0x55db16c275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55db16c2d070 con 0x55db16c1f5d0 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55db16b40060 con 0x55db16c20b40 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db16c2d1f0 con 0x55db16c2a370 2024-09-19T05:09:15.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f3769251640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 0x55db16c29d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f3769251640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 0x55db16c29d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48670/0 (socket says 172.21.15.64:48670) 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f3769251640 1 -- 172.21.15.64:0/4205092512 learned_addr learned my addr 172.21.15.64:0/4205092512 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f376d88e740 10 client.0 _ll_register_callbacks cb 0x55db168938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.915+0000 7f3768a50640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c20b40 0x55db16c26ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3769a52640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c2a370 0x55db16c275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3768a50640 1 -- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c2a370 msgr2=0x55db16c275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3768a50640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c2a370 0x55db16c275e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3768a50640 1 -- 172.21.15.64:0/4205092512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 msgr2=0x55db16c29d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3768a50640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 0x55db16c29d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3768a50640 1 -- 172.21.15.64:0/4205092512 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db168f63c0 con 0x55db16c20b40 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3769251640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 0x55db16c29d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:15.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f3768a50640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c20b40 0x55db16c26ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f375401e690 tx=0x7f375400b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f59b323169bb3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f374cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f374cff9640 1 -- 172.21.15.64:0/4205092512 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375402c060 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f374cff9640 1 -- 172.21.15.64:0/4205092512 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3754005050 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.916+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3754010c20 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f374cff9640 1 -- 172.21.15.64:0/4205092512 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3754005250 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 10 client.25802 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db16c38ae0 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f374cff9640 1 -- 172.21.15.64:0/4205092512 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754010c20 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f374cff9640 1 -- 172.21.15.64:0/4205092512 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37540046c0 con 0x55db16c20b40 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 10 client.25802 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 10 client.25802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:09:15.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 20 client.25802 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16c39ee0 0x55db16c3c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 4 client.25802 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db16b92860 con 0x55db16c39ee0 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.917+0000 7f3769251640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16c39ee0 0x55db16c3c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.918+0000 7f3769251640 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16c39ee0 0x55db16c3c2d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b91cbdcd378cbf server_cookie=8c25abc649cc5488 in_seq=0 out_seq=0 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.918+0000 7f374cff9640 10 client.25802 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f374cff9640 1 -- 172.21.15.64:0/4205092512 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db16b92860 con 0x55db16c39ee0 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f374cff9640 10 client.25802 handle_command_reply: tid=0 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 client.25802 shutdown 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c20b40 msgr2=0x55db16c26ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c20b40 0x55db16c26ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f375401e690 tx=0x7f375400b400 comp rx=0 tx=0).stop 2024-09-19T05:09:15.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 shutdown_connections 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16c39ee0 0x55db16c3c2d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c2a370 0x55db16c275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c20b40 0x55db16c26ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 --2- 172.21.15.64:0/4205092512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c1f5d0 0x55db16c29d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 >> 172.21.15.64:0/4205092512 conn(0x55db16be8310 msgr2=0x55db16c21960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f374cff9640 0 client.25802 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 shutdown_connections 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 1 -- 172.21.15.64:0/4205092512 wait complete. 2024-09-19T05:09:15.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.919+0000 7f376d88e740 20 client.25802 trim_cache size 0 max 16384 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.932+0000 7f376d88e740 1 Processor -- start 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.932+0000 7f376d88e740 1 -- start start 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f376d88e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c36310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f376d88e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 0x55db16c28f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f376d88e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2a370 0x55db16c419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f376d88e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37641b2360 con 0x55db16c20b40 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f376d88e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37641afc30 con 0x55db16c2a370 2024-09-19T05:09:15.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f376d88e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37641ada80 con 0x55db16c1f5d0 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 0x55db16c28f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 0x55db16c28f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48674/0 (socket says 172.21.15.64:48674) 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 -- 172.21.15.64:0/2936169221 learned_addr learned my addr 172.21.15.64:0/2936169221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3769251640 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c36310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3769a52640 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2a370 0x55db16c419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 -- 172.21.15.64:0/2936169221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 msgr2=0x55db16c36310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c36310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 -- 172.21.15.64:0/2936169221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2a370 msgr2=0x55db16c419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2a370 0x55db16c419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.933+0000 7f3768a50640 1 -- 172.21.15.64:0/2936169221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db16c20300 con 0x55db16c20b40 2024-09-19T05:09:15.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.934+0000 7f3769251640 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c36310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.934+0000 7f3768a50640 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 0x55db16c28f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3754012580 tx=0x7f375400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0abad104b4565f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.934+0000 7f374effd640 1 -- 172.21.15.64:0/2936169221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375402c040 con 0x55db16c20b40 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.934+0000 7f374effd640 1 -- 172.21.15.64:0/2936169221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754016040 con 0x55db16c20b40 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.934+0000 7f374effd640 1 -- 172.21.15.64:0/2936169221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754004fa0 con 0x55db16c20b40 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 -- 172.21.15.64:0/2936169221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 msgr2=0x55db16c28f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 0x55db16c28f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3754012580 tx=0x7f375400b390 comp rx=0 tx=0).stop 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 -- 172.21.15.64:0/2936169221 shutdown_connections 2024-09-19T05:09:15.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2a370 0x55db16c419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c20b40 0x55db16c28f60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 --2- 172.21.15.64:0/2936169221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c36310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 -- 172.21.15.64:0/2936169221 >> 172.21.15.64:0/2936169221 conn(0x55db16be8310 msgr2=0x55db16c28850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 -- 172.21.15.64:0/2936169221 shutdown_connections 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.935+0000 7f376d88e740 1 -- 172.21.15.64:0/2936169221 wait complete. 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 -- start start 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 Processor -- start 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c32480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2f450 0x55db16c2f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37641ae0f0 con 0x55db16c2a370 2024-09-19T05:09:15.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37641af560 con 0x55db16c2f450 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37641ae7c0 con 0x55db16c1f5d0 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.936+0000 7f376d88e740 10 client.0 _ll_register_callbacks cb 0x7f375400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c329c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48686/0 (socket says 172.21.15.64:48686) 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 -- 172.21.15.64:0/4264964276 learned_addr learned my addr 172.21.15.64:0/4264964276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3769a52640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2f450 0x55db16c2f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3769251640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c32480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 -- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 msgr2=0x55db16c32480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c32480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 -- 172.21.15.64:0/4264964276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2f450 msgr2=0x55db16c2f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2f450 0x55db16c2f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 -- 172.21.15.64:0/4264964276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db16c2fe10 con 0x55db16c2a370 2024-09-19T05:09:15.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3769251640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c32480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:15.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f3768a50640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c329c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55db16c284f0 tx=0x7f3754012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8422ce198188f2f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:15.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.937+0000 7f374cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:15.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f374cff9640 1 -- 172.21.15.64:0/4264964276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375402c040 con 0x55db16c2a370 2024-09-19T05:09:15.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f374cff9640 1 -- 172.21.15.64:0/4264964276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f37540035f0 con 0x55db16c2a370 2024-09-19T05:09:15.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3754016040 con 0x55db16c2a370 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f374cff9640 1 -- 172.21.15.64:0/4264964276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f375400bad0 con 0x55db16c2a370 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f376d88e740 10 client.16155 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db16ca5540 con 0x55db16c2a370 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f374cff9640 1 -- 172.21.15.64:0/4264964276 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754010c20 con 0x55db16c2a370 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.938+0000 7f374cff9640 1 -- 172.21.15.64:0/4264964276 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f375401e690 con 0x55db16c2a370 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f376d88e740 10 client.16155 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f376d88e740 10 client.16155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:09:15.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f376d88e740 20 client.16155 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f376d88e740 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16ca5b50 0x55db16ca7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f376d88e740 4 client.16155 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3754016040 con 0x55db16ca5b50 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f3769251640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16ca5b50 0x55db16ca7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f3769251640 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16ca5b50 0x55db16ca7f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c05cf1bdaa952f15 server_cookie=8b2a3969d59830ef in_seq=0 out_seq=0 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.939+0000 7f374cff9640 10 client.16155 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f374cff9640 1 -- 172.21.15.64:0/4264964276 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3754016040 con 0x55db16ca5b50 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f374cff9640 10 client.16155 handle_command_reply: tid=0 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 client.16155 shutdown 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 msgr2=0x55db16c329c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c329c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55db16c284f0 tx=0x7f3754012690 comp rx=0 tx=0).stop 2024-09-19T05:09:15.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 shutdown_connections 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55db16ca5b50 0x55db16ca7f70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db16c2f450 0x55db16c2f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db16c2a370 0x55db16c329c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 --2- 172.21.15.64:0/4264964276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db16c1f5d0 0x55db16c32480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 >> 172.21.15.64:0/4264964276 conn(0x55db16be8310 msgr2=0x55db16c20fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f374cff9640 0 client.16155 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 shutdown_connections 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 1 -- 172.21.15.64:0/4264964276 wait complete. 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.940+0000 7f376d88e740 20 client.16155 trim_cache size 0 max 16384 2024-09-19T05:09:15.946 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f37500a3430 msgr2=0x7f37500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f37500a3430 0x7f37500a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3760031440 tx=0x7f3760005890 comp rx=0 tx=0).stop 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 msgr2=0x7f376419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 0x7f376419ef30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f375c00eaa0 tx=0x7f375c00ef60 comp rx=0 tx=0).stop 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 shutdown_connections 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f37500a3430 0x7f37500a58f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f376410e4d0 0x7f376419ef30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101360 0x7f376419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 --2- 172.21.15.64:0/2303007314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37641009b0 0x7f376419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 >> 172.21.15.64:0/2303007314 conn(0x7f37640fc7d0 msgr2=0x7f37640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 shutdown_connections 2024-09-19T05:09:15.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:15.941+0000 7f376b4b5640 1 -- 172.21.15.64:0/2303007314 wait complete. 2024-09-19T05:09:16.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:09:17.019 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:17.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:17 smithi064 ceph-mon[30960]: pgmap v104: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 198 KiB/s wr, 25 op/s 2024-09-19T05:09:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 -- 172.21.15.64:0/2750168879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 msgr2=0x7fe868103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/2750168879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe868103110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe864009930 tx=0x7fe86402a420 comp rx=0 tx=0).stop 2024-09-19T05:09:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 -- 172.21.15.64:0/2750168879 shutdown_connections 2024-09-19T05:09:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/2750168879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8681082a0 0x7fe86810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/2750168879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8681036e0 0x7fe868107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/2750168879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe868103110 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 -- 172.21.15.64:0/2750168879 >> 172.21.15.64:0/2750168879 conn(0x7fe8680fc7d0 msgr2=0x7fe8680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 -- 172.21.15.64:0/2750168879 shutdown_connections 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.492+0000 7fe86f6b3640 1 -- 172.21.15.64:0/2750168879 wait complete. 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.493+0000 7fe86f6b3640 1 Processor -- start 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.493+0000 7fe86f6b3640 1 -- start start 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.493+0000 7fe86f6b3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe8681984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.493+0000 7fe86f6b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8681036e0 0x7fe868198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.493+0000 7fe86f6b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8681082a0 0x7fe86819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86f6b3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe868111f50 con 0x7fe868102d30 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe8681984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86f6b3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe868111dd0 con 0x7fe8681036e0 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86f6b3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681120d0 con 0x7fe8681082a0 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe8681984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48714/0 (socket says 172.21.15.64:48714) 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 -- 172.21.15.64:0/3427147275 learned_addr learned my addr 172.21.15.64:0/3427147275 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8681036e0 0x7fe868198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86dc50640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8681082a0 0x7fe86819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 -- 172.21.15.64:0/3427147275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8681082a0 msgr2=0x7fe86819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8681082a0 0x7fe86819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 -- 172.21.15.64:0/3427147275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8681036e0 msgr2=0x7fe868198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8681036e0 0x7fe868198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 -- 172.21.15.64:0/3427147275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86819d4a0 con 0x7fe868102d30 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe8681984f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe864004770 tx=0x7fe8640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:17.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe85e7fc640 1 -- 172.21.15.64:0/3427147275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86403f070 con 0x7fe868102d30 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.494+0000 7fe85e7fc640 1 -- 172.21.15.64:0/3427147275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe86403b460 con 0x7fe868102d30 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.495+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86819d730 con 0x7fe868102d30 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.495+0000 7fe85e7fc640 1 -- 172.21.15.64:0/3427147275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864035070 con 0x7fe868102d30 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.495+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8681a5070 con 0x7fe868102d30 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 Processor -- start 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 -- start start 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe85e7fc640 1 -- 172.21.15.64:0/3427147275 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe86403b610 con 0x7fe868102d30 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c8271b40 0x5597c8270090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8270a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 0x5597c8281b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597c81d62f0 con 0x5597c827f780 2024-09-19T05:09:17.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597c817dcf0 con 0x5597c8271b40 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe871a8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597c81d22b0 con 0x5597c82705d0 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe86cc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 0x5597c8281b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe86cc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 0x5597c8281b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48724/0 (socket says 172.21.15.64:48724) 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe86cc4e640 1 -- 172.21.15.64:0/2313305454 learned_addr learned my addr 172.21.15.64:0/2313305454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe86dc50640 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c8271b40 0x5597c8270090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.498+0000 7fe86d44f640 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8270a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86cc4e640 1 -- 172.21.15.64:0/2313305454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 msgr2=0x5597c8270a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8270a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86cc4e640 1 -- 172.21.15.64:0/2313305454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c8271b40 msgr2=0x5597c8270090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c8271b40 0x5597c8270090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86cc4e640 1 -- 172.21.15.64:0/2313305454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c81623a0 con 0x5597c827f780 2024-09-19T05:09:17.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe85e7fc640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe8540a3430 0x7fe8540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 0x5597c8281b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe85800bc30 tx=0x7fe8580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cb3dba9758ada1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe85e7fc640 1 -- 172.21.15.64:0/3427147275 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe8640efcc0 con 0x7fe868102d30 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe86dc50640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe8540a3430 0x7fe8540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe8537fe640 1 -- 172.21.15.64:0/2313305454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85801c480 con 0x5597c827f780 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe8537fe640 1 -- 172.21.15.64:0/2313305454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85801cad0 con 0x5597c827f780 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.499+0000 7fe8537fe640 1 -- 172.21.15.64:0/2313305454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858011ae0 con 0x5597c827f780 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 -- 172.21.15.64:0/2313305454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 msgr2=0x5597c8281b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 0x5597c8281b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe85800bc30 tx=0x7fe8580107a0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe86dc50640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe8540a3430 0x7fe8540a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe860005fd0 tx=0x7fe8600095a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 -- 172.21.15.64:0/2313305454 shutdown_connections 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c827f780 0x5597c8281b70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8270a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 --2- 172.21.15.64:0/2313305454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c8271b40 0x5597c8270090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 -- 172.21.15.64:0/2313305454 >> 172.21.15.64:0/2313305454 conn(0x5597c8239310 msgr2=0x5597c826cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 -- 172.21.15.64:0/2313305454 shutdown_connections 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.500+0000 7fe871a8c740 1 -- 172.21.15.64:0/2313305454 wait complete. 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.501+0000 7fe871a8c740 1 -- start start 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.501+0000 7fe871a8c740 1 Processor -- start 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.501+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c82786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.501+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c8278c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c8278c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8279150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c8278c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48730/0 (socket says 172.21.15.64:48730) 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 -- 172.21.15.64:0/1226466103 learned_addr learned my addr 172.21.15.64:0/1226466103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:17.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597c8273550 con 0x5597c8271b40 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597c8191060 con 0x5597c827f780 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597c82736d0 con 0x5597c82705d0 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe871a8c740 10 client.0 _ll_register_callbacks cb 0x5597c7ee48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86d44f640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c82786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 -- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 msgr2=0x5597c82786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c82786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86dc50640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8279150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 -- 172.21.15.64:0/1226466103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 msgr2=0x5597c8279150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8279150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.502+0000 7fe86cc4e640 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c7f473c0 con 0x5597c8271b40 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.503+0000 7fe86dc50640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8279150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.503+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c8278c10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe858011700 tx=0x7fe85800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f827515ada02e5fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:17.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.503+0000 7fe850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.503+0000 7fe850ff9640 1 -- 172.21.15.64:0/1226466103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85802b060 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.503+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85801cad0 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.503+0000 7fe850ff9640 1 -- 172.21.15.64:0/1226466103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe85801c690 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe850ff9640 1 -- 172.21.15.64:0/1226466103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe85801c890 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 10 client.16167 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597c8289b00 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe850ff9640 1 -- 172.21.15.64:0/1226466103 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85801cad0 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe850ff9640 1 -- 172.21.15.64:0/1226466103 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe858005040 con 0x5597c8271b40 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 10 client.16167 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 10 client.16167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 20 client.16167 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c828aeb0 0x5597c828d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 4 client.16167 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597c81e3860 con 0x5597c828aeb0 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.504+0000 7fe86d44f640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c828aeb0 0x5597c828d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.505+0000 7fe850ff9640 10 client.16167 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.505+0000 7fe86d44f640 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c828aeb0 0x5597c828d2a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59d6a5f9cefc4b04 server_cookie=75c2fbd0d13a7e74 in_seq=0 out_seq=0 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe850ff9640 1 -- 172.21.15.64:0/1226466103 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597c81e3860 con 0x5597c828aeb0 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe850ff9640 10 client.16167 handle_command_reply: tid=0 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 client.16167 shutdown 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 msgr2=0x5597c8278c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c8278c10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe858011700 tx=0x7fe85800b980 comp rx=0 tx=0).stop 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 shutdown_connections 2024-09-19T05:09:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8279150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c828aeb0 0x5597c828d2a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c8278c10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 --2- 172.21.15.64:0/1226466103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c82786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe850ff9640 0 client.16167 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 >> 172.21.15.64:0/1226466103 conn(0x5597c8239310 msgr2=0x5597c8272960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 shutdown_connections 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 1 -- 172.21.15.64:0/1226466103 wait complete. 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.506+0000 7fe871a8c740 20 client.16167 trim_cache size 0 max 16384 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.519+0000 7fe871a8c740 1 Processor -- start 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.519+0000 7fe871a8c740 1 -- start start 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 0x5597c8278fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c8271b40 0x5597c8280b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c82810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe871a8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681adf60 con 0x5597c82705d0 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe871a8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681ab830 con 0x5597c827f780 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe871a8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681a9680 con 0x5597c8271b40 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 0x5597c8278fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 0x5597c8278fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48744/0 (socket says 172.21.15.64:48744) 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 -- 172.21.15.64:0/3547183091 learned_addr learned my addr 172.21.15.64:0/3547183091 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86dc50640 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c82810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c8271b40 0x5597c8280b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 -- 172.21.15.64:0/3547183091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c8271b40 msgr2=0x5597c8280b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c8271b40 0x5597c8280b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 -- 172.21.15.64:0/3547183091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 msgr2=0x5597c82810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c82810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 -- 172.21.15.64:0/3547183091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c827f010 con 0x5597c82705d0 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.520+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 0x5597c8278fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe864032ad0 tx=0x7fe8640feed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37849584e523f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe852ffd640 1 -- 172.21.15.64:0/3547183091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640f1040 con 0x5597c82705d0 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe852ffd640 1 -- 172.21.15.64:0/3547183091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8640bc6a0 con 0x5597c82705d0 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe852ffd640 1 -- 172.21.15.64:0/3547183091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864048040 con 0x5597c82705d0 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe871a8c740 1 -- 172.21.15.64:0/3547183091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 msgr2=0x5597c8278fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 0x5597c8278fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe864032ad0 tx=0x7fe8640feed0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe871a8c740 1 -- 172.21.15.64:0/3547183091 shutdown_connections 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c82810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.521+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c8271b40 0x5597c8280b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3547183091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c82705d0 0x5597c8278fa0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- 172.21.15.64:0/3547183091 >> 172.21.15.64:0/3547183091 conn(0x5597c8239310 msgr2=0x5597c8272930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- 172.21.15.64:0/3547183091 shutdown_connections 2024-09-19T05:09:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- 172.21.15.64:0/3547183091 wait complete. 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- start start 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 Processor -- start 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8287280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c82877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8287d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681a9cf0 con 0x5597c8271b40 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681ab160 con 0x5597c827f780 2024-09-19T05:09:17.542 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:17.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.522+0000 7fe871a8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681aa3c0 con 0x5597c82705d0 2024-09-19T05:09:17.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe871a8c740 10 client.0 _ll_register_callbacks cb 0x7fe860009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:17.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c82877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c82877c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48756/0 (socket says 172.21.15.64:48756) 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 -- 172.21.15.64:0/3704605490 learned_addr learned my addr 172.21.15.64:0/3704605490 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8287280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86dc50640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8287d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 -- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 msgr2=0x5597c8287280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8287280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 -- 172.21.15.64:0/3704605490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 msgr2=0x5597c8287d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8287d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 -- 172.21.15.64:0/3704605490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c8274c20 con 0x5597c8271b40 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe86cc4e640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c82877c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b950 tx=0x7fe85800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859f7906bb09a3fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe850ff9640 1 -- 172.21.15.64:0/3704605490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85802b040 con 0x5597c8271b40 2024-09-19T05:09:17.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe850ff9640 1 -- 172.21.15.64:0/3704605490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe85801c690 con 0x5597c8271b40 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.523+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8640bc6a0 con 0x5597c8271b40 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe850ff9640 1 -- 172.21.15.64:0/3704605490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe858005650 con 0x5597c8271b40 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 10 client.16173 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597c8275f20 con 0x5597c8271b40 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe850ff9640 1 -- 172.21.15.64:0/3704605490 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8640bc6a0 con 0x5597c8271b40 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe850ff9640 1 -- 172.21.15.64:0/3704605490 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe858036020 con 0x5597c8271b40 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 10 client.16173 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 10 client.16173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 20 client.16173 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c82f68b0 0x5597c82f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 4 client.16173 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:17.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.524+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597c81ddd70 con 0x5597c82f68b0 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.525+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c82f68b0 0x5597c82f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.525+0000 7fe86d44f640 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c82f68b0 0x5597c82f8ca0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8e83910a857b14f server_cookie=40a548ee441a27b4 in_seq=0 out_seq=0 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.525+0000 7fe850ff9640 10 client.16173 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.525+0000 7fe850ff9640 1 -- 172.21.15.64:0/3704605490 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597c81ddd70 con 0x5597c82f68b0 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.525+0000 7fe850ff9640 10 client.16173 handle_command_reply: tid=0 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.525+0000 7fe871a8c740 1 client.16173 shutdown 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 msgr2=0x5597c82877c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c82877c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b950 tx=0x7fe85800b390 comp rx=0 tx=0).stop 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 shutdown_connections 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597c827f780 0x5597c8287d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5597c82f68b0 0x5597c82f8ca0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597c8271b40 0x5597c82877c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 --2- 172.21.15.64:0/3704605490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597c82705d0 0x5597c8287280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 >> 172.21.15.64:0/3704605490 conn(0x5597c8239310 msgr2=0x5597c8272010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe850ff9640 0 client.16173 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 shutdown_connections 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.526+0000 7fe871a8c740 1 -- 172.21.15.64:0/3704605490 wait complete. 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe871a8c740 20 client.16173 trim_cache size 0 max 16384 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe8540a3430 msgr2=0x7fe8540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe8540a3430 0x7fe8540a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe860005fd0 tx=0x7fe8600095a0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 msgr2=0x7fe8681984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe8681984f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe864004770 tx=0x7fe8640042f0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 shutdown_connections 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.527+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe8540a3430 0x7fe8540a58f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.528+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8681082a0 0x7fe86819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.528+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8681036e0 0x7fe868198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.528+0000 7fe86f6b3640 1 --2- 172.21.15.64:0/3427147275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe868102d30 0x7fe8681984f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:17.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.528+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 >> 172.21.15.64:0/3427147275 conn(0x7fe8680fc7d0 msgr2=0x7fe868105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:17.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.528+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 shutdown_connections 2024-09-19T05:09:17.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:17.528+0000 7fe86f6b3640 1 -- 172.21.15.64:0/3427147275 wait complete. 2024-09-19T05:09:17.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:17 smithi103 ceph-mon[34446]: pgmap v104: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 198 KiB/s wr, 25 op/s 2024-09-19T05:09:17.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:17 smithi174 ceph-mon[34488]: pgmap v104: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 198 KiB/s wr, 25 op/s 2024-09-19T05:09:18.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:09:18.819 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:19.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.340+0000 7f07e44c9640 1 -- 172.21.15.64:0/356945241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc108290 msgr2=0x7f07dc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.340+0000 7f07e44c9640 1 --2- 172.21.15.64:0/356945241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc108290 0x7f07dc10ec30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009cd0 tx=0x7f07d402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 -- 172.21.15.64:0/356945241 shutdown_connections 2024-09-19T05:09:19.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 --2- 172.21.15.64:0/356945241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc108290 0x7f07dc10ec30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 --2- 172.21.15.64:0/356945241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc103740 0x7f07dc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 --2- 172.21.15.64:0/356945241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc102d90 0x7f07dc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 -- 172.21.15.64:0/356945241 >> 172.21.15.64:0/356945241 conn(0x7f07dc0fc7d0 msgr2=0x7f07dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 -- 172.21.15.64:0/356945241 shutdown_connections 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 -- 172.21.15.64:0/356945241 wait complete. 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 Processor -- start 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.341+0000 7f07e44c9640 1 -- start start 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e44c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102d90 0x7f07dc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e44c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 0x7f07dc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e44c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108290 0x7f07dc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e44c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc111f40 con 0x7f07dc103740 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e44c9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc111dc0 con 0x7f07dc102d90 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e44c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1120c0 con 0x7f07dc108290 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 0x7f07dc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 0x7f07dc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48782/0 (socket says 172.21.15.64:48782) 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 -- 172.21.15.64:0/688436322 learned_addr learned my addr 172.21.15.64:0/688436322 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e2a66640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108290 0x7f07dc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e2265640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102d90 0x7f07dc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 -- 172.21.15.64:0/688436322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108290 msgr2=0x7f07dc19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108290 0x7f07dc19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 -- 172.21.15.64:0/688436322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102d90 msgr2=0x7f07dc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102d90 0x7f07dc19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.342+0000 7f07e1a64640 1 -- 172.21.15.64:0/688436322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc19f5e0 con 0x7f07dc103740 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.343+0000 7f07e1a64640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 0x7f07dc19ab70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00d770 tx=0x7f07cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.343+0000 7f07d37fe640 1 -- 172.21.15.64:0/688436322 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc017070 con 0x7f07dc103740 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.343+0000 7f07d37fe640 1 -- 172.21.15.64:0/688436322 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07cc009c30 con 0x7f07dc103740 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.343+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc19f8d0 con 0x7f07dc103740 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.343+0000 7f07d37fe640 1 -- 172.21.15.64:0/688436322 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00f620 con 0x7f07dc103740 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.343+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1a71b0 con 0x7f07dc103740 2024-09-19T05:09:19.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.345+0000 7f07d37fe640 1 -- 172.21.15.64:0/688436322 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f07cc01a070 con 0x7f07dc103740 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.345+0000 7f07e68a2740 1 Processor -- start 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 -- start start 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 0x5622c5e48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e485d0 0x5622c5e48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5622c5dae2f0 con 0x5622c5e53370 2024-09-19T05:09:19.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5622c5d55cf0 con 0x5622c5e49b40 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e68a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622c5daa2b0 con 0x5622c5e485d0 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e2a66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 0x5622c5e48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e2a66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 0x5622c5e48090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42452/0 (socket says 172.21.15.64:42452) 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e2a66640 1 -- 172.21.15.64:0/2391750303 learned_addr learned my addr 172.21.15.64:0/2391750303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e1a64640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.346+0000 7f07e2265640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e485d0 0x5622c5e48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2a66640 1 -- 172.21.15.64:0/2391750303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e485d0 msgr2=0x5622c5e48a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2a66640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e485d0 0x5622c5e48a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2a66640 1 -- 172.21.15.64:0/2391750303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 msgr2=0x5622c5e55760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2a66640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e55760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2a66640 1 -- 172.21.15.64:0/2391750303 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c5d3a3a0 con 0x5622c5e49b40 2024-09-19T05:09:19.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07d37fe640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f07c80a3430 0x7f07c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2265640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e485d0 0x5622c5e48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:19.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e1a64640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:19.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07d37fe640 1 -- 172.21.15.64:0/688436322 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f07cc0cf680 con 0x7f07dc103740 2024-09-19T05:09:19.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2a66640 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 0x5622c5e48090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009cd0 tx=0x7f07d4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=facb98362dc1b6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:19.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07e2265640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f07c80a3430 0x7f07c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07d0ff9640 1 -- 172.21.15.64:0/2391750303 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4049070 con 0x5622c5e49b40 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07d0ff9640 1 -- 172.21.15.64:0/2391750303 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07d4036040 con 0x5622c5e49b40 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.347+0000 7f07d0ff9640 1 -- 172.21.15.64:0/2391750303 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40042a0 con 0x5622c5e49b40 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 -- 172.21.15.64:0/2391750303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 msgr2=0x5622c5e48090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e2265640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f07c80a3430 0x7f07c80a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5622c5e45ac0 tx=0x7f07d800b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 0x5622c5e48090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009cd0 tx=0x7f07d4009a20 comp rx=0 tx=0).stop 2024-09-19T05:09:19.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 -- 172.21.15.64:0/2391750303 shutdown_connections 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e485d0 0x5622c5e48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2391750303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e49b40 0x5622c5e48090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 -- 172.21.15.64:0/2391750303 >> 172.21.15.64:0/2391750303 conn(0x5622c5e11310 msgr2=0x5622c5e44c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 -- 172.21.15.64:0/2391750303 shutdown_connections 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.348+0000 7f07e68a2740 1 -- 172.21.15.64:0/2391750303 wait complete. 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 -- start start 2024-09-19T05:09:19.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 Processor -- start 2024-09-19T05:09:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e52d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 0x5622c5e4ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e53370 0x5622c5e50530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5622c5e56010 con 0x5622c5e49b40 2024-09-19T05:09:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5622c5d69060 con 0x5622c5e485d0 2024-09-19T05:09:19.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e68a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622c5e56190 con 0x5622c5e53370 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e2265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e52d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e1a64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 0x5622c5e4ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e2265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e52d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42458/0 (socket says 172.21.15.64:42458) 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e1a64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 0x5622c5e4ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48800/0 (socket says 172.21.15.64:48800) 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.349+0000 7f07e2265640 1 -- 172.21.15.64:0/1567467291 learned_addr learned my addr 172.21.15.64:0/1567467291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e68a2740 10 client.0 _ll_register_callbacks cb 0x5622c5abc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:19.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e2a66640 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e53370 0x5622c5e50530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e1a64640 1 -- 172.21.15.64:0/1567467291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e53370 msgr2=0x5622c5e50530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e1a64640 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e53370 0x5622c5e50530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e1a64640 1 -- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 msgr2=0x5622c5e52d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e1a64640 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e52d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e1a64640 1 -- 172.21.15.64:0/1567467291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c5b1f3c0 con 0x5622c5e49b40 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e1a64640 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 0x5622c5e4ffb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0ccdd0 tx=0x7f07cc09b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9257ed314bf5caf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:19.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07c5ffb640 1 -- 172.21.15.64:0/1567467291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc021050 con 0x5622c5e49b40 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07c5ffb640 1 -- 172.21.15.64:0/1567467291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f07cc098080 con 0x5622c5e49b40 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d4036040 con 0x5622c5e49b40 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.350+0000 7f07c5ffb640 1 -- 172.21.15.64:0/1567467291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07cc009de0 con 0x5622c5e49b40 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 10 client.16185 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622c5e61a70 con 0x5622c5e49b40 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07c5ffb640 1 -- 172.21.15.64:0/1567467291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07d4036040 con 0x5622c5e49b40 2024-09-19T05:09:19.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07c5ffb640 1 -- 172.21.15.64:0/1567467291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f07cc0d2ce0 con 0x5622c5e49b40 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 10 client.16185 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 20 client.16185 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5e62f30 0x5622c5e65320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 4 client.16185 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622c5dbb860 con 0x5622c5e62f30 2024-09-19T05:09:19.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.351+0000 7f07e2265640 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5e62f30 0x5622c5e65320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.352+0000 7f07c5ffb640 10 client.16185 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.352+0000 7f07e2265640 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5e62f30 0x5622c5e65320 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07eae4317ce0eb8 server_cookie=4ab665536e9457df in_seq=0 out_seq=0 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.352+0000 7f07c5ffb640 1 -- 172.21.15.64:0/1567467291 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622c5dbb860 con 0x5622c5e62f30 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.352+0000 7f07c5ffb640 10 client.16185 handle_command_reply: tid=0 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.352+0000 7f07e68a2740 1 client.16185 shutdown 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 msgr2=0x5622c5e4ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 0x5622c5e4ffb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0ccdd0 tx=0x7f07cc09b7e0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 shutdown_connections 2024-09-19T05:09:19.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e53370 0x5622c5e50530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5e62f30 0x5622c5e65320 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e49b40 0x5622c5e4ffb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 --2- 172.21.15.64:0/1567467291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e52d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 >> 172.21.15.64:0/1567467291 conn(0x5622c5e11310 msgr2=0x5622c5e4a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07c5ffb640 0 client.16185 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 shutdown_connections 2024-09-19T05:09:19.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 1 -- 172.21.15.64:0/1567467291 wait complete. 2024-09-19T05:09:19.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.353+0000 7f07e68a2740 20 client.16185 trim_cache size 0 max 16384 2024-09-19T05:09:19.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.368+0000 7f07e68a2740 1 Processor -- start 2024-09-19T05:09:19.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.368+0000 7f07e68a2740 1 -- start start 2024-09-19T05:09:19.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.368+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e4b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e4b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e68a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1b2240 con 0x5622c5e53370 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e68a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1afb10 con 0x5622c5e485d0 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e4b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e68a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1ad960 con 0x5622c5e49b40 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e4b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48806/0 (socket says 172.21.15.64:48806) 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 -- 172.21.15.64:0/385290658 learned_addr learned my addr 172.21.15.64:0/385290658 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:19.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e1a64640 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e4b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2265640 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 -- 172.21.15.64:0/385290658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 msgr2=0x5622c5e4b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e4b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 -- 172.21.15.64:0/385290658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 msgr2=0x5622c5e51f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e51f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 -- 172.21.15.64:0/385290658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c5e45dd0 con 0x5622c5e53370 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.369+0000 7f07e2a66640 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e4b660 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009bf0 tx=0x7f07d4003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6398fd0413b4a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.370+0000 7f07c7fff640 1 -- 172.21.15.64:0/385290658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4054070 con 0x5622c5e53370 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.370+0000 7f07c7fff640 1 -- 172.21.15.64:0/385290658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07d4036040 con 0x5622c5e53370 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.370+0000 7f07c7fff640 1 -- 172.21.15.64:0/385290658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4049040 con 0x5622c5e53370 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.370+0000 7f07e68a2740 1 -- 172.21.15.64:0/385290658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 msgr2=0x5622c5e4b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.370+0000 7f07e68a2740 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e4b660 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009bf0 tx=0x7f07d4003da0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.370+0000 7f07e68a2740 1 -- 172.21.15.64:0/385290658 shutdown_connections 2024-09-19T05:09:19.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e4b660 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e4b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 --2- 172.21.15.64:0/385290658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e51f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- 172.21.15.64:0/385290658 >> 172.21.15.64:0/385290658 conn(0x5622c5e11310 msgr2=0x5622c5e51810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- 172.21.15.64:0/385290658 shutdown_connections 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- 172.21.15.64:0/385290658 wait complete. 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- start start 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 Processor -- start 2024-09-19T05:09:19.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1adfd0 con 0x5622c5e53370 2024-09-19T05:09:19.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1af440 con 0x5622c5e485d0 2024-09-19T05:09:19.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1ae6a0 con 0x5622c5e49b40 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.371+0000 7f07e68a2740 10 client.0 _ll_register_callbacks cb 0x7f07cc010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e57b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48810/0 (socket says 172.21.15.64:48810) 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 -- 172.21.15.64:0/2096756780 learned_addr learned my addr 172.21.15.64:0/2096756780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e1a64640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2265640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 -- 172.21.15.64:0/2096756780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 msgr2=0x5622c5e57630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e57630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 -- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 msgr2=0x5622c5e570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 -- 172.21.15.64:0/2096756780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c5e444f0 con 0x5622c5e53370 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e2a66640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e57b70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f07d4031b00 tx=0x7f07d4031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e75571699a80ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07c5ffb640 1 -- 172.21.15.64:0/2096756780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4054070 con 0x5622c5e53370 2024-09-19T05:09:19.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07c5ffb640 1 -- 172.21.15.64:0/2096756780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f07d40308c0 con 0x5622c5e53370 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.372+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d4036040 con 0x5622c5e53370 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07c5ffb640 1 -- 172.21.15.64:0/2096756780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07d4030ac0 con 0x5622c5e53370 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 10 client.16191 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622c5ecdc50 con 0x5622c5e53370 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07c5ffb640 1 -- 172.21.15.64:0/2096756780 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07d4036040 con 0x5622c5e53370 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07c5ffb640 1 -- 172.21.15.64:0/2096756780 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f07d4030070 con 0x5622c5e53370 2024-09-19T05:09:19.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 10 client.16191 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 20 client.16191 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5ecde20 0x5622c5ed0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 4 client.16191 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.373+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622c5db5d70 con 0x5622c5ecde20 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.374+0000 7f07e2265640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5ecde20 0x5622c5ed0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.374+0000 7f07c5ffb640 10 client.16191 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:19.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.374+0000 7f07e2265640 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5ecde20 0x5622c5ed0210 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b858a886aa9269 server_cookie=a64035cb6a19670c in_seq=0 out_seq=0 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07c5ffb640 1 -- 172.21.15.64:0/2096756780 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622c5db5d70 con 0x5622c5ecde20 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07c5ffb640 10 client.16191 handle_command_reply: tid=0 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 client.16191 shutdown 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 msgr2=0x5622c5e57b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e57b70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f07d4031b00 tx=0x7f07d4031b30 comp rx=0 tx=0).stop 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 shutdown_connections 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5622c5ecde20 0x5622c5ed0210 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5622c5e53370 0x5622c5e57b70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622c5e49b40 0x5622c5e57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07c5ffb640 0 client.16191 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 --2- 172.21.15.64:0/2096756780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5622c5e485d0 0x5622c5e570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 >> 172.21.15.64:0/2096756780 conn(0x5622c5e11310 msgr2=0x5622c5e52030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 shutdown_connections 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 1 -- 172.21.15.64:0/2096756780 wait complete. 2024-09-19T05:09:19.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.375+0000 7f07e68a2740 20 client.16191 trim_cache size 0 max 16384 2024-09-19T05:09:19.408 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f07c80a3430 msgr2=0x7f07c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f07c80a3430 0x7f07c80a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5622c5e45ac0 tx=0x7f07d800b390 comp rx=0 tx=0).stop 2024-09-19T05:09:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 msgr2=0x7f07dc19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 0x7f07dc19ab70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00d770 tx=0x7f07cc00dc30 comp rx=0 tx=0).stop 2024-09-19T05:09:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 shutdown_connections 2024-09-19T05:09:19.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f07c80a3430 0x7f07c80a58f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108290 0x7f07dc19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07dc103740 0x7f07dc19ab70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 --2- 172.21.15.64:0/688436322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102d90 0x7f07dc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 >> 172.21.15.64:0/688436322 conn(0x7f07dc0fc7d0 msgr2=0x7f07dc100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 shutdown_connections 2024-09-19T05:09:19.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:19.376+0000 7f07e44c9640 1 -- 172.21.15.64:0/688436322 wait complete. 2024-09-19T05:09:19.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:19 smithi064 ceph-mon[30960]: pgmap v105: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-19T05:09:19.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:19 smithi103 ceph-mon[34446]: pgmap v105: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-19T05:09:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:19 smithi174 ceph-mon[34488]: pgmap v105: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-19T05:09:21.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:09:21.087 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:09:21.369 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:21 smithi064 ceph-mon[30960]: pgmap v106: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-19T05:09:21.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:21 smithi103 ceph-mon[34446]: pgmap v106: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-19T05:09:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:21 smithi174 ceph-mon[34488]: pgmap v106: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-19T05:09:21.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 -- 172.21.15.64:0/1375884221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 msgr2=0x7f7730107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 --2- 172.21.15.64:0/1375884221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730107b80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7720009930 tx=0x7f772002a420 comp rx=0 tx=0).stop 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 -- 172.21.15.64:0/1375884221 shutdown_connections 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 --2- 172.21.15.64:0/1375884221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7730108290 0x7f773010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 --2- 172.21.15.64:0/1375884221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730107b80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 --2- 172.21.15.64:0/1375884221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7730102d90 0x7f7730103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 -- 172.21.15.64:0/1375884221 >> 172.21.15.64:0/1375884221 conn(0x7f77300fc7d0 msgr2=0x7f77300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 -- 172.21.15.64:0/1375884221 shutdown_connections 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.886+0000 7f77387c3640 1 -- 172.21.15.64:0/1375884221 wait complete. 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 Processor -- start 2024-09-19T05:09:21.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 -- start start 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7730102d90 0x7f77301009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7730108290 0x7f7730101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7730111f00 con 0x7f7730103740 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7730111d80 con 0x7f7730108290 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f77387c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7730112080 con 0x7f7730102d90 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f7735d5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f7735d5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48834/0 (socket says 172.21.15.64:48834) 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.887+0000 7f7735d5e640 1 -- 172.21.15.64:0/1145237550 learned_addr learned my addr 172.21.15.64:0/1145237550 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f773655f640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7730102d90 0x7f77301009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7736d60640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7730108290 0x7f7730101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7735d5e640 1 -- 172.21.15.64:0/1145237550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7730102d90 msgr2=0x7f77301009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7735d5e640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7730102d90 0x7f77301009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7735d5e640 1 -- 172.21.15.64:0/1145237550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7730108290 msgr2=0x7f7730101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7735d5e640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7730108290 0x7f7730101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7735d5e640 1 -- 172.21.15.64:0/1145237550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77301a69f0 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f7735d5e640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730100f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7720031440 tx=0x7f7720004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f77277fe640 1 -- 172.21.15.64:0/1145237550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772003f030 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f77277fe640 1 -- 172.21.15.64:0/1145237550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7720003aa0 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77301a6c80 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.888+0000 7f77277fe640 1 -- 172.21.15.64:0/1145237550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720035070 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.889+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77301a7110 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.891+0000 7f77277fe640 1 -- 172.21.15.64:0/1145237550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7720042030 con 0x7f7730103740 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.891+0000 7f773ab9c740 1 Processor -- start 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.891+0000 7f773ab9c740 1 -- start start 2024-09-19T05:09:21.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bdb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bde130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773ab9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5578c7b36280 con 0x5578c7bd0b60 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773ab9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5578c7addb10 con 0x5578c7bd1480 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773ab9c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578c7b32240 con 0x5578c7bdbd40 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48836/0 (socket says 172.21.15.64:48836) 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 -- 172.21.15.64:0/83499034 learned_addr learned my addr 172.21.15.64:0/83499034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773655f640 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bdb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7735d5e640 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bde130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 -- 172.21.15.64:0/83499034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 msgr2=0x5578c7bde130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bde130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 -- 172.21.15.64:0/83499034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 msgr2=0x5578c7bdb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bdb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f7736d60640 1 -- 172.21.15.64:0/83499034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c7aea1f0 con 0x5578c7bd0b60 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f773655f640 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bdb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.892+0000 7f77277fe640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f771c0a33b0 0x7f771c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.893+0000 7f7736d60640 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd0f40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7728009e60 tx=0x7f7728010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51290be9f6f7897d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.893+0000 7f77277fe640 1 -- 172.21.15.64:0/1145237550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f77200f0330 con 0x7f7730103740 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.893+0000 7f773655f640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f771c0a33b0 0x7f771c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.893+0000 7f7724ff9640 1 -- 172.21.15.64:0/83499034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728016070 con 0x5578c7bd0b60 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.893+0000 7f7724ff9640 1 -- 172.21.15.64:0/83499034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7728004d20 con 0x5578c7bd0b60 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.893+0000 7f7724ff9640 1 -- 172.21.15.64:0/83499034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728012740 con 0x5578c7bd0b60 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 -- 172.21.15.64:0/83499034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 msgr2=0x5578c7bd0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd0f40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7728009e60 tx=0x7f7728010740 comp rx=0 tx=0).stop 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773655f640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f771c0a33b0 0x7f771c0a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5578c7bcdf50 tx=0x7f772c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 -- 172.21.15.64:0/83499034 shutdown_connections 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bde130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bdb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 --2- 172.21.15.64:0/83499034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd0f40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 -- 172.21.15.64:0/83499034 >> 172.21.15.64:0/83499034 conn(0x5578c7bcd0f0 msgr2=0x5578c7bcd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 -- 172.21.15.64:0/83499034 shutdown_connections 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 -- 172.21.15.64:0/83499034 wait complete. 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 -- start start 2024-09-19T05:09:21.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.894+0000 7f773ab9c740 1 Processor -- start 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd1480 0x5578c7bd9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773655f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773655f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48848/0 (socket says 172.21.15.64:48848) 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773655f640 1 -- 172.21.15.64:0/2050658124 learned_addr learned my addr 172.21.15.64:0/2050658124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7bd96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5578c7bde910 con 0x5578c7bd0b60 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5578c7af0f10 con 0x5578c7bdbd40 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578c7bdea90 con 0x5578c7bd1480 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.895+0000 7f7735d5e640 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd1480 0x5578c7bd9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773ab9c740 10 client.0 _ll_register_callbacks cb 0x5578c7ab9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:21.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773655f640 1 -- 172.21.15.64:0/2050658124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd1480 msgr2=0x5578c7bd9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773655f640 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd1480 0x5578c7bd9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773655f640 1 -- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 msgr2=0x5578c7bd96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773655f640 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7bd96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773655f640 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578c78a76a0 con 0x5578c7bd0b60 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773655f640 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd8c30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f772c011c30 tx=0x7f772c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d88b4dcdee645 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f7719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f7719ffb640 1 -- 172.21.15.64:0/2050658124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0108d0 con 0x5578c7bd0b60 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7728004d20 con 0x5578c7bd0b60 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f7719ffb640 1 -- 172.21.15.64:0/2050658124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f772c010a70 con 0x5578c7bd0b60 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f7719ffb640 1 -- 172.21.15.64:0/2050658124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f772c010c70 con 0x5578c7bd0b60 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773ab9c740 10 client.16203 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.896+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578c7bf2d30 con 0x5578c7bd0b60 2024-09-19T05:09:21.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f7719ffb640 1 -- 172.21.15.64:0/2050658124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7728004d20 con 0x5578c7bd0b60 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f7719ffb640 1 -- 172.21.15.64:0/2050658124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f772c01f410 con 0x5578c7bd0b60 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f773ab9c740 10 client.16203 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f773ab9c740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f773ab9c740 20 client.16203 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7bf4190 0x5578c7bf6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f773ab9c740 4 client.16203 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578c7b437f0 con 0x5578c7bf4190 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.897+0000 7f7735d5e640 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7bf4190 0x5578c7bf6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.898+0000 7f7719ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.898+0000 7f7735d5e640 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7bf4190 0x5578c7bf6620 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51a30d84093af0a server_cookie=55883f8a71e79bde in_seq=0 out_seq=0 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.898+0000 7f7719ffb640 1 -- 172.21.15.64:0/2050658124 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578c7b437f0 con 0x5578c7bf4190 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.898+0000 7f7719ffb640 10 client.16203 handle_command_reply: tid=0 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.898+0000 7f773ab9c740 1 client.16203 shutdown 2024-09-19T05:09:21.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 msgr2=0x5578c7bd8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd8c30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f772c011c30 tx=0x7f772c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 shutdown_connections 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7bd96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd1480 0x5578c7bd9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7bf4190 0x5578c7bf6620 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 --2- 172.21.15.64:0/2050658124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd8c30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 >> 172.21.15.64:0/2050658124 conn(0x5578c7bcd0f0 msgr2=0x5578c7bcc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f7719ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 shutdown_connections 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 1 -- 172.21.15.64:0/2050658124 wait complete. 2024-09-19T05:09:21.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.899+0000 7f773ab9c740 20 client.16203 trim_cache size 0 max 16384 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 Processor -- start 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 -- start start 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bfbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bfc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77301adf50 con 0x5578c7bd0b60 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77301ab820 con 0x5578c7bd1480 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773ab9c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77301a9690 con 0x5578c7bdbd40 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773655f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773655f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48858/0 (socket says 172.21.15.64:48858) 2024-09-19T05:09:21.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f773655f640 1 -- 172.21.15.64:0/494063905 learned_addr learned my addr 172.21.15.64:0/494063905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f7735d5e640 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bfbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.917+0000 7f7736d60640 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bfc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f773655f640 1 -- 172.21.15.64:0/494063905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 msgr2=0x5578c7bfc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f773655f640 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bfc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f773655f640 1 -- 172.21.15.64:0/494063905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 msgr2=0x5578c7bfbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f773655f640 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bfbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f773655f640 1 -- 172.21.15.64:0/494063905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c7bd7be0 con 0x5578c7bd0b60 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f773655f640 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd2d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f772c004690 tx=0x7f772c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad812c6f412f85d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f771bfff640 1 -- 172.21.15.64:0/494063905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c011d20 con 0x5578c7bd0b60 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f771bfff640 1 -- 172.21.15.64:0/494063905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f772c01faf0 con 0x5578c7bd0b60 2024-09-19T05:09:21.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.918+0000 7f771bfff640 1 -- 172.21.15.64:0/494063905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0052a0 con 0x5578c7bd0b60 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 -- 172.21.15.64:0/494063905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 msgr2=0x5578c7bd2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd2d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f772c004690 tx=0x7f772c005c50 comp rx=0 tx=0).stop 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 -- 172.21.15.64:0/494063905 shutdown_connections 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bdbd40 0x5578c7bfc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bd1480 0x5578c7bfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 --2- 172.21.15.64:0/494063905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd0b60 0x5578c7bd2d10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 -- 172.21.15.64:0/494063905 >> 172.21.15.64:0/494063905 conn(0x5578c7bcd0f0 msgr2=0x5578c7bd2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 -- 172.21.15.64:0/494063905 shutdown_connections 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 -- 172.21.15.64:0/494063905 wait complete. 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.919+0000 7f773ab9c740 1 -- start start 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 Processor -- start 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd0b60 0x5578c7bdfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 0x5578c7be03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7c5dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77301a9ce0 con 0x5578c7bd1480 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77301ab150 con 0x5578c7bdbd40 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77301aa3b0 con 0x5578c7bd0b60 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773ab9c740 10 client.0 _ll_register_callbacks cb 0x7f772c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 0x5578c7be03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 0x5578c7be03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48868/0 (socket says 172.21.15.64:48868) 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 -- 172.21.15.64:0/3664431628 learned_addr learned my addr 172.21.15.64:0/3664431628 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f773655f640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd0b60 0x5578c7bdfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7736d60640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7c5dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 -- 172.21.15.64:0/3664431628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd0b60 msgr2=0x5578c7bdfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd0b60 0x5578c7bdfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 -- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 msgr2=0x5578c7c5dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7c5dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.920+0000 7f7735d5e640 1 -- 172.21.15.64:0/3664431628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578c7c5e400 con 0x5578c7bd1480 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7735d5e640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 0x5578c7be03a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f77200b8a70 tx=0x7f772002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55a194ca700d089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7719ffb640 1 -- 172.21.15.64:0/3664431628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720049040 con 0x5578c7bd1480 2024-09-19T05:09:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7719ffb640 1 -- 172.21.15.64:0/3664431628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f77200ba6b0 con 0x5578c7bd1480 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f772c01faf0 con 0x5578c7bd1480 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7719ffb640 1 -- 172.21.15.64:0/3664431628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77200bc610 con 0x5578c7bd1480 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f773ab9c740 10 client.16209 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578c7c5f620 con 0x5578c7bd1480 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7719ffb640 1 -- 172.21.15.64:0/3664431628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f772c01faf0 con 0x5578c7bd1480 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f7719ffb640 1 -- 172.21.15.64:0/3664431628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f77200edd00 con 0x5578c7bd1480 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f773ab9c740 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.921+0000 7f773ab9c740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f773ab9c740 20 client.16209 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f773ab9c740 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7c5fda0 0x5578c7c621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f773ab9c740 4 client.16209 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5578c7b3dd00 con 0x5578c7c5fda0 2024-09-19T05:09:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f773655f640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7c5fda0 0x5578c7c621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f7719ffb640 10 client.16209 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.922+0000 7f773655f640 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7c5fda0 0x5578c7c621c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aec41458db2c20a server_cookie=6940b00a06c9b519 in_seq=0 out_seq=0 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.923+0000 7f7719ffb640 1 -- 172.21.15.64:0/3664431628 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5578c7b3dd00 con 0x5578c7c5fda0 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f7719ffb640 10 client.16209 handle_command_reply: tid=0 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 client.16209 shutdown 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 msgr2=0x5578c7be03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 0x5578c7be03a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f77200b8a70 tx=0x7f772002a420 comp rx=0 tx=0).stop 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 shutdown_connections 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578c7bdbd40 0x5578c7c5dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5578c7c5fda0 0x5578c7c621c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5578c7bd1480 0x5578c7be03a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 --2- 172.21.15.64:0/3664431628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578c7bd0b60 0x5578c7bdfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f7719ffb640 0 client.16209 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 >> 172.21.15.64:0/3664431628 conn(0x5578c7bcd0f0 msgr2=0x5578c7bd93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 shutdown_connections 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 1 -- 172.21.15.64:0/3664431628 wait complete. 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.924+0000 7f773ab9c740 20 client.16209 trim_cache size 0 max 16384 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "b00afc76-0144-42b7-b4a7-75e7302aa79d", 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:09:21.930 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f771c0a33b0 msgr2=0x7f771c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f771c0a33b0 0x7f771c0a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5578c7bcdf50 tx=0x7f772c005f90 comp rx=0 tx=0).stop 2024-09-19T05:09:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 msgr2=0x7f7730100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730100f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7720031440 tx=0x7f7720004270 comp rx=0 tx=0).stop 2024-09-19T05:09:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 shutdown_connections 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f771c0a33b0 0x7f771c0a5870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7730108290 0x7f7730101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7730103740 0x7f7730100f30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 --2- 172.21.15.64:0/1145237550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7730102d90 0x7f77301009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 >> 172.21.15.64:0/1145237550 conn(0x7f77300fc7d0 msgr2=0x7f7730105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 shutdown_connections 2024-09-19T05:09:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:21.925+0000 7f77387c3640 1 -- 172.21.15.64:0/1145237550 wait complete. 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 -- 172.21.15.64:0/2065490081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74101ad0 msgr2=0x7f6b74101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/2065490081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74101ad0 0x7f6b74101f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009930 tx=0x7f6b6802a420 comp rx=0 tx=0).stop 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 -- 172.21.15.64:0/2065490081 shutdown_connections 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/2065490081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b74102470 0x7f6b7410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/2065490081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74101ad0 0x7f6b74101f30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/2065490081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b74107b70 0x7f6b74107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 -- 172.21.15.64:0/2065490081 >> 172.21.15.64:0/2065490081 conn(0x7f6b740fd7d0 msgr2=0x7f6b740ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 -- 172.21.15.64:0/2065490081 shutdown_connections 2024-09-19T05:09:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.289+0000 7f6b7a44c640 1 -- 172.21.15.64:0/2065490081 wait complete. 2024-09-19T05:09:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 Processor -- start 2024-09-19T05:09:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 -- start start 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b74101ad0 0x7f6b740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 0x7f6b74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b74107b70 0x7f6b74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74111d90 con 0x7f6b74102470 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74111c10 con 0x7f6b74101ad0 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b7a44c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74111f10 con 0x7f6b74107b70 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 0x7f6b74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 0x7f6b74074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48886/0 (socket says 172.21.15.64:48886) 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b737fe640 1 -- 172.21.15.64:0/3507784905 learned_addr learned my addr 172.21.15.64:0/3507784905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b74101ad0 0x7f6b740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.290+0000 7f6b789e9640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b74107b70 0x7f6b74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b737fe640 1 -- 172.21.15.64:0/3507784905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b74107b70 msgr2=0x7f6b74074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b74107b70 0x7f6b74074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b737fe640 1 -- 172.21.15.64:0/3507784905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b74101ad0 msgr2=0x7f6b740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b74101ad0 0x7f6b740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b737fe640 1 -- 172.21.15.64:0/3507784905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b741a67a0 con 0x7f6b74102470 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 0x7f6b74074a30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005940 tx=0x7f6b680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b717fa640 1 -- 172.21.15.64:0/3507784905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6803f030 con 0x7f6b74102470 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b741a6a30 con 0x7f6b74102470 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b717fa640 1 -- 172.21.15.64:0/3507784905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b680318a0 con 0x7f6b74102470 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b717fa640 1 -- 172.21.15.64:0/3507784905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68035070 con 0x7f6b74102470 2024-09-19T05:09:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.291+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b741a6f20 con 0x7f6b74102470 2024-09-19T05:09:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.293+0000 7f6b717fa640 1 -- 172.21.15.64:0/3507784905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6b68042030 con 0x7f6b74102470 2024-09-19T05:09:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.294+0000 7f6b7d1e3740 1 Processor -- start 2024-09-19T05:09:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.294+0000 7f6b7d1e3740 1 -- start start 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.294+0000 7f6b717fa640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b5c0a3460 0x7f6b5c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.294+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef990ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.294+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef990d410 0x564ef9917770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef9917cb0 0x564ef991a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ef98712f0 con 0x564ef9917cb0 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ef9819270 con 0x564ef990d410 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ef986d2b0 con 0x564ef990caf0 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b789e9640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b5c0a3460 0x7f6b5c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b717fa640 1 -- 172.21.15.64:0/3507784905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6b680f0470 con 0x7f6b74102470 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b789e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef9917cb0 0x564ef991a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef990ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef990ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:53468/0 (socket says 172.21.15.64:53468) 2024-09-19T05:09:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 -- 172.21.15.64:0/3167533904 learned_addr learned my addr 172.21.15.64:0/3167533904 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef990d410 0x564ef9917770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 -- 172.21.15.64:0/3167533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef990d410 msgr2=0x564ef9917770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef990d410 0x564ef9917770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 -- 172.21.15.64:0/3167533904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef9917cb0 msgr2=0x564ef991a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef9917cb0 0x564ef991a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 -- 172.21.15.64:0/3167533904 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef9818340 con 0x564ef990caf0 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b789e9640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b5c0a3460 0x7f6b5c0a5920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005fd0 tx=0x7f6b6c009f10 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.295+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef990ced0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400bc50 tx=0x7f6b640107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba111bc88a5533a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b5a7fc640 1 -- 172.21.15.64:0/3167533904 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6401c480 con 0x564ef990caf0 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b5a7fc640 1 -- 172.21.15.64:0/3167533904 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b6401cad0 con 0x564ef990caf0 2024-09-19T05:09:22.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b5a7fc640 1 -- 172.21.15.64:0/3167533904 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64011a50 con 0x564ef990caf0 2024-09-19T05:09:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3167533904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 msgr2=0x564ef990ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef990ced0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400bc50 tx=0x7f6b640107a0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3167533904 shutdown_connections 2024-09-19T05:09:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef9917cb0 0x564ef991a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef990d410 0x564ef9917770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3167533904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef990ced0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3167533904 >> 172.21.15.64:0/3167533904 conn(0x564ef9909060 msgr2=0x564ef9909440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3167533904 shutdown_connections 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.296+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3167533904 wait complete. 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 -- start start 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 Processor -- start 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9914c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9915190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef99156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ef991a880 con 0x564ef990d410 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ef97fa290 con 0x564ef9917cb0 2024-09-19T05:09:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ef991aa00 con 0x564ef990caf0 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.297+0000 7f6b7d1e3740 10 client.0 _ll_register_callbacks cb 0x564ef982b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9915190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9915190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48902/0 (socket says 172.21.15.64:48902) 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 -- 172.21.15.64:0/670861619 learned_addr learned my addr 172.21.15.64:0/670861619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b73fff640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9914c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b789e9640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef99156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 -- 172.21.15.64:0/670861619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 msgr2=0x564ef9914c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9914c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 -- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 msgr2=0x564ef99156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef99156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 -- 172.21.15.64:0/670861619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef9584210 con 0x564ef990d410 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b789e9640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef99156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b737fe640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9915190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009900 tx=0x7f6b680bc1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcb4cd9851ae6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.298+0000 7f6b537fe640 1 -- 172.21.15.64:0/670861619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68049070 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b537fe640 1 -- 172.21.15.64:0/670861619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6b6803bab0 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b6401cad0 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b537fe640 1 -- 172.21.15.64:0/670861619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b6803bcb0 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 10 client.16221 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ef9650620 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b537fe640 1 -- 172.21.15.64:0/670861619 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b6401cad0 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b537fe640 1 -- 172.21.15.64:0/670861619 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b680ede70 con 0x564ef990d410 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 10 client.16221 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 20 client.16221 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9927ca0 0x564ef992a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 4 client.16221 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.299+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef987ea20 con 0x564ef9927ca0 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.300+0000 7f6b73fff640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9927ca0 0x564ef992a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.300+0000 7f6b537fe640 10 client.16221 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.300+0000 7f6b73fff640 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9927ca0 0x564ef992a0b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0b3826a2fda380 server_cookie=2d2a150796ed9c4b in_seq=0 out_seq=0 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b537fe640 1 -- 172.21.15.64:0/670861619 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ef987ea20 con 0x564ef9927ca0 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b537fe640 10 client.16221 handle_command_reply: tid=0 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 client.16221 shutdown 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 msgr2=0x564ef9915190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9915190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009900 tx=0x7f6b680bc1c0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 shutdown_connections 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef99156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9927ca0 0x564ef992a0b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9915190 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b537fe640 0 client.16221 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/670861619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9914c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 >> 172.21.15.64:0/670861619 conn(0x564ef9909060 msgr2=0x564ef990ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 shutdown_connections 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/670861619 wait complete. 2024-09-19T05:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.301+0000 7f6b7d1e3740 20 client.16221 trim_cache size 0 max 16384 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 Processor -- start 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 -- start start 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9915500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9916bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef992f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741b2250 con 0x564ef990d410 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741afb20 con 0x564ef9917cb0 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741ad970 con 0x564ef990caf0 2024-09-19T05:09:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9916bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9916bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48914/0 (socket says 172.21.15.64:48914) 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b737fe640 1 -- 172.21.15.64:0/2564744266 learned_addr learned my addr 172.21.15.64:0/2564744266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.329+0000 7f6b789e9640 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef992f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b73fff640 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9915500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b737fe640 1 -- 172.21.15.64:0/2564744266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 msgr2=0x564ef9915500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b737fe640 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9915500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b737fe640 1 -- 172.21.15.64:0/2564744266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 msgr2=0x564ef992f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b737fe640 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef992f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b737fe640 1 -- 172.21.15.64:0/2564744266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef9915a40 con 0x564ef990d410 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b789e9640 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef992f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b737fe640 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9916bc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6b68004650 tx=0x7f6b680bacb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769004e8f8266836 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b59ffb640 1 -- 172.21.15.64:0/2564744266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68049070 con 0x564ef990d410 2024-09-19T05:09:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b59ffb640 1 -- 172.21.15.64:0/2564744266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b680dde50 con 0x564ef990d410 2024-09-19T05:09:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.330+0000 7f6b59ffb640 1 -- 172.21.15.64:0/2564744266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b8f10 con 0x564ef990d410 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/2564744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 msgr2=0x564ef9916bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9916bc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6b68004650 tx=0x7f6b680bacb0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/2564744266 shutdown_connections 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef992f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef9916bc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/2564744266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef9915500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/2564744266 >> 172.21.15.64:0/2564744266 conn(0x564ef9909060 msgr2=0x564ef99164b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/2564744266 shutdown_connections 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/2564744266 wait complete. 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.331+0000 7f6b7d1e3740 1 -- start start 2024-09-19T05:09:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 Processor -- start 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef991ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef99204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef991b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741adfe0 con 0x564ef990d410 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741af450 con 0x564ef9917cb0 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741ae6b0 con 0x564ef990caf0 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b7d1e3740 10 client.0 _ll_register_callbacks cb 0x7f6b680b8a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef99204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef99204b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48928/0 (socket says 172.21.15.64:48928) 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 -- 172.21.15.64:0/3298859217 learned_addr learned my addr 172.21.15.64:0/3298859217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef991ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b789e9640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef991b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 -- 172.21.15.64:0/3298859217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 msgr2=0x564ef991ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef991ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 -- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 msgr2=0x564ef991b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef991b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.332+0000 7f6b737fe640 1 -- 172.21.15.64:0/3298859217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef991bb20 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b737fe640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef99204b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6b680bc730 tx=0x7f6b6802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2f3d6ea61b65f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b537fe640 1 -- 172.21.15.64:0/3298859217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68049070 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b537fe640 1 -- 172.21.15.64:0/3298859217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f6b680c4ed0 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b680dde50 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b537fe640 1 -- 172.21.15.64:0/3298859217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b680c5360 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 10 client.16227 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ef991bb20 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b537fe640 1 -- 172.21.15.64:0/3298859217 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b680dde50 con 0x564ef990d410 2024-09-19T05:09:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b537fe640 1 -- 172.21.15.64:0/3298859217 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b680b8680 con 0x564ef990d410 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 10 client.16227 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 20 client.16227 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9992770 0x564ef9994b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 4 client.16227 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.333+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ef9878d70 con 0x564ef9992770 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.334+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9992770 0x564ef9994b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.334+0000 7f6b73fff640 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9992770 0x564ef9994b60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc90cb0b19c56908 server_cookie=ea01691add34f4f9 in_seq=0 out_seq=0 2024-09-19T05:09:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.334+0000 7f6b537fe640 10 client.16227 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b537fe640 1 -- 172.21.15.64:0/3298859217 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ef9878d70 con 0x564ef9992770 2024-09-19T05:09:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b537fe640 10 client.16227 handle_command_reply: tid=0 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 client.16227 shutdown 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 msgr2=0x564ef99204b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef99204b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6b680bc730 tx=0x7f6b6802a420 comp rx=0 tx=0).stop 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 shutdown_connections 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ef9917cb0 0x564ef991b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564ef9992770 0x564ef9994b60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef990d410 0x564ef99204b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 --2- 172.21.15.64:0/3298859217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ef990caf0 0x564ef991ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 >> 172.21.15.64:0/3298859217 conn(0x564ef9909060 msgr2=0x564ef99081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b537fe640 0 client.16227 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 shutdown_connections 2024-09-19T05:09:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 1 -- 172.21.15.64:0/3298859217 wait complete. 2024-09-19T05:09:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.338+0000 7f6b7d1e3740 20 client.16227 trim_cache size 0 max 16384 2024-09-19T05:09:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:09:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:09:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:09:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b5c0a3460 msgr2=0x7f6b5c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b5c0a3460 0x7f6b5c0a5920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005fd0 tx=0x7f6b6c009f10 comp rx=0 tx=0).stop 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 msgr2=0x7f6b74074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 0x7f6b74074a30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005940 tx=0x7f6b680042f0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 shutdown_connections 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6b5c0a3460 0x7f6b5c0a5920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b74107b70 0x7f6b74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b74102470 0x7f6b74074a30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 --2- 172.21.15.64:0/3507784905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b74101ad0 0x7f6b740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 >> 172.21.15.64:0/3507784905 conn(0x7f6b740fd7d0 msgr2=0x7f6b740fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 shutdown_connections 2024-09-19T05:09:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:22.339+0000 7f6b7a44c640 1 -- 172.21.15.64:0/3507784905 wait complete. 2024-09-19T05:09:22.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:23.129 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:23 smithi064 ceph-mon[30960]: pgmap v107: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:23 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:23 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:23 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:23 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 -- 172.21.15.64:0/1848998125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 msgr2=0x7f0bf8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 --2- 172.21.15.64:0/1848998125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf8107b90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009930 tx=0x7f0be402a420 comp rx=0 tx=0).stop 2024-09-19T05:09:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 -- 172.21.15.64:0/1848998125 shutdown_connections 2024-09-19T05:09:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 --2- 172.21.15.64:0/1848998125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81082a0 0x7f0bf810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 --2- 172.21.15.64:0/1848998125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf8107b90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 --2- 172.21.15.64:0/1848998125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8102d30 0x7f0bf8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 -- 172.21.15.64:0/1848998125 >> 172.21.15.64:0/1848998125 conn(0x7f0bf80fc7d0 msgr2=0x7f0bf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:23.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 -- 172.21.15.64:0/1848998125 shutdown_connections 2024-09-19T05:09:23.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.636+0000 7f0bfedba640 1 -- 172.21.15.64:0/1848998125 wait complete. 2024-09-19T05:09:23.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 Processor -- start 2024-09-19T05:09:23.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 -- start start 2024-09-19T05:09:23.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8102d30 0x7f0bf819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81082a0 0x7f0bf819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf8111f50 con 0x7f0bf81036e0 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf8111dd0 con 0x7f0bf81082a0 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfedba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81120d0 con 0x7f0bf8102d30 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48948/0 (socket says 172.21.15.64:48948) 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bf7fff640 1 -- 172.21.15.64:0/859343097 learned_addr learned my addr 172.21.15.64:0/859343097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfd357640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81082a0 0x7f0bf819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.637+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8102d30 0x7f0bf819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf7fff640 1 -- 172.21.15.64:0/859343097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8102d30 msgr2=0x7f0bf819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8102d30 0x7f0bf819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf7fff640 1 -- 172.21.15.64:0/859343097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81082a0 msgr2=0x7f0bf819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81082a0 0x7f0bf819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf7fff640 1 -- 172.21.15.64:0/859343097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf819f680 con 0x7f0bf81036e0 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf819ac10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0be4031440 tx=0x7f0be4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf5ffb640 1 -- 172.21.15.64:0/859343097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be403f030 con 0x7f0bf81036e0 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf5ffb640 1 -- 172.21.15.64:0/859343097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be4003aa0 con 0x7f0bf81036e0 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf819f910 con 0x7f0bf81036e0 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bf5ffb640 1 -- 172.21.15.64:0/859343097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4035070 con 0x7f0bf81036e0 2024-09-19T05:09:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.638+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf81a7250 con 0x7f0bf81036e0 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 Processor -- start 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 -- start start 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c62812880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564c627782e0 con 0x564c6281d680 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c6271fce0 con 0x564c628124a0 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0c01193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c627742a0 con 0x564c62812dc0 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.641+0000 7f0bf5ffb640 1 -- 172.21.15.64:0/859343097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0be4042030 con 0x7f0bf81036e0 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48962/0 (socket says 172.21.15.64:48962) 2024-09-19T05:09:23.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 -- 172.21.15.64:0/1245286731 learned_addr learned my addr 172.21.15.64:0/1245286731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bfd357640 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c62812880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 -- 172.21.15.64:0/1245286731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 msgr2=0x564c6281d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 -- 172.21.15.64:0/1245286731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 msgr2=0x564c62812880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c62812880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 -- 172.21.15.64:0/1245286731 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c62704440 con 0x564c6281d680 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:23.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.642+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281fa70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ed260 tx=0x7f0be40f3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e0a5013363a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:23.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0be2ffd640 1 -- 172.21.15.64:0/1245286731 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40f6c00 con 0x564c6281d680 2024-09-19T05:09:23.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0be2ffd640 1 -- 172.21.15.64:0/1245286731 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be40f6da0 con 0x564c6281d680 2024-09-19T05:09:23.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0be2ffd640 1 -- 172.21.15.64:0/1245286731 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be41005f0 con 0x564c6281d680 2024-09-19T05:09:23.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0bf5ffb640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0be80a3430 0x7f0be80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0bf5ffb640 1 -- 172.21.15.64:0/859343097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0be40efc30 con 0x7f0bf81036e0 2024-09-19T05:09:23.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0bfd357640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0be80a3430 0x7f0be80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0c01193740 1 -- 172.21.15.64:0/1245286731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 msgr2=0x564c6281fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.643+0000 7f0c01193740 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281fa70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ed260 tx=0x7f0be40f3f80 comp rx=0 tx=0).stop 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 -- 172.21.15.64:0/1245286731 shutdown_connections 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281fa70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 --2- 172.21.15.64:0/1245286731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c62812880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 -- 172.21.15.64:0/1245286731 >> 172.21.15.64:0/1245286731 conn(0x564c6280ea30 msgr2=0x564c6280ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0bfd357640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0be80a3430 0x7f0be80a58f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0005fd0 tx=0x7f0bf0009600 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 -- 172.21.15.64:0/1245286731 shutdown_connections 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 -- 172.21.15.64:0/1245286731 wait complete. 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 -- start start 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.644+0000 7f0c01193740 1 Processor -- start 2024-09-19T05:09:23.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564c62820010 con 0x564c6281d680 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c62733050 con 0x564c628124a0 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c62820190 con 0x564c62812dc0 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0bfcb56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0c01193740 10 client.0 _ll_register_callbacks cb 0x564c62773160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0bfd357640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0bfcb56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42628/0 (socket says 172.21.15.64:42628) 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.645+0000 7f0bfcb56640 1 -- 172.21.15.64:0/2723814240 learned_addr learned my addr 172.21.15.64:0/2723814240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bf7fff640 1 -- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 msgr2=0x564c6281cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bf7fff640 1 -- 172.21.15.64:0/2723814240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 msgr2=0x564c6281a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bf7fff640 1 -- 172.21.15.64:0/2723814240 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c624e93c0 con 0x564c62812dc0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bfd357640 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281a190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be40f4ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfeaf14d7367df13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bd7fff640 1 -- 172.21.15.64:0/2723814240 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4049030 con 0x564c62812dc0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bd7fff640 1 -- 172.21.15.64:0/2723814240 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0be40ed290 con 0x564c62812dc0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be40f6da0 con 0x564c62812dc0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0bd7fff640 1 -- 172.21.15.64:0/2723814240 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0be40b8150 con 0x564c62812dc0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0c01193740 10 client.25633 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.646+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c62820010 con 0x564c62812dc0 2024-09-19T05:09:23.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0bd7fff640 1 -- 172.21.15.64:0/2723814240 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be40f6da0 con 0x564c62812dc0 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0bd7fff640 1 -- 172.21.15.64:0/2723814240 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0be40b8520 con 0x564c62812dc0 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0c01193740 10 client.25633 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0c01193740 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0c01193740 20 client.25633 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0c01193740 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c6282cce0 0x564c6282f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0c01193740 4 client.25633 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c62785850 con 0x564c6282cce0 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.647+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c6282cce0 0x564c6282f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.648+0000 7f0bd7fff640 10 client.25633 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:23.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.648+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c6282cce0 0x564c6282f120 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63a8fae8477923c server_cookie=94e04b644f6c9cce in_seq=0 out_seq=0 2024-09-19T05:09:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.648+0000 7f0bd7fff640 1 -- 172.21.15.64:0/2723814240 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c62785850 con 0x564c6282cce0 2024-09-19T05:09:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.648+0000 7f0bd7fff640 10 client.25633 handle_command_reply: tid=0 2024-09-19T05:09:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.648+0000 7f0c01193740 1 client.25633 shutdown 2024-09-19T05:09:23.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 msgr2=0x564c6281a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281a190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be40f4ac0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 shutdown_connections 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c6282cce0 0x564c6282f120 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c6281d680 0x564c6281a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c62812dc0 0x564c6281a190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 --2- 172.21.15.64:0/2723814240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 >> 172.21.15.64:0/2723814240 conn(0x564c6280ea30 msgr2=0x564c62814710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0bd7fff640 0 client.25633 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 shutdown_connections 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 1 -- 172.21.15.64:0/2723814240 wait complete. 2024-09-19T05:09:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.649+0000 7f0c01193740 20 client.25633 trim_cache size 0 max 16384 2024-09-19T05:09:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 Processor -- start 2024-09-19T05:09:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 -- start start 2024-09-19T05:09:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c628349f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c6281d680 0x564c62834f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81b22f0 con 0x564c62812dc0 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81afbc0 con 0x564c628124a0 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0c01193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81ada10 con 0x564c6281d680 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c628349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c628349f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48984/0 (socket says 172.21.15.64:48984) 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 -- 172.21.15.64:0/172957763 learned_addr learned my addr 172.21.15.64:0/172957763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 -- 172.21.15.64:0/172957763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c6281d680 msgr2=0x564c62834f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c6281d680 0x564c62834f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 -- 172.21.15.64:0/172957763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 msgr2=0x564c6281c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.677+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.678+0000 7f0bf7fff640 1 -- 172.21.15.64:0/172957763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c6280e860 con 0x564c62812dc0 2024-09-19T05:09:23.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.678+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:23.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.678+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c628349f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009900 tx=0x7f0be40ba3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db665c474cd2b4b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:23.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.678+0000 7f0be27fc640 1 -- 172.21.15.64:0/172957763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4049070 con 0x564c62812dc0 2024-09-19T05:09:23.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.678+0000 7f0be27fc640 1 -- 172.21.15.64:0/172957763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be40f68d0 con 0x564c62812dc0 2024-09-19T05:09:23.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.678+0000 7f0be27fc640 1 -- 172.21.15.64:0/172957763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40f6c00 con 0x564c62812dc0 2024-09-19T05:09:23.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 -- 172.21.15.64:0/172957763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 msgr2=0x564c628349f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c628349f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009900 tx=0x7f0be40ba3c0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 -- 172.21.15.64:0/172957763 shutdown_connections 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c6281d680 0x564c62834f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c628349f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 --2- 172.21.15.64:0/172957763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c628124a0 0x564c6281c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 -- 172.21.15.64:0/172957763 >> 172.21.15.64:0/172957763 conn(0x564c6280ea30 msgr2=0x564c6281b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 -- 172.21.15.64:0/172957763 shutdown_connections 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 -- 172.21.15.64:0/172957763 wait complete. 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 -- start start 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.679+0000 7f0c01193740 1 Processor -- start 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c628124a0 0x564c62825510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c62825a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c6281d680 0x564c62816980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81ae080 con 0x564c62812dc0 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81af4f0 con 0x564c6281d680 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81ae750 con 0x564c628124a0 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0c01193740 10 client.0 _ll_register_callbacks cb 0x7f0bf0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c62825a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c62825a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48992/0 (socket says 172.21.15.64:48992) 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 -- 172.21.15.64:0/890943741 learned_addr learned my addr 172.21.15.64:0/890943741 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bfd357640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c6281d680 0x564c62816980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c628124a0 0x564c62825510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 -- 172.21.15.64:0/890943741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c628124a0 msgr2=0x564c62825510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c628124a0 0x564c62825510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 -- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c6281d680 msgr2=0x564c62816980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:23.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c6281d680 0x564c62816980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bf7fff640 1 -- 172.21.15.64:0/890943741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c62817020 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.680+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c628124a0 0x564c62825510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bf7fff640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c62825a50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ba160 tx=0x7f0be4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6664c1e728b266 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bd7fff640 1 -- 172.21.15.64:0/890943741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4049070 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bd7fff640 1 -- 172.21.15.64:0/890943741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0be40b8520 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be40f68d0 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bd7fff640 1 -- 172.21.15.64:0/890943741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0be40bb810 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0c01193740 10 client.16245 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c62817020 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bd7fff640 1 -- 172.21.15.64:0/890943741 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be40c0260 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0bd7fff640 1 -- 172.21.15.64:0/890943741 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0be40f3970 con 0x564c62812dc0 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.681+0000 7f0c01193740 10 client.16245 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:23.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0c01193740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0c01193740 20 client.16245 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0c01193740 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c62897610 0x564c62899a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0c01193740 4 client.16245 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0be40f68d0 con 0x564c62897610 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c62897610 0x564c62899a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0bd7fff640 10 client.16245 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.682+0000 7f0bfcb56640 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c62897610 0x564c62899a00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc0b04e29ecea67 server_cookie=7c9fa568b7cc536e in_seq=0 out_seq=0 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0bd7fff640 1 -- 172.21.15.64:0/890943741 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f0be40f68d0 con 0x564c62897610 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0bd7fff640 10 client.16245 handle_command_reply: tid=0 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 client.16245 shutdown 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 msgr2=0x564c62825a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c62825a50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ba160 tx=0x7f0be4005be0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 shutdown_connections 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c6281d680 0x564c62816980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564c62897610 0x564c62899a00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c62812dc0 0x564c62825a50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 --2- 172.21.15.64:0/890943741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c628124a0 0x564c62825510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 >> 172.21.15.64:0/890943741 conn(0x564c6280ea30 msgr2=0x564c6280db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0bd7fff640 0 client.16245 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 shutdown_connections 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 1 -- 172.21.15.64:0/890943741 wait complete. 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.683+0000 7f0c01193740 20 client.16245 trim_cache size 0 max 16384 2024-09-19T05:09:23.689 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:23.690 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:09:23.690 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:09:23.690 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:23.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0be80a3430 msgr2=0x7f0be80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0be80a3430 0x7f0be80a58f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0005fd0 tx=0x7f0bf0009600 comp rx=0 tx=0).stop 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 msgr2=0x7f0bf819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf819ac10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0be4031440 tx=0x7f0be4004270 comp rx=0 tx=0).stop 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 shutdown_connections 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0be80a3430 0x7f0be80a58f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81082a0 0x7f0bf819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf81036e0 0x7f0bf819ac10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 --2- 172.21.15.64:0/859343097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8102d30 0x7f0bf819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 >> 172.21.15.64:0/859343097 conn(0x7f0bf80fc7d0 msgr2=0x7f0bf8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 shutdown_connections 2024-09-19T05:09:23.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:23.684+0000 7f0bfedba640 1 -- 172.21.15.64:0/859343097 wait complete. 2024-09-19T05:09:23.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:23 smithi103 ceph-mon[34446]: pgmap v107: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T05:09:23.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:23 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:09:23.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:23 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:23.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:23 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:09:23.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:23 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:23.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:23 smithi174 ceph-mon[34488]: pgmap v107: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T05:09:23.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:23 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:09:23.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:23 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:23.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:23 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:09:23.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:23 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:24.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:09:24.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:09:24.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:24 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:09:24.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:24 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:09:24.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:24 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:09:24.791 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 -- 172.21.15.64:0/2958429277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28081027a0 msgr2=0x7f28081067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 --2- 172.21.15.64:0/2958429277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28081027a0 0x7f28081067f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009900 tx=0x7f27f802a410 comp rx=0 tx=0).stop 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 -- 172.21.15.64:0/2958429277 shutdown_connections 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 --2- 172.21.15.64:0/2958429277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2808106f20 0x7f280810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 --2- 172.21.15.64:0/2958429277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28081027a0 0x7f28081067f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 --2- 172.21.15.64:0/2958429277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2808101df0 0x7f28081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 -- 172.21.15.64:0/2958429277 >> 172.21.15.64:0/2958429277 conn(0x7f28080fd8b0 msgr2=0x7f28080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 -- 172.21.15.64:0/2958429277 shutdown_connections 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.314+0000 7f2810469640 1 -- 172.21.15.64:0/2958429277 wait complete. 2024-09-19T05:09:25.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 Processor -- start 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 -- start start 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 0x7f2808198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28081027a0 0x7f28081989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2808106f20 0x7f280819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2808111e50 con 0x7f2808101df0 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2808111cd0 con 0x7f28081027a0 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f2810469640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2808111fd0 con 0x7f2808106f20 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f280e205640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 0x7f2808198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f280e205640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 0x7f2808198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34358/0 (socket says 172.21.15.64:34358) 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.315+0000 7f280e205640 1 -- 172.21.15.64:0/3280482202 learned_addr learned my addr 172.21.15.64:0/3280482202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280da04640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28081027a0 0x7f28081989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280e205640 1 -- 172.21.15.64:0/3280482202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2808106f20 msgr2=0x7f280819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:25.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280ea06640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2808106f20 0x7f280819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280e205640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2808106f20 0x7f280819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280e205640 1 -- 172.21.15.64:0/3280482202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28081027a0 msgr2=0x7f28081989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280e205640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28081027a0 0x7f28081989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280e205640 1 -- 172.21.15.64:0/3280482202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280819d420 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280da04640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28081027a0 0x7f28081989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f280e205640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 0x7f2808198470 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f280400e9f0 tx=0x7f280400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400cd80 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28040108e0 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280819d710 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804018920 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.316+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081a4ff0 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.317+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28081021d0 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.319+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2804010a90 con 0x7f2808101df0 2024-09-19T05:09:25.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.320+0000 7f27ff7fe640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27f40a33b0 0x7f27f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:25.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.320+0000 7f280da04640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27f40a33b0 0x7f27f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:25.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.320+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f28040d0630 con 0x7f2808101df0 2024-09-19T05:09:25.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.320+0000 7f280da04640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27f40a33b0 0x7f27f40a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f8031400 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:25.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.323+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f280409d010 con 0x7f2808101df0 2024-09-19T05:09:25.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.543+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28081997d0 con 0x7f2808101df0 2024-09-19T05:09:25.545 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:25 smithi064 ceph-mon[30960]: pgmap v108: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:09:25.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.544+0000 7f27ff7fe640 1 -- 172.21.15.64:0/3280482202 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f280409c910 con 0x7f2808101df0 2024-09-19T05:09:25.546 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:09:25.546 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:09:25.546 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:09:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27f40a33b0 msgr2=0x7f27f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:25.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27f40a33b0 0x7f27f40a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f8031400 comp rx=0 tx=0).stop 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 msgr2=0x7f2808198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 0x7f2808198470 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f280400e9f0 tx=0x7f280400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 shutdown_connections 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27f40a33b0 0x7f27f40a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2808106f20 0x7f280819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28081027a0 0x7f28081989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 --2- 172.21.15.64:0/3280482202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2808101df0 0x7f2808198470 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 >> 172.21.15.64:0/3280482202 conn(0x7f28080fd8b0 msgr2=0x7f28080fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 shutdown_connections 2024-09-19T05:09:25.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:25.547+0000 7f2810469640 1 -- 172.21.15.64:0/3280482202 wait complete. 2024-09-19T05:09:25.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:25 smithi103 ceph-mon[34446]: pgmap v108: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:09:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:25 smithi174 ceph-mon[34488]: pgmap v108: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:09:26.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:26 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3280482202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:26.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:26 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3280482202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:26.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:26 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3280482202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:27.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:09:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:27.645 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:27 smithi064 ceph-mon[30960]: pgmap v109: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:09:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:27 smithi103 ceph-mon[34446]: pgmap v109: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:09:27.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:27 smithi174 ceph-mon[34488]: pgmap v109: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:09:29.660 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:29 smithi064 ceph-mon[30960]: pgmap v110: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:09:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:29 smithi103 ceph-mon[34446]: pgmap v110: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:09:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:29 smithi174 ceph-mon[34488]: pgmap v110: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:09:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 -- 172.21.15.64:0/3871638386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec10c180 msgr2=0x7f30ec10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 --2- 172.21.15.64:0/3871638386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec10c180 0x7f30ec10e5f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009ca0 tx=0x7f30e402a950 comp rx=0 tx=0).stop 2024-09-19T05:09:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 -- 172.21.15.64:0/3871638386 shutdown_connections 2024-09-19T05:09:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 --2- 172.21.15.64:0/3871638386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec10c180 0x7f30ec10e5f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 --2- 172.21.15.64:0/3871638386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec103700 0x7f30ec10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 --2- 172.21.15.64:0/3871638386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30ec102de0 0x7f30ec1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 -- 172.21.15.64:0/3871638386 >> 172.21.15.64:0/3871638386 conn(0x7f30ec0fc780 msgr2=0x7f30ec0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 -- 172.21.15.64:0/3871638386 shutdown_connections 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 -- 172.21.15.64:0/3871638386 wait complete. 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 Processor -- start 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.596+0000 7f30f253b640 1 -- start start 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f253b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec102de0 0x7f30ec19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f253b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 0x7f30ec19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f253b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30ec10c180 0x7f30ec19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f253b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec111dc0 con 0x7f30ec103700 2024-09-19T05:09:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f253b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec111c40 con 0x7f30ec102de0 2024-09-19T05:09:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f253b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec111f40 con 0x7f30ec10c180 2024-09-19T05:09:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 0x7f30ec19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 0x7f30ec19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34374/0 (socket says 172.21.15.64:34374) 2024-09-19T05:09:31.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 -- 172.21.15.64:0/3885597768 learned_addr learned my addr 172.21.15.64:0/3885597768 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:31.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30ec10c180 0x7f30ec19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30ebfff640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec102de0 0x7f30ec19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 -- 172.21.15.64:0/3885597768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30ec10c180 msgr2=0x7f30ec19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30ec10c180 0x7f30ec19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 -- 172.21.15.64:0/3885597768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec102de0 msgr2=0x7f30ec19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec102de0 0x7f30ec19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 -- 172.21.15.64:0/3885597768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec19f4c0 con 0x7f30ec103700 2024-09-19T05:09:31.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30ebfff640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec102de0 0x7f30ec19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:31.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 0x7f30ec19aa50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30e000c890 tx=0x7f30e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:31.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.597+0000 7f30e97fa640 1 -- 172.21.15.64:0/3885597768 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0004220 con 0x7f30ec103700 2024-09-19T05:09:31.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.598+0000 7f30e97fa640 1 -- 172.21.15.64:0/3885597768 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e00043c0 con 0x7f30ec103700 2024-09-19T05:09:31.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.598+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec19f7b0 con 0x7f30ec103700 2024-09-19T05:09:31.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.598+0000 7f30e97fa640 1 -- 172.21.15.64:0/3885597768 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0004a00 con 0x7f30ec103700 2024-09-19T05:09:31.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.598+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1a7090 con 0x7f30ec103700 2024-09-19T05:09:31.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30e97fa640 1 -- 172.21.15.64:0/3885597768 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f30e0004bc0 con 0x7f30ec103700 2024-09-19T05:09:31.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 Processor -- start 2024-09-19T05:09:31.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 -- start start 2024-09-19T05:09:31.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956267b40 0x55d956266090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956266a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956273760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9561cc2f0 con 0x55d956271370 2024-09-19T05:09:31.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d956173cf0 con 0x55d956267b40 2024-09-19T05:09:31.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f4914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d9561c82b0 con 0x55d9562665d0 2024-09-19T05:09:31.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956273760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f0ad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956267b40 0x55d956266090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f0ad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956267b40 0x55d956266090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:59436/0 (socket says 172.21.15.64:59436) 2024-09-19T05:09:31.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956273760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34388/0 (socket says 172.21.15.64:34388) 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.600+0000 7f30f0ad8640 1 -- 172.21.15.64:0/2436277146 learned_addr learned my addr 172.21.15.64:0/2436277146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30ebfff640 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956266a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30eb7fe640 1 -- 172.21.15.64:0/2436277146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 msgr2=0x55d956266a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956266a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30e97fa640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f30d40a3430 0x7f30d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30eb7fe640 1 -- 172.21.15.64:0/2436277146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956267b40 msgr2=0x55d956266090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956267b40 0x55d956266090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30eb7fe640 1 -- 172.21.15.64:0/2436277146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9561583a0 con 0x55d956271370 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30ebfff640 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956266a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f30d40a3430 0x7f30d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30e97fa640 1 -- 172.21.15.64:0/3885597768 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f30e00cf8e0 con 0x7f30ec103700 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956273760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f30e00cd030 tx=0x7f30e00d5f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bacaa6edb2175b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30d27fc640 1 -- 172.21.15.64:0/2436277146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e001d030 con 0x55d956271370 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30d27fc640 1 -- 172.21.15.64:0/2436277146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e009c2c0 con 0x55d956271370 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.601+0000 7f30d27fc640 1 -- 172.21.15.64:0/2436277146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e009b990 con 0x55d956271370 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 -- 172.21.15.64:0/2436277146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 msgr2=0x55d956273760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f30d40a3430 0x7f30d40a58f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009a60 tx=0x7f30e403c040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956273760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f30e00cd030 tx=0x7f30e00d5f60 comp rx=0 tx=0).stop 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 -- 172.21.15.64:0/2436277146 shutdown_connections 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956273760 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956266a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 --2- 172.21.15.64:0/2436277146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956267b40 0x55d956266090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 -- 172.21.15.64:0/2436277146 >> 172.21.15.64:0/2436277146 conn(0x55d95622f310 msgr2=0x55d956262c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 -- 172.21.15.64:0/2436277146 shutdown_connections 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 -- 172.21.15.64:0/2436277146 wait complete. 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.602+0000 7f30f4914740 1 -- start start 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 Processor -- start 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d956270da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d95626e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956271370 0x55d95626e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d956274070 con 0x55d956267b40 2024-09-19T05:09:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d956187060 con 0x55d9562665d0 2024-09-19T05:09:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d9562741f0 con 0x55d956271370 2024-09-19T05:09:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f4914740 10 client.0 _ll_register_callbacks cb 0x55d955eda8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d95626e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d95626e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34390/0 (socket says 172.21.15.64:34390) 2024-09-19T05:09:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 -- 172.21.15.64:0/1402984348 learned_addr learned my addr 172.21.15.64:0/1402984348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956271370 0x55d95626e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30ebfff640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d956270da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 -- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956271370 msgr2=0x55d95626e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956271370 0x55d95626e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 -- 172.21.15.64:0/1402984348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 msgr2=0x55d956270da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d956270da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30eb7fe640 1 -- 172.21.15.64:0/1402984348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d955f3d3c0 con 0x55d956267b40 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.603+0000 7f30ebfff640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d956270da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d95626e050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f30e00a4300 tx=0x7f30e00052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c210576276df62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30cb7fe640 1 -- 172.21.15.64:0/1402984348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e001d050 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e009c2c0 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30cb7fe640 1 -- 172.21.15.64:0/1402984348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f30e0097910 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30cb7fe640 1 -- 172.21.15.64:0/1402984348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30e009b520 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30f4914740 10 client.16263 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d956274070 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30cb7fe640 1 -- 172.21.15.64:0/1402984348 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e009c2c0 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.604+0000 7f30cb7fe640 1 -- 172.21.15.64:0/1402984348 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f30e009be20 con 0x55d956267b40 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30f4914740 10 client.16263 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30f4914740 10 client.16263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30f4914740 20 client.16263 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30f4914740 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d956280cd0 0x55d9562830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30f4914740 4 client.16263 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9561d9860 con 0x55d956280cd0 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30ebfff640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d956280cd0 0x55d9562830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30cb7fe640 10 client.16263 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.605+0000 7f30ebfff640 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d956280cd0 0x55d9562830c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c1671f63acf8a39 server_cookie=8e2ab7047402922c in_seq=0 out_seq=0 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30cb7fe640 1 -- 172.21.15.64:0/1402984348 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9561d9860 con 0x55d956280cd0 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30cb7fe640 10 client.16263 handle_command_reply: tid=0 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 client.16263 shutdown 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 msgr2=0x55d95626e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d95626e050 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f30e00a4300 tx=0x7f30e00052e0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 shutdown_connections 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d956280cd0 0x55d9562830c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956271370 0x55d95626e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d95626e050 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 --2- 172.21.15.64:0/1402984348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d956270da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30cb7fe640 0 client.16263 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.606+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 >> 172.21.15.64:0/1402984348 conn(0x55d95622f310 msgr2=0x55d956268960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.607+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 shutdown_connections 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.607+0000 7f30f4914740 1 -- 172.21.15.64:0/1402984348 wait complete. 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.607+0000 7f30f4914740 20 client.16263 trim_cache size 0 max 16384 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 Processor -- start 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 -- start start 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d95626d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956267b40 0x55d956288830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956288d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b2440 con 0x55d956271370 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1afd10 con 0x55d9562665d0 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f4914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1adb60 con 0x55d956267b40 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956288d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956288d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34394/0 (socket says 172.21.15.64:34394) 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 -- 172.21.15.64:0/1608627185 learned_addr learned my addr 172.21.15.64:0/1608627185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956267b40 0x55d956288830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30ebfff640 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d95626d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 -- 172.21.15.64:0/1608627185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956267b40 msgr2=0x55d956288830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956267b40 0x55d956288830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 -- 172.21.15.64:0/1608627185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 msgr2=0x55d95626d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d95626d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 -- 172.21.15.64:0/1608627185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95626c5a0 con 0x55d956271370 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.620+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956288d70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009ca0 tx=0x7f30e4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b41786e21b38c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30d1ffb640 1 -- 172.21.15.64:0/1608627185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4056070 con 0x55d956271370 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30d1ffb640 1 -- 172.21.15.64:0/1608627185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e4003e20 con 0x55d956271370 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30d1ffb640 1 -- 172.21.15.64:0/1608627185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4034040 con 0x55d956271370 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 -- 172.21.15.64:0/1608627185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 msgr2=0x55d956288d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956288d70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009ca0 tx=0x7f30e4042000 comp rx=0 tx=0).stop 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 -- 172.21.15.64:0/1608627185 shutdown_connections 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956271370 0x55d956288d70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d956267b40 0x55d956288830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 --2- 172.21.15.64:0/1608627185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d9562665d0 0x55d95626d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 -- 172.21.15.64:0/1608627185 >> 172.21.15.64:0/1608627185 conn(0x55d95622f310 msgr2=0x55d9562700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 -- 172.21.15.64:0/1608627185 shutdown_connections 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 -- 172.21.15.64:0/1608627185 wait complete. 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.621+0000 7f30f4914740 1 -- start start 2024-09-19T05:09:31.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 Processor -- start 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956279ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d956274de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956271370 0x55d956275320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1ae1d0 con 0x55d956267b40 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1af640 con 0x55d956271370 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1ae8a0 con 0x55d9562665d0 2024-09-19T05:09:31.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f4914740 10 client.0 _ll_register_callbacks cb 0x7f30e4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d956274de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d956274de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34402/0 (socket says 172.21.15.64:34402) 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 -- 172.21.15.64:0/170799026 learned_addr learned my addr 172.21.15.64:0/170799026 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30ebfff640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956279ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956271370 0x55d956275320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 -- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 msgr2=0x55d956279ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956279ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 -- 172.21.15.64:0/170799026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956271370 msgr2=0x55d956275320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956271370 0x55d956275320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30eb7fe640 1 -- 172.21.15.64:0/170799026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d956275a10 con 0x55d956267b40 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30ebfff640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956279ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.622+0000 7f30f0ad8640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956271370 0x55d956275320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30eb7fe640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d956274de0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f30e0004f00 tx=0x7f30e00d5f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ae6d59706219f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30cb7fe640 1 -- 172.21.15.64:0/170799026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e001d040 con 0x55d956267b40 2024-09-19T05:09:31.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30cb7fe640 1 -- 172.21.15.64:0/170799026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f30e00d5530 con 0x55d956267b40 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4003e20 con 0x55d956267b40 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30cb7fe640 1 -- 172.21.15.64:0/170799026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30e009b700 con 0x55d956267b40 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 10 client.16269 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d9562eb6e0 con 0x55d956267b40 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30cb7fe640 1 -- 172.21.15.64:0/170799026 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e4003e20 con 0x55d956267b40 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30cb7fe640 1 -- 172.21.15.64:0/170799026 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f30e00ba350 con 0x55d956267b40 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 10 client.16269 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:31.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 10 client.16269 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 20 client.16269 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d9562ebc90 0x55d9562ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 4 client.16269 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.623+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9561d3d70 con 0x55d9562ebc90 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30ebfff640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d9562ebc90 0x55d9562ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30cb7fe640 10 client.16269 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30ebfff640 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d9562ebc90 0x55d9562ee0b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ede9012480f3c839 server_cookie=775290173d9b6fef in_seq=0 out_seq=0 2024-09-19T05:09:31.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30cb7fe640 1 -- 172.21.15.64:0/170799026 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9561d3d70 con 0x55d9562ebc90 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30cb7fe640 10 client.16269 handle_command_reply: tid=0 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 client.16269 shutdown 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 msgr2=0x55d956274de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d956274de0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f30e0004f00 tx=0x7f30e00d5f60 comp rx=0 tx=0).stop 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 shutdown_connections 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d9562ebc90 0x55d9562ee0b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d956271370 0x55d956275320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d956267b40 0x55d956274de0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30cb7fe640 0 client.16269 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:31.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 --2- 172.21.15.64:0/170799026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d9562665d0 0x55d956279ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 >> 172.21.15.64:0/170799026 conn(0x55d95622f310 msgr2=0x55d95626f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:31.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 shutdown_connections 2024-09-19T05:09:31.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.624+0000 7f30f4914740 1 -- 172.21.15.64:0/170799026 wait complete. 2024-09-19T05:09:31.483 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:31.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f4914740 20 client.16269 trim_cache size 0 max 16384 2024-09-19T05:09:31.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f30d40a3430 msgr2=0x7f30d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f30d40a3430 0x7f30d40a58f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009a60 tx=0x7f30e403c040 comp rx=0 tx=0).stop 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 msgr2=0x7f30ec19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 0x7f30ec19aa50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30e000c890 tx=0x7f30e000cd50 comp rx=0 tx=0).stop 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 shutdown_connections 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f30d40a3430 0x7f30d40a58f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30ec10c180 0x7f30ec19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30ec103700 0x7f30ec19aa50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 --2- 172.21.15.64:0/3885597768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec102de0 0x7f30ec19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:31.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 >> 172.21.15.64:0/3885597768 conn(0x7f30ec0fc780 msgr2=0x7f30ec100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:31.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 shutdown_connections 2024-09-19T05:09:31.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:30.625+0000 7f30f253b640 1 -- 172.21.15.64:0/3885597768 wait complete. 2024-09-19T05:09:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:31 smithi064 ceph-mon[30960]: pgmap v111: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:09:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:31 smithi103 ceph-mon[34446]: pgmap v111: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:09:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:31 smithi174 ceph-mon[34488]: pgmap v111: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:09:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:33 smithi064 ceph-mon[30960]: pgmap v112: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:09:33.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:33 smithi103 ceph-mon[34446]: pgmap v112: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:09:33.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:33 smithi174 ceph-mon[34488]: pgmap v112: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T05:09:34.762 INFO:tasks.workunit.client.0.smithi064.stdout:writing pattern 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-09-19T05:09:34.763 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:34.764 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:34.765 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:34.766 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:34.767 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:34.768 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:34.769 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:34.784 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:34.785 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:34.786 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:34.787 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:34.788 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:34.789 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:34.790 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:34.790 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:34.790 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:34.790 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:34.790 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:34.802 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:34.802 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:34.803 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:34.804 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:34.805 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:34.806 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:34.807 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:34.808 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:34.809 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:34.809 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:34.809 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:34.809 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:34.809 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:34.809 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:34.821 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:34.821 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:34.822 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:34.823 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:34.824 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:34.825 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-19T05:09:34.826 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:34.827 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:34.828 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:34.828 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:34.828 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:34.842 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:34.842 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:34.842 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:34.843 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:34.844 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:34.845 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:34.846 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:34.847 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:34.848 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:34.849 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:34.849 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:34.849 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:34.849 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:34.849 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:34.849 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:34.863 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:34.864 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:34.865 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:34.866 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:34.867 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:34.868 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:34.869 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:34.879 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:34.879 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:34.880 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:34.881 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:34.882 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:34.883 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:34.884 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:34.885 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:34.901 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:34.902 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:34.903 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-19T05:09:34.904 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-19T05:09:34.905 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:34.906 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:34.908 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:34.909 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:34.909 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:34.909 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:34.909 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:34.921 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:34.921 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:34.921 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:34.921 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:34.921 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:34.922 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:34.923 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:34.924 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:34.925 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:34.926 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:34.927 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:34.928 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:34.928 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:34.928 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:34.941 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:34.942 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:34.943 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:34.944 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:34.945 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:34.946 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:34.947 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:34.959 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:34.960 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:34.961 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:34.962 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:34.963 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:34.964 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:34.965 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:34.978 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:34.978 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:34.978 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:34.978 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:34.978 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:34.979 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-19T05:09:34.980 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-19T05:09:34.981 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:34.982 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:34.983 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:34.984 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:34.998 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:34.999 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:35.000 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:35.001 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:35.002 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:35.003 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:35.004 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:35.018 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:35.019 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:35.020 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:35.021 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:35.022 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:35.023 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:35.024 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:35.034 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:35.034 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:35.034 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:35.034 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:35.034 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:35.035 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:35.036 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:35.037 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:35.038 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:35.039 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:35.040 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:35.054 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:35.054 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:35.054 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:35.054 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:35.055 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-19T05:09:35.056 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:35.057 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:35.058 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:35.059 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:35.060 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:35.075 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:35.075 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:35.075 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:35.076 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:35.077 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:35.078 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:35.079 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:35.080 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:35.081 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:35.093 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:35.094 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:35.095 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:35.096 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:35.097 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:35.098 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:35.099 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:35.111 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:35.112 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:35.113 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:35.114 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:35.116 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:35.117 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:35.118 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:35.119 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-09-19T05:09:35.131 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-19T05:09:35.132 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:35.133 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:35.134 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:35.135 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:35.136 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:35.137 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:35.152 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:35.153 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:35.154 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:35.155 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:35.156 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:35.157 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:35.158 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:35.158 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:35.158 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:35.158 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:35.158 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:35.170 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:35.171 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:35.172 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:35.173 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:35.174 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:35.175 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:35.176 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:35.206 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:35.207 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:35.208 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:35.209 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:35.210 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:35.211 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:35.212 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:35.212 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:35.212 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:35.227 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-09-19T05:09:35.227 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-09-19T05:09:35.228 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:35.229 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:35.230 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:35.231 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:35.232 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:35.233 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:35.234 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:35.249 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:35.249 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:35.249 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:35.249 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:35.249 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:35.250 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:35.251 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:35.252 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:35.253 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:35.254 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:35.255 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:35.267 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:35.268 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:35.269 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:35.270 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:35.271 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:35.272 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:35.273 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:35.273 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:35.273 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:35.273 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:35.284 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:35.284 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:35.285 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:35.286 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:35.287 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:35.288 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:35.289 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-09-19T05:09:35.290 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-19T05:09:35.291 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-09-19T05:09:35.291 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-19T05:09:35.291 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-09-19T05:09:35.291 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-19T05:09:35.305 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-09-19T05:09:35.305 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:35.306 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:35.307 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:35.308 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:35.309 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:35.310 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:35.311 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:35.312 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:35.312 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:35.312 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:35.326 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:35.327 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:35.328 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:35.329 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:35.330 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:35.331 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:35.332 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:35.344 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:35.345 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:35.346 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:35.347 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:35.348 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:35.349 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:35.350 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 -- 172.21.15.64:0/1990799245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa00101350 msgr2=0x7faa0010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 --2- 172.21.15.64:0/1990799245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa00101350 0x7faa0010d880 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009cd0 tx=0x7fa9f802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 -- 172.21.15.64:0/1990799245 shutdown_connections 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 --2- 172.21.15.64:0/1990799245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa00101350 0x7faa0010d880 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 --2- 172.21.15.64:0/1990799245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa001009b0 0x7faa00100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 --2- 172.21.15.64:0/1990799245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001069c0 0x7faa00106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 -- 172.21.15.64:0/1990799245 >> 172.21.15.64:0/1990799245 conn(0x7faa00077990 msgr2=0x7faa000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 -- 172.21.15.64:0/1990799245 shutdown_connections 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.433+0000 7faa079a6640 1 -- 172.21.15.64:0/1990799245 wait complete. 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 Processor -- start 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 -- start start 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001009b0 0x7faa00199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa00101350 0x7faa001995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 0x7faa001a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110cb0 con 0x7faa001069c0 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110b30 con 0x7faa001009b0 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa079a6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110e30 con 0x7faa00101350 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa05f43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 0x7faa001a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa05742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001009b0 0x7faa00199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa04f41640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa00101350 0x7faa001995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa05f43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 0x7faa001a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57646/0 (socket says 172.21.15.64:57646) 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.434+0000 7faa05f43640 1 -- 172.21.15.64:0/1453927873 learned_addr learned my addr 172.21.15.64:0/1453927873 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05742640 1 -- 172.21.15.64:0/1453927873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa00101350 msgr2=0x7faa001995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05742640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa00101350 0x7faa001995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05742640 1 -- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 msgr2=0x7faa001a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05742640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 0x7faa001a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05742640 1 -- 172.21.15.64:0/1453927873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001a4640 con 0x7faa001009b0 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa04f41640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa00101350 0x7faa001995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05f43640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 0x7faa001a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa05742640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001009b0 0x7faa00199080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e9f0 tx=0x7fa9fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7fa9f67fc640 1 -- 172.21.15.64:0/1453927873 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00ccc0 con 0x7faa001009b0 2024-09-19T05:09:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001a4810 con 0x7faa001009b0 2024-09-19T05:09:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7fa9f67fc640 1 -- 172.21.15.64:0/1453927873 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9fc0108e0 con 0x7faa001009b0 2024-09-19T05:09:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7fa9f67fc640 1 -- 172.21.15.64:0/1453927873 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc018880 con 0x7faa001009b0 2024-09-19T05:09:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.435+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a4d30 con 0x7faa001009b0 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.438+0000 7fa9f67fc640 1 -- 172.21.15.64:0/1453927873 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa9fc010a90 con 0x7faa001009b0 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.438+0000 7faa0a81f740 1 Processor -- start 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.438+0000 7faa0a81f740 1 -- start start 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.438+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e6cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.438+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6d2f0 0x562577e7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e7e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa0a81f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562577dd12f0 con 0x562577e6c9d0 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa0a81f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562577d79270 con 0x562577e6d2f0 2024-09-19T05:09:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa0a81f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562577dcd2b0 con 0x562577e7c0c0 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa05f43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e6cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa05f43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e6cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57652/0 (socket says 172.21.15.64:57652) 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa05f43640 1 -- 172.21.15.64:0/3191396342 learned_addr learned my addr 172.21.15.64:0/3191396342 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e7e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa05742640 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6d2f0 0x562577e7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 -- 172.21.15.64:0/3191396342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6d2f0 msgr2=0x562577e7bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6d2f0 0x562577e7bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7fa9f67fc640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9ec0a3460 0x7fa9ec0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 -- 172.21.15.64:0/3191396342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 msgr2=0x562577e6cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e6cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 -- 172.21.15.64:0/3191396342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562577d78340 con 0x562577e7c0c0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa05f43640 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e6cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7fa9f67fc640 1 -- 172.21.15.64:0/1453927873 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa9fc0cffe0 con 0x7faa001009b0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa05f43640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9ec0a3460 0x7fa9ec0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.439+0000 7faa04f41640 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e7e4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009fa0 tx=0x7fa9f00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6def02ad1924ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7fa9eb7fe640 1 -- 172.21.15.64:0/3191396342 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0004310 con 0x562577e7c0c0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7fa9eb7fe640 1 -- 172.21.15.64:0/3191396342 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f00044b0 con 0x562577e7c0c0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7fa9eb7fe640 1 -- 172.21.15.64:0/3191396342 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000f980 con 0x562577e7c0c0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 -- 172.21.15.64:0/3191396342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 msgr2=0x562577e7e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e7e4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009fa0 tx=0x7fa9f00118d0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa05f43640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9ec0a3460 0x7fa9ec0a5920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009df0 tx=0x7fa9f80095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 -- 172.21.15.64:0/3191396342 shutdown_connections 2024-09-19T05:09:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e7e4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6d2f0 0x562577e7bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3191396342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e6cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 -- 172.21.15.64:0/3191396342 >> 172.21.15.64:0/3191396342 conn(0x562577e69060 msgr2=0x562577e69440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.440+0000 7faa0a81f740 1 -- 172.21.15.64:0/3191396342 shutdown_connections 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 -- 172.21.15.64:0/3191396342 wait complete. 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 -- start start 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 Processor -- start 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e79ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562577e7ecf0 con 0x562577e6d2f0 2024-09-19T05:09:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562577d5a290 con 0x562577e6c9d0 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562577e7ee70 con 0x562577e7c0c0 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.441+0000 7faa0a81f740 10 client.0 _ll_register_callbacks cb 0x562577d8b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa05742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa05742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e79040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42600/0 (socket says 172.21.15.64:42600) 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa05742640 1 -- 172.21.15.64:0/3635490606 learned_addr learned my addr 172.21.15.64:0/3635490606 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa05f43640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e79ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 -- 172.21.15.64:0/3635490606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 msgr2=0x562577e79ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e79ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 -- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 msgr2=0x562577e79040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e79040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 -- 172.21.15.64:0/3635490606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562577ae4210 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa05742640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa04f41640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e79580 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009ef0 tx=0x7fa9f000f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d45f38541dc3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7fa9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/3635490606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0029030 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/3635490606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa9f00046c0 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f00044b0 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/3635490606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0033050 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa0a81f740 10 client.16281 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.442+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562577bb0620 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/3635490606 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f00044b0 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/3635490606 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9f000fa70 con 0x562577e6d2f0 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa0a81f740 10 client.16281 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa0a81f740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa0a81f740 20 client.16281 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577e8c0a0 0x562577e8e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa0a81f740 4 client.16281 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562577ddea20 con 0x562577e8c0a0 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.443+0000 7faa05742640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577e8c0a0 0x562577e8e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.444+0000 7faa05742640 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577e8c0a0 0x562577e8e490 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964ec56b41cd794c server_cookie=547a82b69c4e8981 in_seq=0 out_seq=0 2024-09-19T05:09:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.444+0000 7fa9e8ff9640 10 client.16281 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.444+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/3635490606 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562577ddea20 con 0x562577e8c0a0 2024-09-19T05:09:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.444+0000 7fa9e8ff9640 10 client.16281 handle_command_reply: tid=0 2024-09-19T05:09:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.444+0000 7faa0a81f740 1 client.16281 shutdown 2024-09-19T05:09:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 msgr2=0x562577e79580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e79580 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009ef0 tx=0x7fa9f000f4e0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 shutdown_connections 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e79ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577e8c0a0 0x562577e8e490 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e79580 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 --2- 172.21.15.64:0/3635490606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7fa9e8ff9640 0 client.16281 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 >> 172.21.15.64:0/3635490606 conn(0x562577e69060 msgr2=0x562577e6ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 shutdown_connections 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 1 -- 172.21.15.64:0/3635490606 wait complete. 2024-09-19T05:09:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.445+0000 7faa0a81f740 20 client.16281 trim_cache size 0 max 16384 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:35.455 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:35.456 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:35.457 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:35.458 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:35.459 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:35.460 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-19T05:09:35.461 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.461+0000 7faa0a81f740 1 Processor -- start 2024-09-19T05:09:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.461+0000 7faa0a81f740 1 -- start start 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e7afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e93bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa0a81f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa001ade00 con 0x562577e6d2f0 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa0a81f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa001ab6d0 con 0x562577e6c9d0 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa0a81f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa001a9540 con 0x562577e7c0c0 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e93bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa05742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e7afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e93bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57682/0 (socket says 172.21.15.64:57682) 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 -- 172.21.15.64:0/2373338993 learned_addr learned my addr 172.21.15.64:0/2373338993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa05742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e7afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42606/0 (socket says 172.21.15.64:42606) 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa05f43640 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 -- 172.21.15.64:0/2373338993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 msgr2=0x562577e940f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e940f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 -- 172.21.15.64:0/2373338993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 msgr2=0x562577e7afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e7afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 -- 172.21.15.64:0/2373338993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562577e88ad0 con 0x562577e6d2f0 2024-09-19T05:09:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7faa04f41640 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e93bb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00059f0 tx=0x7fa9f0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e57ca0cf20258 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7fa9eaffd640 1 -- 172.21.15.64:0/2373338993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0029040 con 0x562577e6d2f0 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7fa9eaffd640 1 -- 172.21.15.64:0/2373338993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f0003640 con 0x562577e6d2f0 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.462+0000 7fa9eaffd640 1 -- 172.21.15.64:0/2373338993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000fb70 con 0x562577e6d2f0 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 -- 172.21.15.64:0/2373338993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 msgr2=0x562577e93bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e93bb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00059f0 tx=0x7fa9f0005b90 comp rx=0 tx=0).stop 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 -- 172.21.15.64:0/2373338993 shutdown_connections 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e7c0c0 0x562577e940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6d2f0 0x562577e93bb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 --2- 172.21.15.64:0/2373338993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e6c9d0 0x562577e7afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 -- 172.21.15.64:0/2373338993 >> 172.21.15.64:0/2373338993 conn(0x562577e69060 msgr2=0x562577e7a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 -- 172.21.15.64:0/2373338993 shutdown_connections 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.463+0000 7faa0a81f740 1 -- 172.21.15.64:0/2373338993 wait complete. 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 -- start start 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 Processor -- start 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e849a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e6d2f0 0x562577e7fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e7c0c0 0x562577e80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa001a9b90 con 0x562577e6c9d0 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa001ab000 con 0x562577e7c0c0 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa001aa260 con 0x562577e6d2f0 2024-09-19T05:09:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa0a81f740 10 client.0 _ll_register_callbacks cb 0x7fa9f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa05742640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa05742640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e849a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57686/0 (socket says 172.21.15.64:57686) 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa05742640 1 -- 172.21.15.64:0/1992322567 learned_addr learned my addr 172.21.15.64:0/1992322567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa05f43640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e7c0c0 0x562577e80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.464+0000 7faa04f41640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e6d2f0 0x562577e7fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa05742640 1 -- 172.21.15.64:0/1992322567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e6d2f0 msgr2=0x562577e7fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa05742640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e6d2f0 0x562577e7fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa05742640 1 -- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e7c0c0 msgr2=0x562577e80450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa05742640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e7c0c0 0x562577e80450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa05742640 1 -- 172.21.15.64:0/1992322567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562577ef5b80 con 0x562577e6c9d0 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa05742640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e849a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0cd7e0 tx=0x7fa9fc0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742dd23387d0dccd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7fa9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/1992322567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc09bcd0 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/1992322567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fa9fc0989e0 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f0003640 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/1992322567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9fc010e00 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa0a81f740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562577ef6d30 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.465+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/1992322567 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f0003640 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/1992322567 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa9fc0b5140 con 0x562577e6c9d0 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa0a81f740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa0a81f740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa0a81f740 20 client.16287 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa0a81f740 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577ef7400 0x562577ef97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa0a81f740 4 client.16287 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T05:09:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562577dd8d70 con 0x562577ef7400 2024-09-19T05:09:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa04f41640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577ef7400 0x562577ef97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7fa9e8ff9640 10 client.16287 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.466+0000 7faa04f41640 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577ef7400 0x562577ef97f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d47f2a607f7d824 server_cookie=8d655fa5bf41b2eb in_seq=0 out_seq=0 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.498+0000 7fa9e8ff9640 1 -- 172.21.15.64:0/1992322567 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562577dd8d70 con 0x562577ef7400 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.498+0000 7fa9e8ff9640 10 client.16287 handle_command_reply: tid=0 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.498+0000 7faa0a81f740 1 client.16287 shutdown 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 msgr2=0x562577e849a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e849a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0cd7e0 tx=0x7fa9fc0bc040 comp rx=0 tx=0).stop 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 shutdown_connections 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562577e7c0c0 0x562577e80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562577e6d2f0 0x562577e7fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562577ef7400 0x562577ef97f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 --2- 172.21.15.64:0/1992322567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562577e6c9d0 0x562577e849a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7fa9e8ff9640 0 client.16287 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 >> 172.21.15.64:0/1992322567 conn(0x562577e69060 msgr2=0x562577e79800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 shutdown_connections 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 1 -- 172.21.15.64:0/1992322567 wait complete. 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa0a81f740 20 client.16287 trim_cache size 0 max 16384 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-19T05:09:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-19T05:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9ec0a3460 msgr2=0x7fa9ec0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9ec0a3460 0x7fa9ec0a5920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009df0 tx=0x7fa9f80095c0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001009b0 msgr2=0x7faa00199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001009b0 0x7faa00199080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e9f0 tx=0x7fa9fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 shutdown_connections 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9ec0a3460 0x7fa9ec0a5920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.499+0000 7faa079a6640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa001069c0 0x7faa001a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.500+0000 7faa079a6640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa00101350 0x7faa001995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.500+0000 7faa079a6640 1 --2- 172.21.15.64:0/1453927873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa001009b0 0x7faa00199080 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.500+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 >> 172.21.15.64:0/1453927873 conn(0x7faa00077990 msgr2=0x7faa00103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.500+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 shutdown_connections 2024-09-19T05:09:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-19T05:09:35.500+0000 7faa079a6640 1 -- 172.21.15.64:0/1453927873 wait complete. 2024-09-19T05:09:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:35 smithi064 ceph-mon[30960]: pgmap v113: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:09:35.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:35 smithi103 ceph-mon[34446]: pgmap v113: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:09:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:35 smithi174 ceph-mon[34488]: pgmap v113: 97 pgs: 97 active+clean; 7.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T05:09:35.879 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-19T05:09:35.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:35.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:35.882 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:35.883 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:35.884 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-19T05:09:35.885 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:35.886 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:36.309 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:36.310 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:36.311 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-19T05:09:36.312 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:36.313 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:36.314 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-19T05:09:36.315 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:36.316 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:36.316 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:36.316 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-19T05:09:36.710 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:36.710 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:36.710 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:36.710 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-19T05:09:36.710 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:36.711 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:36.712 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:36.713 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:36.714 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-19T05:09:36.715 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:36.716 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:36.717 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:36.717 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-19T05:09:36.717 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:36.717 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:36.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:09:37.059 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:37.060 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:37.061 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:37.062 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:37.063 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:37.064 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-19T05:09:37.065 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:37.066 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:37.066 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:37.066 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-19T05:09:37.066 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:37.066 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:37.081 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:37.502 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:37.502 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-19T05:09:37.502 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:37.503 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:37.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:37.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-19T05:09:37.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:37.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:37.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-19T05:09:37.542 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:37 smithi064 ceph-mon[30960]: pgmap v114: 97 pgs: 97 active+clean; 9.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 -- 172.21.15.64:0/338930624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b38101df0 msgr2=0x7f3b381021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 --2- 172.21.15.64:0/338930624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b38101df0 0x7f3b381021d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009900 tx=0x7f3b2802a410 comp rx=0 tx=0).stop 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 -- 172.21.15.64:0/338930624 shutdown_connections 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 --2- 172.21.15.64:0/338930624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38106f20 0x7f3b3810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 --2- 172.21.15.64:0/338930624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b381027a0 0x7f3b381067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 --2- 172.21.15.64:0/338930624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b38101df0 0x7f3b381021d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 -- 172.21.15.64:0/338930624 >> 172.21.15.64:0/338930624 conn(0x7f3b380fd8b0 msgr2=0x7f3b380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 -- 172.21.15.64:0/338930624 shutdown_connections 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.629+0000 7f3b3e347640 1 -- 172.21.15.64:0/338930624 wait complete. 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 Processor -- start 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 -- start start 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38101df0 0x7f3b380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 0x7f3b38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b38106f20 0x7f3b38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38111e30 con 0x7f3b381027a0 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38111cb0 con 0x7f3b38101df0 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b3e347640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38111fb0 con 0x7f3b38106f20 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.630+0000 7f3b377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 0x7f3b38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38101df0 0x7f3b380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 0x7f3b38074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57710/0 (socket says 172.21.15.64:57710) 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 -- 172.21.15.64:0/3374248828 learned_addr learned my addr 172.21.15.64:0/3374248828 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b38106f20 0x7f3b38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 -- 172.21.15.64:0/3374248828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b38106f20 msgr2=0x7f3b38074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b38106f20 0x7f3b38074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 -- 172.21.15.64:0/3374248828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38101df0 msgr2=0x7f3b380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38101df0 0x7f3b380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 -- 172.21.15.64:0/3374248828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b381a6830 con 0x7f3b381027a0 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b37fff640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38101df0 0x7f3b380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b377fe640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 0x7f3b38074a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9f0 tx=0x7f3b2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b357fa640 1 -- 172.21.15.64:0/3374248828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00cd80 con 0x7f3b381027a0 2024-09-19T05:09:37.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b381a6b20 con 0x7f3b381027a0 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b357fa640 1 -- 172.21.15.64:0/3374248828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b2c0054d0 con 0x7f3b381027a0 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b357fa640 1 -- 172.21.15.64:0/3374248828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c010490 con 0x7f3b381027a0 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.631+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b381a7060 con 0x7f3b381027a0 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b357fa640 1 -- 172.21.15.64:0/3374248828 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3b2c005020 con 0x7f3b381027a0 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 Processor -- start 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 -- start start 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55134090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa551345d0 0x55aa55134a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa5513f370 0x55aa55141760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa5509a2f0 con 0x55aa551345d0 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa55041cf0 con 0x55aa5513f370 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.634+0000 7f3b40720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa550962b0 con 0x55aa55135b40 2024-09-19T05:09:37.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa551345d0 0x55aa55134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:43942/0 (socket says 172.21.15.64:43942) 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/1512785933 learned_addr learned my addr 172.21.15.64:0/1512785933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b377fe640 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa5513f370 0x55aa55141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/1512785933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa5513f370 msgr2=0x55aa55141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa5513f370 0x55aa55141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/1512785933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa551345d0 msgr2=0x55aa55134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa551345d0 0x55aa55134a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/1512785933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa550263a0 con 0x55aa55135b40 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b357fa640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b200a33b0 0x7f3b200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b37fff640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b200a33b0 0x7f3b200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b357fa640 1 -- 172.21.15.64:0/3374248828 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3b2c0d0080 con 0x7f3b381027a0 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55134090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009f50 tx=0x7f3b300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feda89ba7faa16f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b1e7fc640 1 -- 172.21.15.64:0/1512785933 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004d20 con 0x55aa55135b40 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b1e7fc640 1 -- 172.21.15.64:0/1512785933 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b30004050 con 0x55aa55135b40 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.635+0000 7f3b1e7fc640 1 -- 172.21.15.64:0/1512785933 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30005280 con 0x55aa55135b40 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 -- 172.21.15.64:0/1512785933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 msgr2=0x55aa55134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b37fff640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b200a33b0 0x7f3b200a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b28031440 tx=0x7f3b28005b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55134090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009f50 tx=0x7f3b300118d0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 -- 172.21.15.64:0/1512785933 shutdown_connections 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa5513f370 0x55aa55141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa551345d0 0x55aa55134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 --2- 172.21.15.64:0/1512785933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55134090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 -- 172.21.15.64:0/1512785933 >> 172.21.15.64:0/1512785933 conn(0x55aa550fd310 msgr2=0x55aa55130c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 -- 172.21.15.64:0/1512785933 shutdown_connections 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.636+0000 7f3b40720740 1 -- 172.21.15.64:0/1512785933 wait complete. 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 -- start start 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 Processor -- start 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5513bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa5513c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa55142070 con 0x55aa5513f370 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa55055060 con 0x55aa55135b40 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa551421f0 con 0x55aa551345d0 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b40720740 10 client.0 _ll_register_callbacks cb 0x55aa54da88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa5513c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa5513c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57720/0 (socket says 172.21.15.64:57720) 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5513bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/3207717285 learned_addr learned my addr 172.21.15.64:0/3207717285 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.637+0000 7f3b37fff640 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 msgr2=0x55aa5513ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/3207717285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 msgr2=0x55aa5513bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5513bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/3207717285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa54e0b3c0 con 0x55aa5513f370 2024-09-19T05:09:37.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa5513c570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009250 tx=0x7f3b30004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f182b611b3032b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b177fe640 1 -- 172.21.15.64:0/3207717285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30028070 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b177fe640 1 -- 172.21.15.64:0/3207717285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3b30005040 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30004050 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b177fe640 1 -- 172.21.15.64:0/3207717285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b30009be0 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b40720740 10 client.16299 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aa5514dac0 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.638+0000 7f3b177fe640 1 -- 172.21.15.64:0/3207717285 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b30004050 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b177fe640 1 -- 172.21.15.64:0/3207717285 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b30024580 con 0x55aa5513f370 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b40720740 10 client.16299 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b40720740 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:09:37.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b40720740 20 client.16299 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b40720740 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa5514eed0 0x55aa551512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b40720740 4 client.16299 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa550a7860 con 0x55aa5514eed0 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b37fff640 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa5514eed0 0x55aa551512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b177fe640 10 client.16299 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.639+0000 7f3b37fff640 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa5514eed0 0x55aa551512c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59821ff075f6af6e server_cookie=6d84cdbb1dabe1a3 in_seq=0 out_seq=0 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b177fe640 1 -- 172.21.15.64:0/3207717285 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa550a7860 con 0x55aa5514eed0 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b177fe640 10 client.16299 handle_command_reply: tid=0 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 client.16299 shutdown 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 msgr2=0x55aa5513c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa5513c570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009250 tx=0x7f3b30004430 comp rx=0 tx=0).stop 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 shutdown_connections 2024-09-19T05:09:37.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa5514eed0 0x55aa551512c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa5513c570 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5513bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 --2- 172.21.15.64:0/3207717285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b177fe640 0 client.16299 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 >> 172.21.15.64:0/3207717285 conn(0x55aa550fd310 msgr2=0x55aa55136930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 shutdown_connections 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 1 -- 172.21.15.64:0/3207717285 wait complete. 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.640+0000 7f3b40720740 20 client.16299 trim_cache size 0 max 16384 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.654+0000 7f3b40720740 1 Processor -- start 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.654+0000 7f3b40720740 1 -- start start 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.654+0000 7f3b40720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.654+0000 7f3b40720740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5514b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.654+0000 7f3b40720740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55137120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.654+0000 7f3b40720740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381b2230 con 0x55aa5513f370 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b40720740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381afb00 con 0x55aa55135b40 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b40720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381ad950 con 0x55aa551345d0 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55137120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5514b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55137120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57730/0 (socket says 172.21.15.64:57730) 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/2430503916 learned_addr learned my addr 172.21.15.64:0/2430503916 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/2430503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 msgr2=0x55aa5513e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/2430503916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 msgr2=0x55aa5514b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5514b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/2430503916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa5513a520 con 0x55aa5513f370 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b3c8e4640 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55137120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3b30011910 tx=0x7f3b300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f37063bee57961 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b1dffb640 1 -- 172.21.15.64:0/2430503916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3003d070 con 0x55aa5513f370 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b1dffb640 1 -- 172.21.15.64:0/2430503916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b300044a0 con 0x55aa5513f370 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.655+0000 7f3b1dffb640 1 -- 172.21.15.64:0/2430503916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300246a0 con 0x55aa5513f370 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 -- 172.21.15.64:0/2430503916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 msgr2=0x55aa55137120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55137120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3b30011910 tx=0x7f3b300047c0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 -- 172.21.15.64:0/2430503916 shutdown_connections 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55137120 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa55135b40 0x55aa5514b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 --2- 172.21.15.64:0/2430503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa551345d0 0x55aa5513e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 -- 172.21.15.64:0/2430503916 >> 172.21.15.64:0/2430503916 conn(0x55aa550fd310 msgr2=0x55aa5512fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:37.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 -- 172.21.15.64:0/2430503916 shutdown_connections 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 -- 172.21.15.64:0/2430503916 wait complete. 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.656+0000 7f3b40720740 1 -- start start 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 Processor -- start 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa551345d0 0x55aa55147c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55142f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381adfc0 con 0x55aa5513f370 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381af430 con 0x55aa551345d0 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b381ae690 con 0x55aa55135b40 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b40720740 10 client.0 _ll_register_callbacks cb 0x7f3b2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa551345d0 0x55aa55147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b3c8e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57736/0 (socket says 172.21.15.64:57736) 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa551345d0 0x55aa55147c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42696/0 (socket says 172.21.15.64:42696) 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.657+0000 7f3b3c8e4640 1 -- 172.21.15.64:0/2399616137 learned_addr learned my addr 172.21.15.64:0/2399616137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b377fe640 1 -- 172.21.15.64:0/2399616137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa551345d0 msgr2=0x55aa55147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b377fe640 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa551345d0 0x55aa55147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b377fe640 1 -- 172.21.15.64:0/2399616137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 msgr2=0x55aa55143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b377fe640 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b377fe640 1 -- 172.21.15.64:0/2399616137 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa55143c80 con 0x55aa55135b40 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b377fe640 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55142f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0cd7d0 tx=0x7f3b2c0986c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d8c329688a72d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b177fe640 1 -- 172.21.15.64:0/2399616137 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c005310 con 0x55aa55135b40 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b177fe640 1 -- 172.21.15.64:0/2399616137 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3b2c0989d0 con 0x55aa55135b40 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b300044a0 con 0x55aa55135b40 2024-09-19T05:09:37.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b177fe640 1 -- 172.21.15.64:0/2399616137 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b2c005680 con 0x55aa55135b40 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b40720740 10 client.25690 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aa551b99e0 con 0x55aa55135b40 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.658+0000 7f3b177fe640 1 -- 172.21.15.64:0/2399616137 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b300044a0 con 0x55aa55135b40 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b177fe640 1 -- 172.21.15.64:0/2399616137 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b2c0a4b40 con 0x55aa55135b40 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 10 client.25690 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 10 client.25690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 20 client.25690 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa551ba110 0x55aa551bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 4 client.25690 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa550a1d70 con 0x55aa551ba110 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b37fff640 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa551ba110 0x55aa551bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b177fe640 10 client.25690 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b37fff640 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa551ba110 0x55aa551bc530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7e8dd7d92aaacbc server_cookie=4504f9afff368a45 in_seq=0 out_seq=0 2024-09-19T05:09:37.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b177fe640 1 -- 172.21.15.64:0/2399616137 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa550a1d70 con 0x55aa551ba110 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b177fe640 10 client.25690 handle_command_reply: tid=0 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.659+0000 7f3b40720740 1 client.25690 shutdown 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 msgr2=0x55aa55142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55142f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0cd7d0 tx=0x7f3b2c0986c0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 shutdown_connections 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55aa551ba110 0x55aa551bc530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:37.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa5513f370 0x55aa55143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b177fe640 0 client.25690 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa55135b40 0x55aa55142f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 --2- 172.21.15.64:0/2399616137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa551345d0 0x55aa55147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 >> 172.21.15.64:0/2399616137 conn(0x55aa550fd310 msgr2=0x55aa5514a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 shutdown_connections 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 1 -- 172.21.15.64:0/2399616137 wait complete. 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b40720740 20 client.25690 trim_cache size 0 max 16384 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b200a33b0 msgr2=0x7f3b200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b3e347640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b200a33b0 0x7f3b200a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b28031440 tx=0x7f3b28005b50 comp rx=0 tx=0).stop 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 msgr2=0x7f3b38074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.660+0000 7f3b3e347640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 0x7f3b38074a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9f0 tx=0x7f3b2c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 shutdown_connections 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b200a33b0 0x7f3b200a5870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b38106f20 0x7f3b38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b381027a0 0x7f3b38074a30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 --2- 172.21.15.64:0/3374248828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b38101df0 0x7f3b380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 >> 172.21.15.64:0/3374248828 conn(0x7f3b380fd8b0 msgr2=0x7f3b380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 shutdown_connections 2024-09-19T05:09:37.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:37.661+0000 7f3b3e347640 1 -- 172.21.15.64:0/3374248828 wait complete. 2024-09-19T05:09:37.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:37 smithi103 ceph-mon[34446]: pgmap v114: 97 pgs: 97 active+clean; 9.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-09-19T05:09:37.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:37 smithi174 ceph-mon[34488]: pgmap v114: 97 pgs: 97 active+clean; 9.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:37.889 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:37.890 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:37.891 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:37.892 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-19T05:09:37.893 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:37.894 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:37.895 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-19T05:09:37.895 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:37.895 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:38.293 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:38.294 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:38.295 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:38.296 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-19T05:09:38.297 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:38.298 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-19T05:09:38.299 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:38.299 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:38.299 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:38.299 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-19T05:09:38.299 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:38.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:09:38.617 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:38.617 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:38.617 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-19T05:09:38.617 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:38.618 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:38.619 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:38.620 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:38.621 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:38.622 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-19T05:09:38.623 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:38.767 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:39.005 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:39.006 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:39.007 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-19T05:09:39.008 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:39.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:39.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-19T05:09:39.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:39.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:39.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:39.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-19T05:09:39.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:39.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:39.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.314+0000 7f2869f96640 1 -- 172.21.15.64:0/2196154299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 msgr2=0x7f286410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.314+0000 7f2869f96640 1 --2- 172.21.15.64:0/2196154299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 0x7f286410ec40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f285c009cd0 tx=0x7f285c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 -- 172.21.15.64:0/2196154299 shutdown_connections 2024-09-19T05:09:39.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 --2- 172.21.15.64:0/2196154299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 0x7f286410ec40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 --2- 172.21.15.64:0/2196154299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28641036e0 0x7f2864107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 --2- 172.21.15.64:0/2196154299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2864102d30 0x7f2864103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 -- 172.21.15.64:0/2196154299 >> 172.21.15.64:0/2196154299 conn(0x7f28640fc7d0 msgr2=0x7f28640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 -- 172.21.15.64:0/2196154299 shutdown_connections 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 -- 172.21.15.64:0/2196154299 wait complete. 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.315+0000 7f2869f96640 1 Processor -- start 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 -- start start 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 0x7f2864198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 0x7f2864198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 0x7f286419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2864111fb0 con 0x7f28641082a0 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2864111e30 con 0x7f2864102d30 2024-09-19T05:09:39.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2869f96640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2864112130 con 0x7f28641036e0 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 0x7f2864198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f28637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 0x7f2864198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 0x7f2864198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:43960/0 (socket says 172.21.15.64:43960) 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f28637fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 0x7f2864198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42712/0 (socket says 172.21.15.64:42712) 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2862ffd640 1 -- 172.21.15.64:0/851831788 learned_addr learned my addr 172.21.15.64:0/851831788 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.316+0000 7f2863fff640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 0x7f286419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f28637fe640 1 -- 172.21.15.64:0/851831788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 msgr2=0x7f2864198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f28637fe640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 0x7f2864198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f28637fe640 1 -- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 msgr2=0x7f286419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f28637fe640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 0x7f286419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f28637fe640 1 -- 172.21.15.64:0/851831788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286419d4b0 con 0x7f2864102d30 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f2862ffd640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 0x7f2864198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:39.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f28637fe640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 0x7f2864198500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f285800b3f0 tx=0x7f285800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f2860ff9640 1 -- 172.21.15.64:0/851831788 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858015020 con 0x7f2864102d30 2024-09-19T05:09:39.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286419d7a0 con 0x7f2864102d30 2024-09-19T05:09:39.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f2860ff9640 1 -- 172.21.15.64:0/851831788 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2858011ce0 con 0x7f2864102d30 2024-09-19T05:09:39.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f2860ff9640 1 -- 172.21.15.64:0/851831788 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858010d80 con 0x7f2864102d30 2024-09-19T05:09:39.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.317+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28641008c0 con 0x7f2864102d30 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.319+0000 7f2860ff9640 1 -- 172.21.15.64:0/851831788 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2858018020 con 0x7f2864102d30 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 Processor -- start 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 -- start start 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f2860ff9640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f284c0a33b0 0x7f284c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132aa5d0 0x55e7132aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132b5370 0x55e7132b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e7132102f0 con 0x55e7132abb40 2024-09-19T05:09:39.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e7131b7cf0 con 0x55e7132aa5d0 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f286c36f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e71320c2b0 con 0x55e7132b5370 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.320+0000 7f2863fff640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f284c0a33b0 0x7f284c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132aa5d0 0x55e7132aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132aa5d0 0x55e7132aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42724/0 (socket says 172.21.15.64:42724) 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2862ffd640 1 -- 172.21.15.64:0/560257361 learned_addr learned my addr 172.21.15.64:0/560257361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2863fff640 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132b5370 0x55e7132b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2860ff9640 1 -- 172.21.15.64:0/851831788 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f28580cf440 con 0x7f2864102d30 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 -- 172.21.15.64:0/560257361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132b5370 msgr2=0x55e7132b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132b5370 0x55e7132b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 -- 172.21.15.64:0/560257361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132aa5d0 msgr2=0x55e7132aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132aa5d0 0x55e7132aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 -- 172.21.15.64:0/560257361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71319c3a0 con 0x55e7132abb40 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2863fff640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f284c0a33b0 0x7f284c0a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f285c005e10 tx=0x7f285c0316e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f28637fe640 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132aa090 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f28580ccc40 tx=0x7f2858065930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ccf0366e2ccc62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2849ffb640 1 -- 172.21.15.64:0/560257361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858064370 con 0x55e7132abb40 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2849ffb640 1 -- 172.21.15.64:0/560257361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2858010480 con 0x55e7132abb40 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.321+0000 7f2849ffb640 1 -- 172.21.15.64:0/560257361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858009200 con 0x55e7132abb40 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 -- 172.21.15.64:0/560257361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 msgr2=0x55e7132aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132aa090 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f28580ccc40 tx=0x7f2858065930 comp rx=0 tx=0).stop 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 -- 172.21.15.64:0/560257361 shutdown_connections 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132b5370 0x55e7132b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132aa5d0 0x55e7132aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 --2- 172.21.15.64:0/560257361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132aa090 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 -- 172.21.15.64:0/560257361 >> 172.21.15.64:0/560257361 conn(0x55e713273310 msgr2=0x55e7132a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 -- 172.21.15.64:0/560257361 shutdown_connections 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.322+0000 7f286c36f740 1 -- 172.21.15.64:0/560257361 wait complete. 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 -- start start 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 Processor -- start 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 0x55e7132b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132abb40 0x55e7132b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e7132b7fb0 con 0x55e7132aa5d0 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e7131cb060 con 0x55e7132b5370 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e7132b8130 con 0x55e7132abb40 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f286c36f740 10 client.0 _ll_register_callbacks cb 0x55e712f1e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f28637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 0x55e7132b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f28637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 0x55e7132b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57770/0 (socket says 172.21.15.64:57770) 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f28637fe640 1 -- 172.21.15.64:0/556914234 learned_addr learned my addr 172.21.15.64:0/556914234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.323+0000 7f2863fff640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2862ffd640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132abb40 0x55e7132b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f28637fe640 1 -- 172.21.15.64:0/556914234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132abb40 msgr2=0x55e7132b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f28637fe640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132abb40 0x55e7132b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f28637fe640 1 -- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 msgr2=0x55e7132b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f28637fe640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f28637fe640 1 -- 172.21.15.64:0/556914234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e712f813c0 con 0x55e7132aa5d0 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2863fff640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f28637fe640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 0x55e7132b4db0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2858011040 tx=0x7f28580047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bc5345af100202 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2842ffd640 1 -- 172.21.15.64:0/556914234 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28580b3070 con 0x55e7132aa5d0 2024-09-19T05:09:39.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2842ffd640 1 -- 172.21.15.64:0/556914234 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2858064110 con 0x55e7132aa5d0 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e71321d860 con 0x55e7132aa5d0 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2842ffd640 1 -- 172.21.15.64:0/556914234 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28580ccfb0 con 0x55e7132aa5d0 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f286c36f740 10 client.16317 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7132b7fb0 con 0x55e7132aa5d0 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.324+0000 7f2842ffd640 1 -- 172.21.15.64:0/556914234 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e71321d860 con 0x55e7132aa5d0 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f2842ffd640 1 -- 172.21.15.64:0/556914234 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2858060b40 con 0x55e7132aa5d0 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f286c36f740 10 client.16317 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f286c36f740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f286c36f740 20 client.16317 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f286c36f740 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e7132c4c20 0x55e7132c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f286c36f740 4 client.16317 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:39.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e713217d70 con 0x55e7132c4c20 2024-09-19T05:09:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f2862ffd640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e7132c4c20 0x55e7132c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f2842ffd640 10 client.16317 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:39.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.325+0000 7f2862ffd640 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e7132c4c20 0x55e7132c7010 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8905588cd7ddd634 server_cookie=e3d98fd83182d84a in_seq=0 out_seq=0 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f2842ffd640 1 -- 172.21.15.64:0/556914234 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e713217d70 con 0x55e7132c4c20 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f2842ffd640 10 client.16317 handle_command_reply: tid=0 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 client.16317 shutdown 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 msgr2=0x55e7132b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 0x55e7132b4db0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2858011040 tx=0x7f28580047d0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 shutdown_connections 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132abb40 0x55e7132b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e7132c4c20 0x55e7132c7010 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 --2- 172.21.15.64:0/556914234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132aa5d0 0x55e7132b4db0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f2842ffd640 0 client.16317 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:39.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 >> 172.21.15.64:0/556914234 conn(0x55e713273310 msgr2=0x55e7132ac990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 shutdown_connections 2024-09-19T05:09:39.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 1 -- 172.21.15.64:0/556914234 wait complete. 2024-09-19T05:09:39.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.326+0000 7f286c36f740 20 client.16317 trim_cache size 0 max 16384 2024-09-19T05:09:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 Processor -- start 2024-09-19T05:09:39.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 -- start start 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28641b23c0 con 0x55e7132abb40 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28641afc90 con 0x55e7132b5370 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f286c36f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28641adae0 con 0x55e7132aa5d0 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57780/0 (socket says 172.21.15.64:57780) 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 -- 172.21.15.64:0/989914397 learned_addr learned my addr 172.21.15.64:0/989914397 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:39.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2863fff640 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 -- 172.21.15.64:0/989914397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 msgr2=0x55e7132ad120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132ad120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 -- 172.21.15.64:0/989914397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 msgr2=0x55e7132cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.341+0000 7f2862ffd640 1 -- 172.21.15.64:0/989914397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7132b05b0 con 0x55e7132abb40 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f2862ffd640 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132ad660 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55e7132a7ac0 tx=0x7f2854005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af8524ebde85048 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f28497fa640 1 -- 172.21.15.64:0/989914397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854004250 con 0x55e7132abb40 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f28497fa640 1 -- 172.21.15.64:0/989914397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28540043f0 con 0x55e7132abb40 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f28497fa640 1 -- 172.21.15.64:0/989914397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540047a0 con 0x55e7132abb40 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 -- 172.21.15.64:0/989914397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 msgr2=0x55e7132ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132ad660 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55e7132a7ac0 tx=0x7f2854005f90 comp rx=0 tx=0).stop 2024-09-19T05:09:39.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 -- 172.21.15.64:0/989914397 shutdown_connections 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132ad660 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 --2- 172.21.15.64:0/989914397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.342+0000 7f286c36f740 1 -- 172.21.15.64:0/989914397 >> 172.21.15.64:0/989914397 conn(0x55e713273310 msgr2=0x55e7132b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.343+0000 7f286c36f740 1 -- 172.21.15.64:0/989914397 shutdown_connections 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.343+0000 7f286c36f740 1 -- 172.21.15.64:0/989914397 wait complete. 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.343+0000 7f286c36f740 1 -- start start 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.343+0000 7f286c36f740 1 Processor -- start 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132bdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28641ae150 con 0x55e7132abb40 2024-09-19T05:09:39.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28641af5c0 con 0x55e7132b5370 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28641ae820 con 0x55e7132aa5d0 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 10 client.0 _ll_register_callbacks cb 0x7f2854009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f2862ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132bdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132bd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:43994/0 (socket says 172.21.15.64:43994) 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 -- 172.21.15.64:0/2817954944 learned_addr learned my addr 172.21.15.64:0/2817954944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f2863fff640 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 -- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 msgr2=0x55e7132ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 -- 172.21.15.64:0/2817954944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 msgr2=0x55e7132bdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132bdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 -- 172.21.15.64:0/2817954944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7132aef60 con 0x55e7132aa5d0 2024-09-19T05:09:39.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f28637fe640 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132bd5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2858097be0 tx=0x7f28580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5960450c192a7bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f2842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f2842ffd640 1 -- 172.21.15.64:0/2817954944 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858009200 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28540043f0 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.344+0000 7f2842ffd640 1 -- 172.21.15.64:0/2817954944 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f28580ccd20 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f2842ffd640 1 -- 172.21.15.64:0/2817954944 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f285809af20 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 10 client.25702 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7132aef60 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f2842ffd640 1 -- 172.21.15.64:0/2817954944 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28540043f0 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f2842ffd640 1 -- 172.21.15.64:0/2817954944 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2858083060 con 0x55e7132aa5d0 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 10 client.25702 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 20 client.25702 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:39.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e71332f4c0 0x55e7133318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 4 client.25702 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:39.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e713194170 con 0x55e71332f4c0 2024-09-19T05:09:39.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.345+0000 7f2862ffd640 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e71332f4c0 0x55e7133318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f2862ffd640 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e71332f4c0 0x55e7133318e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc73e73768d59ec0 server_cookie=2394656238c71c1f in_seq=0 out_seq=0 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f2842ffd640 10 client.25702 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f2842ffd640 1 -- 172.21.15.64:0/2817954944 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e713194170 con 0x55e71332f4c0 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f2842ffd640 10 client.25702 handle_command_reply: tid=0 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 client.25702 shutdown 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 msgr2=0x55e7132bd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132bd5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2858097be0 tx=0x7f28580046c0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 shutdown_connections 2024-09-19T05:09:39.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e7132b5370 0x55e7132ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e71332f4c0 0x55e7133318e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7132abb40 0x55e7132bdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 --2- 172.21.15.64:0/2817954944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7132aa5d0 0x55e7132bd5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f2842ffd640 0 client.25702 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 >> 172.21.15.64:0/2817954944 conn(0x55e713273310 msgr2=0x55e7132c0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 shutdown_connections 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 1 -- 172.21.15.64:0/2817954944 wait complete. 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.346+0000 7f286c36f740 20 client.25702 trim_cache size 0 max 16384 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f284c0a33b0 msgr2=0x7f284c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f284c0a33b0 0x7f284c0a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f285c005e10 tx=0x7f285c0316e0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 msgr2=0x7f2864198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 0x7f2864198500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f285800b3f0 tx=0x7f285800b8b0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 shutdown_connections 2024-09-19T05:09:39.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f284c0a33b0 0x7f284c0a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28641082a0 0x7f286419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28641036e0 0x7f2864198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 --2- 172.21.15.64:0/851831788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2864102d30 0x7f2864198500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 >> 172.21.15.64:0/851831788 conn(0x7f28640fc7d0 msgr2=0x7f2864105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 shutdown_connections 2024-09-19T05:09:39.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:39.347+0000 7f2869f96640 1 -- 172.21.15.64:0/851831788 wait complete. 2024-09-19T05:09:39.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:39.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:39.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:39.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:39.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:39.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:39.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-19T05:09:39.480 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:39.481 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:39.481 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:39 smithi064 ceph-mon[30960]: pgmap v115: 97 pgs: 97 active+clean; 9.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 170 KiB/s wr, 6 op/s 2024-09-19T05:09:39.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:39 smithi103 ceph-mon[34446]: pgmap v115: 97 pgs: 97 active+clean; 9.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 170 KiB/s wr, 6 op/s 2024-09-19T05:09:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:39 smithi174 ceph-mon[34488]: pgmap v115: 97 pgs: 97 active+clean; 9.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 170 KiB/s wr, 6 op/s 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:39.848 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:39.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:39.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:39.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:39.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:39.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:39.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-19T05:09:40.232 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:40.233 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:40.234 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-19T05:09:40.235 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:40.236 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:40.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-19T05:09:40.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:40.580 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:40.580 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:40.580 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:40.581 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:40.582 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:40.583 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:40.584 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:40.585 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-19T05:09:40.586 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:40.963 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:40.963 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:40.964 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-19T05:09:40.965 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:40.966 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:40.967 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-19T05:09:40.968 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:40.969 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:41.251 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:09:41.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:09:41.333 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:41.333 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-19T05:09:41.333 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:41.333 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:41.333 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:41.334 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:41.335 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-19T05:09:41.336 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-19T05:09:41.337 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:41.338 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:41.339 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:41.535 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:41.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:41 smithi064 ceph-mon[30960]: pgmap v116: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 249 op/s 2024-09-19T05:09:41.690 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-19T05:09:41.690 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:41.690 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:41.690 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:41.691 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:41.692 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-19T05:09:41.693 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:41.694 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:41.695 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:41.696 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-19T05:09:41.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:41 smithi103 ceph-mon[34446]: pgmap v116: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 249 op/s 2024-09-19T05:09:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:41 smithi174 ceph-mon[34488]: pgmap v116: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 249 op/s 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 -- 172.21.15.64:0/3386446146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100100e20 msgr2=0x7f5100101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 --2- 172.21.15.64:0/3386446146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100100e20 0x7f5100101200 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009900 tx=0x7f50f002a480 comp rx=0 tx=0).stop 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 -- 172.21.15.64:0/3386446146 shutdown_connections 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 --2- 172.21.15.64:0/3386446146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5100102ef0 0x7f510010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 --2- 172.21.15.64:0/3386446146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5100102570 0x7f51001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 --2- 172.21.15.64:0/3386446146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100100e20 0x7f5100101200 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 -- 172.21.15.64:0/3386446146 >> 172.21.15.64:0/3386446146 conn(0x7f51000fc7d0 msgr2=0x7f51000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:42.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 -- 172.21.15.64:0/3386446146 shutdown_connections 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.030+0000 7f5105e65640 1 -- 172.21.15.64:0/3386446146 wait complete. 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 Processor -- start 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 -- start start 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5100110e30 con 0x7f5100102570 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5100110cb0 con 0x7f5100100e20 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f5105e65640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5100110fb0 con 0x7f5100102ef0 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f50feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f50feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57818/0 (socket says 172.21.15.64:57818) 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f50feffd640 1 -- 172.21.15.64:0/2003172728 learned_addr learned my addr 172.21.15.64:0/2003172728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:42.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.031+0000 7f50fffff640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50feffd640 1 -- 172.21.15.64:0/2003172728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5100102ef0 msgr2=0x7f510019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50feffd640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50feffd640 1 -- 172.21.15.64:0/2003172728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5100100e20 msgr2=0x7f5100197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50feffd640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50feffd640 1 -- 172.21.15.64:0/2003172728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510019c300 con 0x7f5100102570 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50feffd640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 0x7f5100197890 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009900 tx=0x7f50f4009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50fcff9640 1 -- 172.21.15.64:0/2003172728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f400d030 con 0x7f5100102570 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510019c5f0 con 0x7f5100102570 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50fcff9640 1 -- 172.21.15.64:0/2003172728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f400d1d0 con 0x7f5100102570 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f50fcff9640 1 -- 172.21.15.64:0/2003172728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4005280 con 0x7f5100102570 2024-09-19T05:09:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.032+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5100069f70 con 0x7f5100102570 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f50fcff9640 1 -- 172.21.15.64:0/2003172728 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f50f4005420 con 0x7f5100102570 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 Processor -- start 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 -- start start 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafb7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ddbaf0e280 con 0x55ddbafb8150 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddbaeb5b10 con 0x55ddbafacf70 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.035+0000 7f510823e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddbaf0a240 con 0x55ddbafad890 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42754/0 (socket says 172.21.15.64:42754) 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 -- 172.21.15.64:0/3590483446 learned_addr learned my addr 172.21.15.64:0/3590483446 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafb7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fcff9640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50e80a33b0 0x7f50e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 -- 172.21.15.64:0/3590483446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 msgr2=0x55ddbafb7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafb7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 -- 172.21.15.64:0/3590483446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 msgr2=0x55ddbafba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 -- 172.21.15.64:0/3590483446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbaec21f0 con 0x55ddbafacf70 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50e80a33b0 0x7f50e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fcff9640 1 -- 172.21.15.64:0/2003172728 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f50f40d0fa0 con 0x7f5100102570 2024-09-19T05:09:42.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50fffff640 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafad350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f50f8011590 tx=0x7f50f8011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ecad5a4921e060 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50e5ffb640 1 -- 172.21.15.64:0/3590483446 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8017050 con 0x55ddbafacf70 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50e5ffb640 1 -- 172.21.15.64:0/3590483446 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f801bb90 con 0x55ddbafacf70 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.036+0000 7f50e5ffb640 1 -- 172.21.15.64:0/3590483446 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f800e3b0 con 0x55ddbafacf70 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 -- 172.21.15.64:0/3590483446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 msgr2=0x55ddbafad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f50fffff640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50e80a33b0 0x7f50e80a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50f8022000 tx=0x7f50f8023000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafad350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f50f8011590 tx=0x7f50f8011a50 comp rx=0 tx=0).stop 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 -- 172.21.15.64:0/3590483446 shutdown_connections 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafb7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 --2- 172.21.15.64:0/3590483446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafad350 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 -- 172.21.15.64:0/3590483446 >> 172.21.15.64:0/3590483446 conn(0x55ddbafa9500 msgr2=0x55ddbafa98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 -- 172.21.15.64:0/3590483446 shutdown_connections 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.037+0000 7f510823e740 1 -- 172.21.15.64:0/3590483446 wait complete. 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 -- start start 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 Processor -- start 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafacf70 0x55ddbafb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 0x55ddbafb5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafb5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ddbafbad20 con 0x55ddbafad890 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddbaec8f10 con 0x55ddbafb8150 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddbafbaea0 con 0x55ddbafacf70 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f510823e740 10 client.0 _ll_register_callbacks cb 0x55ddbae91a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f50feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 0x55ddbafb5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f50feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 0x55ddbafb5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57836/0 (socket says 172.21.15.64:57836) 2024-09-19T05:09:42.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f50fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafb5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f50feffd640 1 -- 172.21.15.64:0/3258014567 learned_addr learned my addr 172.21.15.64:0/3258014567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.038+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafacf70 0x55ddbafb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50feffd640 1 -- 172.21.15.64:0/3258014567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafacf70 msgr2=0x55ddbafb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50feffd640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafacf70 0x55ddbafb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50feffd640 1 -- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 msgr2=0x55ddbafb5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50feffd640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafb5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50feffd640 1 -- 172.21.15.64:0/3258014567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbac7f6a0 con 0x55ddbafad890 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50fffff640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafb5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50feffd640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 0x55ddbafb5570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f50f4099d00 tx=0x7f50f4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e457cf02f3052a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50deffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50deffd640 1 -- 172.21.15.64:0/3258014567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f401e070 con 0x55ddbafad890 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50deffd640 1 -- 172.21.15.64:0/3258014567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f50f40bd0c0 con 0x55ddbafad890 2024-09-19T05:09:42.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f801bb90 con 0x55ddbafad890 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50deffd640 1 -- 172.21.15.64:0/3258014567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50f409ce70 con 0x55ddbafad890 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f510823e740 10 client.16335 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ddbafc6920 con 0x55ddbafad890 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.039+0000 7f50deffd640 1 -- 172.21.15.64:0/3258014567 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f801bb90 con 0x55ddbafad890 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f50deffd640 1 -- 172.21.15.64:0/3258014567 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50f40a1140 con 0x55ddbafad890 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f510823e740 10 client.16335 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f510823e740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f510823e740 20 client.16335 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f510823e740 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbafc7d30 0x55ddbafca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f510823e740 4 client.16335 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddbaf1b7f0 con 0x55ddbafc7d30 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbafc7d30 0x55ddbafca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbafc7d30 0x55ddbafca140 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167c5a97a24e0f02 server_cookie=518fcf27aa94722d in_seq=0 out_seq=0 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.040+0000 7f50deffd640 10 client.16335 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.041+0000 7f50deffd640 1 -- 172.21.15.64:0/3258014567 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddbaf1b7f0 con 0x55ddbafc7d30 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.041+0000 7f50deffd640 10 client.16335 handle_command_reply: tid=0 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.041+0000 7f510823e740 1 client.16335 shutdown 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.041+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 msgr2=0x55ddbafb5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.041+0000 7f510823e740 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 0x55ddbafb5570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f50f4099d00 tx=0x7f50f4005d30 comp rx=0 tx=0).stop 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 shutdown_connections 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafb5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbafc7d30 0x55ddbafca140 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafad890 0x55ddbafb5570 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 --2- 172.21.15.64:0/3258014567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafacf70 0x55ddbafb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 >> 172.21.15.64:0/3258014567 conn(0x55ddbafa9500 msgr2=0x55ddbafa4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:42.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f50deffd640 0 client.16335 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:42.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 shutdown_connections 2024-09-19T05:09:42.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 1 -- 172.21.15.64:0/3258014567 wait complete. 2024-09-19T05:09:42.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.042+0000 7f510823e740 20 client.16335 trim_cache size 0 max 16384 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 Processor -- start 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 -- start start 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafaf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafcfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafcff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51001aded0 con 0x55ddbafb8150 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51001ab7a0 con 0x55ddbafacf70 2024-09-19T05:09:42.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f510823e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51001a95f0 con 0x55ddbafad890 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafcff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50feffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafcfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafaf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50feffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafcfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:44020/0 (socket says 172.21.15.64:44020) 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafcff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57840/0 (socket says 172.21.15.64:57840) 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafaf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42762/0 (socket says 172.21.15.64:42762) 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.060+0000 7f50feffd640 1 -- 172.21.15.64:0/1174421883 learned_addr learned my addr 172.21.15.64:0/1174421883 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50fffff640 1 -- 172.21.15.64:0/1174421883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 msgr2=0x55ddbafcfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50fffff640 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafcfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50fffff640 1 -- 172.21.15.64:0/1174421883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 msgr2=0x55ddbafaf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50fffff640 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafaf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50fffff640 1 -- 172.21.15.64:0/1174421883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbafb7900 con 0x55ddbafb8150 2024-09-19T05:09:42.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafaf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:42.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50fffff640 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafcff50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000f80 tx=0x7f50f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7207fc490c6a0550 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:42.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50e57fa640 1 -- 172.21.15.64:0/1174421883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8035070 con 0x55ddbafb8150 2024-09-19T05:09:42.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50e57fa640 1 -- 172.21.15.64:0/1174421883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f800f7a0 con 0x55ddbafb8150 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.061+0000 7f50e57fa640 1 -- 172.21.15.64:0/1174421883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f802a040 con 0x55ddbafb8150 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 -- 172.21.15.64:0/1174421883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 msgr2=0x55ddbafcff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafcff50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000f80 tx=0x7f50f8014000 comp rx=0 tx=0).stop 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 -- 172.21.15.64:0/1174421883 shutdown_connections 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafb8150 0x55ddbafcff50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafcfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 --2- 172.21.15.64:0/1174421883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafacf70 0x55ddbafaf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 -- 172.21.15.64:0/1174421883 >> 172.21.15.64:0/1174421883 conn(0x55ddbafa9500 msgr2=0x55ddbafa4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 -- 172.21.15.64:0/1174421883 shutdown_connections 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 -- 172.21.15.64:0/1174421883 wait complete. 2024-09-19T05:09:42.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.062+0000 7f510823e740 1 -- start start 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 Processor -- start 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 0x55ddbafc09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafbbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafbc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f51001a9c60 con 0x55ddbafacf70 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51001ab0d0 con 0x55ddbafb8150 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51001aa330 con 0x55ddbafad890 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f510823e740 10 client.0 _ll_register_callbacks cb 0x7f50f4009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 0x55ddbafc09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 0x55ddbafc09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57842/0 (socket says 172.21.15.64:57842) 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50feffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafbbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafbc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 -- 172.21.15.64:0/3077209136 learned_addr learned my addr 172.21.15.64:0/3077209136 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 -- 172.21.15.64:0/3077209136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 msgr2=0x55ddbafbbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafbbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 -- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 msgr2=0x55ddbafbc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafbc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.063+0000 7f50ff7fe640 1 -- 172.21.15.64:0/3077209136 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbb0319e0 con 0x55ddbafacf70 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50ff7fe640 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 0x55ddbafc09c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009900 tx=0x7f50f0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bbe305237c8bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50deffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50deffd640 1 -- 172.21.15.64:0/3077209136 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0049070 con 0x55ddbafacf70 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50deffd640 1 -- 172.21.15.64:0/3077209136 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f50f0031840 con 0x55ddbafacf70 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddbaf15d00 con 0x55ddbafacf70 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50deffd640 1 -- 172.21.15.64:0/3077209136 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50f0035040 con 0x55ddbafacf70 2024-09-19T05:09:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f510823e740 10 client.16341 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ddbb032c10 con 0x55ddbafacf70 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50deffd640 1 -- 172.21.15.64:0/3077209136 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ddbaf15d00 con 0x55ddbafacf70 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f50deffd640 1 -- 172.21.15.64:0/3077209136 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50f003c600 con 0x55ddbafacf70 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f510823e740 10 client.16341 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f510823e740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.064+0000 7f510823e740 20 client.16341 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.065+0000 7f510823e740 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbb033310 0x55ddbb035730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.065+0000 7f510823e740 4 client.16341 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.065+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ddbae92110 con 0x55ddbb033310 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.065+0000 7f50feffd640 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbb033310 0x55ddbb035730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.065+0000 7f50feffd640 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbb033310 0x55ddbb035730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e6a9c2bf8ea02 server_cookie=e661e143ae946676 in_seq=0 out_seq=0 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.065+0000 7f50deffd640 10 client.16341 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:42.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f50deffd640 1 -- 172.21.15.64:0/3077209136 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ddbae92110 con 0x55ddbb033310 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f50deffd640 10 client.16341 handle_command_reply: tid=0 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 client.16341 shutdown 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 msgr2=0x55ddbafc09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 0x55ddbafc09c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009900 tx=0x7f50f0004390 comp rx=0 tx=0).stop 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 shutdown_connections 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddbafb8150 0x55ddbafbc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddbafad890 0x55ddbafbbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddbb033310 0x55ddbb035730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 --2- 172.21.15.64:0/3077209136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddbafacf70 0x55ddbafc09c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f50deffd640 0 client.16341 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 >> 172.21.15.64:0/3077209136 conn(0x55ddbafa9500 msgr2=0x55ddbafb4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 shutdown_connections 2024-09-19T05:09:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 1 -- 172.21.15.64:0/3077209136 wait complete. 2024-09-19T05:09:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.066+0000 7f510823e740 20 client.16341 trim_cache size 0 max 16384 2024-09-19T05:09:42.077 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:09:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "38c66e28-9cc2-4b2d-b613-8136925cd509", 2024-09-19T05:09:42.077 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:09:42.077 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50e80a33b0 msgr2=0x7f50e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50e80a33b0 0x7f50e80a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50f8022000 tx=0x7f50f8023000 comp rx=0 tx=0).stop 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 msgr2=0x7f5100197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 0x7f5100197890 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009900 tx=0x7f50f4009dc0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 shutdown_connections 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50e80a33b0 0x7f50e80a5870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5100102ef0 0x7f510019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5100102570 0x7f5100197890 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 --2- 172.21.15.64:0/2003172728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5100100e20 0x7f5100197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:42.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 >> 172.21.15.64:0/2003172728 conn(0x7f51000fc7d0 msgr2=0x7f51000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:42.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 shutdown_connections 2024-09-19T05:09:42.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:42.067+0000 7f5105e65640 1 -- 172.21.15.64:0/2003172728 wait complete. 2024-09-19T05:09:42.079 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:42.079 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:42.079 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:42.079 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-19T05:09:42.079 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:42.079 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:42.080 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:42.081 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:42.082 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-19T05:09:42.083 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-19T05:09:42.084 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:42.085 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-19T05:09:42.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:42.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:42.504 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:42.505 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-19T05:09:42.506 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:42.507 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:42.508 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:42.509 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:42.510 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-19T05:09:42.511 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:42.836 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:42.837 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-19T05:09:42.838 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:42.839 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:42.840 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-19T05:09:42.841 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:42.842 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:42.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:43.237 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:43.238 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:43.239 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:43.240 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-19T05:09:43.241 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:43.242 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-19T05:09:43.243 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:43.244 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:43.244 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:43.250 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:43.640 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-19T05:09:43.640 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:43.640 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:43.640 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:43.640 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-19T05:09:43.640 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:43.641 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-19T05:09:43.642 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:43.643 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:43.644 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:43.645 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:43.646 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-19T05:09:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:43 smithi064 ceph-mon[30960]: pgmap v117: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 248 op/s 2024-09-19T05:09:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:43 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:09:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:43 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:43 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:09:43.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:43 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:43.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:43 smithi103 ceph-mon[34446]: pgmap v117: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 248 op/s 2024-09-19T05:09:43.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:43 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:09:43.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:43 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:43.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:43 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:09:43.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:43 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:43 smithi174 ceph-mon[34488]: pgmap v117: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 248 op/s 2024-09-19T05:09:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:43 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:09:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:43 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:43 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:09:43.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:43 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 -- 172.21.15.64:0/977562262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59981027c0 msgr2=0x7f5998106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 --2- 172.21.15.64:0/977562262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59981027c0 0x7f5998106810 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f598c009900 tx=0x7f598c02a410 comp rx=0 tx=0).stop 2024-09-19T05:09:43.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 -- 172.21.15.64:0/977562262 shutdown_connections 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 --2- 172.21.15.64:0/977562262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998106f40 0x7f599810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 --2- 172.21.15.64:0/977562262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59981027c0 0x7f5998106810 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 --2- 172.21.15.64:0/977562262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5998101e10 0x7f59981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 -- 172.21.15.64:0/977562262 >> 172.21.15.64:0/977562262 conn(0x7f59980fd8d0 msgr2=0x7f59980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 -- 172.21.15.64:0/977562262 shutdown_connections 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.770+0000 7f599e8dc640 1 -- 172.21.15.64:0/977562262 wait complete. 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 Processor -- start 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 -- start start 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 0x7f59981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 0x7f59981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5998106f40 0x7f599819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5998111e50 con 0x7f5998106f40 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5998111cd0 con 0x7f5998101e10 2024-09-19T05:09:43.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599e8dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5998111fd0 con 0x7f59981027c0 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f5997fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 0x7f59981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f59977fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 0x7f59981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f599ce79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5998106f40 0x7f599819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f5997fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 0x7f59981983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42784/0 (socket says 172.21.15.64:42784) 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f59977fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 0x7f59981988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:44050/0 (socket says 172.21.15.64:44050) 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.771+0000 7f5997fff640 1 -- 172.21.15.64:0/2552520905 learned_addr learned my addr 172.21.15.64:0/2552520905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f5997fff640 1 -- 172.21.15.64:0/2552520905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 msgr2=0x7f59981988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f5997fff640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 0x7f59981988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f5997fff640 1 -- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5998106f40 msgr2=0x7f599819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f5997fff640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5998106f40 0x7f599819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f5997fff640 1 -- 172.21.15.64:0/2552520905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599819d360 con 0x7f5998101e10 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f59977fe640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 0x7f59981988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f5997fff640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 0x7f59981983b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f598800e9f0 tx=0x7f598800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f59957fa640 1 -- 172.21.15.64:0/2552520905 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598800cd80 con 0x7f5998101e10 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f59957fa640 1 -- 172.21.15.64:0/2552520905 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59880054d0 con 0x7f5998101e10 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599819d650 con 0x7f5998101e10 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f59957fa640 1 -- 172.21.15.64:0/2552520905 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988010490 con 0x7f5998101e10 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.772+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59981a4f30 con 0x7f5998101e10 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59957fa640 1 -- 172.21.15.64:0/2552520905 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5988005020 con 0x7f5998101e10 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 Processor -- start 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 -- start start 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 0x55f842840880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f84284b680 0x55f84284da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f8427a62e0 con 0x55f8428404a0 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f84274dce0 con 0x55f84284b680 2024-09-19T05:09:43.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f8427a22a0 con 0x55f842840dc0 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f599ce79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 0x55f842840880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f59957fa640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59800a3430 0x7f59800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f599ce79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 0x55f842840880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57866/0 (socket says 172.21.15.64:57866) 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.775+0000 7f599ce79640 1 -- 172.21.15.64:0/4045625277 learned_addr learned my addr 172.21.15.64:0/4045625277 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f5997fff640 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f84284b680 0x55f84284da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f599ce79640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59800a3430 0x7f59800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 -- 172.21.15.64:0/4045625277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 msgr2=0x55f84284b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 -- 172.21.15.64:0/4045625277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 msgr2=0x55f842840880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 0x55f842840880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 -- 172.21.15.64:0/4045625277 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f842732440 con 0x55f84284b680 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f599ce79640 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 0x55f842840880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:43.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59957fa640 1 -- 172.21.15.64:0/2552520905 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f59880d0080 con 0x7f5998101e10 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f59977fe640 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f84284b680 0x55f84284da70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f598c009230 tx=0x7f598c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a04db05617d7935 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f599ce79640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59800a3430 0x7f59800a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59900049d0 tx=0x7f5990004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f597e7fc640 1 -- 172.21.15.64:0/4045625277 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c040070 con 0x55f84284b680 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f597e7fc640 1 -- 172.21.15.64:0/4045625277 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f598c031860 con 0x55f84284b680 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.776+0000 7f597e7fc640 1 -- 172.21.15.64:0/4045625277 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c035040 con 0x55f84284b680 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 -- 172.21.15.64:0/4045625277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f84284b680 msgr2=0x55f84284da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f84284b680 0x55f84284da70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f598c009230 tx=0x7f598c005d90 comp rx=0 tx=0).stop 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 -- 172.21.15.64:0/4045625277 shutdown_connections 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f84284b680 0x55f84284da70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/4045625277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8428404a0 0x55f842840880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 -- 172.21.15.64:0/4045625277 >> 172.21.15.64:0/4045625277 conn(0x55f84283ca30 msgr2=0x55f84283ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 -- 172.21.15.64:0/4045625277 shutdown_connections 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 -- 172.21.15.64:0/4045625277 wait complete. 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.777+0000 7f59a0cb5740 1 -- start start 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 Processor -- start 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f84284af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 0x55f8428481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84284b680 0x55f842848720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f84284e250 con 0x55f842840dc0 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f842761050 con 0x55f8428404a0 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f84284e3d0 con 0x55f84284b680 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59a0cb5740 10 client.0 _ll_register_callbacks cb 0x55f8427a1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 0x55f8428481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 0x55f8428481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57882/0 (socket says 172.21.15.64:57882) 2024-09-19T05:09:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59977fe640 1 -- 172.21.15.64:0/1853521004 learned_addr learned my addr 172.21.15.64:0/1853521004 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f5997fff640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f84284af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f599ce79640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84284b680 0x55f842848720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f5997fff640 1 -- 172.21.15.64:0/1853521004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84284b680 msgr2=0x55f842848720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f5997fff640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84284b680 0x55f842848720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f5997fff640 1 -- 172.21.15.64:0/1853521004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 msgr2=0x55f8428481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f5997fff640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 0x55f8428481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f5997fff640 1 -- 172.21.15.64:0/1853521004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8425173c0 con 0x55f8428404a0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f599ce79640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84284b680 0x55f842848720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.778+0000 7f59977fe640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 0x55f8428481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f5997fff640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f84284af30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f59880cd770 tx=0x7f598809bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12395fc6a0c5361 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59777fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59777fe640 1 -- 172.21.15.64:0/1853521004 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598809ca00 con 0x55f8428404a0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59777fe640 1 -- 172.21.15.64:0/1853521004 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f59880b5260 con 0x55f8428404a0 2024-09-19T05:09:43.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c031860 con 0x55f8428404a0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59777fe640 1 -- 172.21.15.64:0/1853521004 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f59880a4f70 con 0x55f8428404a0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59a0cb5740 10 client.25982 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f842859c00 con 0x55f8428404a0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.779+0000 7f59777fe640 1 -- 172.21.15.64:0/1853521004 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f598c031860 con 0x55f8428404a0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59777fe640 1 -- 172.21.15.64:0/1853521004 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f59880a4b70 con 0x55f8428404a0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59a0cb5740 10 client.25982 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59a0cb5740 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59a0cb5740 20 client.25982 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f84285b0c0 0x55f84285d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59a0cb5740 4 client.25982 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8427b3850 con 0x55f84285b0c0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59977fe640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f84285b0c0 0x55f84285d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59977fe640 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f84285b0c0 0x55f84285d4f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2bf3d0d078917e4 server_cookie=659802e0276b2534 in_seq=0 out_seq=0 2024-09-19T05:09:43.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.780+0000 7f59777fe640 10 client.25982 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59777fe640 1 -- 172.21.15.64:0/1853521004 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8427b3850 con 0x55f84285b0c0 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59777fe640 10 client.25982 handle_command_reply: tid=0 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 client.25982 shutdown 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 msgr2=0x55f84284af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f84284af30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f59880cd770 tx=0x7f598809bce0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 shutdown_connections 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84284b680 0x55f842848720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f84285b0c0 0x55f84285d4f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f842840dc0 0x55f8428481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/1853521004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f84284af30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 >> 172.21.15.64:0/1853521004 conn(0x55f84283ca30 msgr2=0x55f842842760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59777fe640 0 client.25982 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 shutdown_connections 2024-09-19T05:09:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 1 -- 172.21.15.64:0/1853521004 wait complete. 2024-09-19T05:09:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.781+0000 7f59a0cb5740 20 client.25982 trim_cache size 0 max 16384 2024-09-19T05:09:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 Processor -- start 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 -- start start 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8428404a0 0x55f8428574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 0x55f84283ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f8428437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59981b0160 con 0x55f84284b680 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ada30 con 0x55f842840dc0 2024-09-19T05:09:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.801+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ab880 con 0x55f8428404a0 2024-09-19T05:09:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f599ce79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f8428437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 0x55f84283ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 0x55f84283ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42808/0 (socket says 172.21.15.64:42808) 2024-09-19T05:09:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 -- 172.21.15.64:0/190270578 learned_addr learned my addr 172.21.15.64:0/190270578 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f5997fff640 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8428404a0 0x55f8428574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 -- 172.21.15.64:0/190270578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8428404a0 msgr2=0x55f8428574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8428404a0 0x55f8428574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 -- 172.21.15.64:0/190270578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 msgr2=0x55f8428437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f8428437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 -- 172.21.15.64:0/190270578 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84284b2a0 con 0x55f842840dc0 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f59977fe640 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 0x55f84283ce10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f598c033000 tx=0x7f598c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ff4cacfe0a64f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f597dffb640 1 -- 172.21.15.64:0/190270578 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c040040 con 0x55f842840dc0 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f597dffb640 1 -- 172.21.15.64:0/190270578 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f598c0035f0 con 0x55f842840dc0 2024-09-19T05:09:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.802+0000 7f597dffb640 1 -- 172.21.15.64:0/190270578 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c035040 con 0x55f842840dc0 2024-09-19T05:09:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 -- 172.21.15.64:0/190270578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 msgr2=0x55f84283ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 0x55f84283ce10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f598c033000 tx=0x7f598c005e80 comp rx=0 tx=0).stop 2024-09-19T05:09:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 -- 172.21.15.64:0/190270578 shutdown_connections 2024-09-19T05:09:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f8428437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f842840dc0 0x55f84283ce10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/190270578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8428404a0 0x55f8428574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 -- 172.21.15.64:0/190270578 >> 172.21.15.64:0/190270578 conn(0x55f84283ca30 msgr2=0x55f8428430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 -- 172.21.15.64:0/190270578 shutdown_connections 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.803+0000 7f59a0cb5740 1 -- 172.21.15.64:0/190270578 wait complete. 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 -- start start 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 Processor -- start 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f842853a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f84284f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59981abef0 con 0x55f84284b680 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ad360 con 0x55f8428404a0 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ac5c0 con 0x55f842840dc0 2024-09-19T05:09:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f59a0cb5740 10 client.0 _ll_register_callbacks cb 0x7f598809be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f599ce79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f84284f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f5997fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f842853a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.804+0000 7f599ce79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f84284f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57900/0 (socket says 172.21.15.64:57900) 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f599ce79640 1 -- 172.21.15.64:0/3421966453 learned_addr learned my addr 172.21.15.64:0/3421966453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f59977fe640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f5997fff640 1 -- 172.21.15.64:0/3421966453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 msgr2=0x55f84284ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f5997fff640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f5997fff640 1 -- 172.21.15.64:0/3421966453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 msgr2=0x55f84284f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f5997fff640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f84284f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f5997fff640 1 -- 172.21.15.64:0/3421966453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f84284f800 con 0x55f8428404a0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f59977fe640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f5997fff640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f842853a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f598809c290 tx=0x7f5988004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa84d5df43f2cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f59777fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:09:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.805+0000 7f59777fe640 1 -- 172.21.15.64:0/3421966453 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598809c0d0 con 0x55f8428404a0 2024-09-19T05:09:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59777fe640 1 -- 172.21.15.64:0/3421966453 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f59880a0180 con 0x55f8428404a0 2024-09-19T05:09:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c0035f0 con 0x55f8428404a0 2024-09-19T05:09:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59777fe640 1 -- 172.21.15.64:0/3421966453 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f59880cdc00 con 0x55f8428404a0 2024-09-19T05:09:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 10 client.25988 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8428c5a50 con 0x55f8428404a0 2024-09-19T05:09:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59777fe640 1 -- 172.21.15.64:0/3421966453 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f598809ca00 con 0x55f8428404a0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59777fe640 1 -- 172.21.15.64:0/3421966453 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f59880cd400 con 0x55f8428404a0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 10 client.25988 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 20 client.25988 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f8428c5fd0 0x55f8428c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 4 client.25988 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.806+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f598c0035f0 con 0x55f8428c5fd0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59977fe640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f8428c5fd0 0x55f8428c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59977fe640 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f8428c5fd0 0x55f8428c83c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43dbd44e9b30d665 server_cookie=6ebc17e242994728 in_seq=0 out_seq=0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59777fe640 10 client.25988 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59777fe640 1 -- 172.21.15.64:0/3421966453 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f598c0035f0 con 0x55f8428c5fd0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59777fe640 10 client.25988 handle_command_reply: tid=0 2024-09-19T05:09:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59a0cb5740 1 client.25988 shutdown 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 msgr2=0x55f842853a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.807+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f842853a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f598809c290 tx=0x7f5988004aa0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 shutdown_connections 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55f8428c5fd0 0x55f8428c83c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f84284b680 0x55f84284f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f842840dc0 0x55f84284ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59777fe640 0 client.25988 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 --2- 172.21.15.64:0/3421966453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8428404a0 0x55f842853a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 >> 172.21.15.64:0/3421966453 conn(0x55f84283ca30 msgr2=0x55f84283bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 shutdown_connections 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 1 -- 172.21.15.64:0/3421966453 wait complete. 2024-09-19T05:09:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f59a0cb5740 20 client.25988 trim_cache size 0 max 16384 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59800a3430 msgr2=0x7f59800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f599e8dc640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59800a3430 0x7f59800a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59900049d0 tx=0x7f5990004ab0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 msgr2=0x7f59981983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.808+0000 7f599e8dc640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 0x7f59981983b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f598800e9f0 tx=0x7f598800edf0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 shutdown_connections 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f59800a3430 0x7f59800a58f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5998106f40 0x7f599819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59981027c0 0x7f59981988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 --2- 172.21.15.64:0/2552520905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5998101e10 0x7f59981983b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 >> 172.21.15.64:0/2552520905 conn(0x7f59980fd8d0 msgr2=0x7f59980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 shutdown_connections 2024-09-19T05:09:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:43.809+0000 7f599e8dc640 1 -- 172.21.15.64:0/2552520905 wait complete. 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:44.009 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:44.010 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-19T05:09:44.011 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:44.012 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-19T05:09:44.013 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:44.014 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:44.473 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:44.474 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-19T05:09:44.475 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:44.476 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-19T05:09:44.477 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:44.478 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-19T05:09:44.479 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:44.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:44 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:09:44.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:44 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:09:44.738 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:09:44.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:09:44.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:44 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:09:44.849 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:44.850 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:44.851 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:44.852 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:44.853 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-19T05:09:44.854 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:44.855 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:45.061 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-19T05:09:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:45.267 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-19T05:09:45.268 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:45.269 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:45.270 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-19T05:09:45.271 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-19T05:09:45.272 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:45.273 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:45.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:45 smithi064 ceph-mon[30960]: pgmap v118: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 246 op/s 2024-09-19T05:09:45.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 -- 172.21.15.64:0/304324292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08106f10 msgr2=0x7f7a0810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 --2- 172.21.15.64:0/304324292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08106f10 0x7f7a0810ea90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009ca0 tx=0x7f7a0002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 -- 172.21.15.64:0/304324292 shutdown_connections 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 --2- 172.21.15.64:0/304324292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08106f10 0x7f7a0810ea90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 --2- 172.21.15.64:0/304324292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a08102790 0x7f7a081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 --2- 172.21.15.64:0/304324292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a08101de0 0x7f7a081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.577+0000 7f7a0d415640 1 -- 172.21.15.64:0/304324292 >> 172.21.15.64:0/304324292 conn(0x7f7a080fd8c0 msgr2=0x7f7a080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.578+0000 7f7a0d415640 1 -- 172.21.15.64:0/304324292 shutdown_connections 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.578+0000 7f7a0d415640 1 -- 172.21.15.64:0/304324292 wait complete. 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.578+0000 7f7a0d415640 1 Processor -- start 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.578+0000 7f7a0d415640 1 -- start start 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a0d415640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 0x7f7a08198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a0d415640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a08102790 0x7f7a081988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a0d415640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a08106f10 0x7f7a0819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a0d415640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08111e20 con 0x7f7a08101de0 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a0d415640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08111ca0 con 0x7f7a08102790 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a0d415640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08111fa0 con 0x7f7a08106f10 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 0x7f7a08198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 0x7f7a08198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52274/0 (socket says 172.21.15.64:52274) 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 -- 172.21.15.64:0/1704000528 learned_addr learned my addr 172.21.15.64:0/1704000528 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a077fe640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a08106f10 0x7f7a0819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a067fc640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a08102790 0x7f7a081988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 -- 172.21.15.64:0/1704000528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a08106f10 msgr2=0x7f7a0819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a08106f10 0x7f7a0819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 -- 172.21.15.64:0/1704000528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a08102790 msgr2=0x7f7a081988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a08102790 0x7f7a081988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 -- 172.21.15.64:0/1704000528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0819d310 con 0x7f7a08101de0 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.579+0000 7f7a06ffd640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 0x7f7a08198360 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f79f800e9f0 tx=0x7f79f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.580+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f800cd80 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.580+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0819d600 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.580+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f80054d0 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.580+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8010490 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.580+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a081a4ee0 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.581+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a081021c0 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.582+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f79f8005020 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.583+0000 7f79effff640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79f00a33b0 0x7f79f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.583+0000 7f7a067fc640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79f00a33b0 0x7f79f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.583+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f79f80d0080 con 0x7f7a08101de0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.584+0000 7f7a067fc640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79f00a33b0 0x7f79f00a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a080fdd70 tx=0x7f79fc005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.588+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79f809ca60 con 0x7f7a08101de0 2024-09-19T05:09:45.662 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-19T05:09:45.662 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:45.662 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:45.663 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-19T05:09:45.664 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:45.665 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:45.666 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-19T05:09:45.667 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:45.668 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-19T05:09:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:45 smithi103 ceph-mon[34446]: pgmap v118: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 246 op/s 2024-09-19T05:09:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:45 smithi174 ceph-mon[34488]: pgmap v118: 97 pgs: 97 active+clean; 311 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 246 op/s 2024-09-19T05:09:45.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.837+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a0810a150 con 0x7f7a08101de0 2024-09-19T05:09:45.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.838+0000 7f79effff640 1 -- 172.21.15.64:0/1704000528 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f79f809c360 con 0x7f7a08101de0 2024-09-19T05:09:45.839 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:09:45.840 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:09:45.840 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79f00a33b0 msgr2=0x7f79f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79f00a33b0 0x7f79f00a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a080fdd70 tx=0x7f79fc005f90 comp rx=0 tx=0).stop 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 msgr2=0x7f7a08198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 0x7f7a08198360 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f79f800e9f0 tx=0x7f79f800edf0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 shutdown_connections 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79f00a33b0 0x7f79f00a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a08106f10 0x7f7a0819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a08102790 0x7f7a081988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 --2- 172.21.15.64:0/1704000528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a08101de0 0x7f7a08198360 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 >> 172.21.15.64:0/1704000528 conn(0x7f7a080fd8c0 msgr2=0x7f7a080fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 shutdown_connections 2024-09-19T05:09:45.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:45.840+0000 7f7a0d415640 1 -- 172.21.15.64:0/1704000528 wait complete. 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:46.023 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-19T05:09:46.024 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:46.025 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:46.026 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:46.027 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:46.028 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-19T05:09:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:46.029 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-19T05:09:46.421 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:46.421 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:46.422 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:46.423 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-19T05:09:46.424 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:46.425 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-19T05:09:46.426 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-19T05:09:46.427 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:46.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:46 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1704000528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:46.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:09:46.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:46 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1704000528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:46.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:46 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1704000528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:46.813 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:46.813 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:46.814 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:46.815 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:46.816 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-19T05:09:46.817 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:46.818 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:46.819 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:46.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:46.880 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:46.881 INFO:tasks.workunit.client.0.smithi064.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-19T05:09:46.882 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:09:46.882 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:09:46.917 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-09-19T05:09:46.917 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T05:09:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:47.287 DEBUG:teuthology.parallel:result is None 2024-09-19T05:09:47.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:09:47.322 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:09:47.322 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T05:09:47.336 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:09:47.371 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T05:09:47.383 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T05:09:47.383 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:09:47.383 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:09:47.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T05:09:47.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:47 smithi064 ceph-mon[30960]: pgmap v119: 97 pgs: 97 active+clean; 386 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 470 op/s 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 -- 172.21.15.64:0/1092351158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 msgr2=0x7f1c001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 --2- 172.21.15.64:0/1092351158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c001029b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009900 tx=0x7f1be802a480 comp rx=0 tx=0).stop 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 -- 172.21.15.64:0/1092351158 shutdown_connections 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 --2- 172.21.15.64:0/1092351158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c00102ef0 0x7f1c0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 --2- 172.21.15.64:0/1092351158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c001029b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 --2- 172.21.15.64:0/1092351158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c00100e20 0x7f1c00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 -- 172.21.15.64:0/1092351158 >> 172.21.15.64:0/1092351158 conn(0x7f1c000fc7d0 msgr2=0x7f1c000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 -- 172.21.15.64:0/1092351158 shutdown_connections 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.428+0000 7f1c06545640 1 -- 172.21.15.64:0/1092351158 wait complete. 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 Processor -- start 2024-09-19T05:09:47.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 -- start start 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c00100e20 0x7f1c00198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c00198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c00102ef0 0x7f1c0019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00110e30 con 0x7f1c00102570 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00110cb0 con 0x7f1c00102ef0 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.429+0000 7f1c06545640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00110fb0 con 0x7f1c00100e20 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c00198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c00198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52294/0 (socket says 172.21.15.64:52294) 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 -- 172.21.15.64:0/337317537 learned_addr learned my addr 172.21.15.64:0/337317537 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bfffff640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c00100e20 0x7f1c00198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c00102ef0 0x7f1c0019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 -- 172.21.15.64:0/337317537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c00100e20 msgr2=0x7f1c00198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c00100e20 0x7f1c00198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 -- 172.21.15.64:0/337317537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c00102ef0 msgr2=0x7f1c0019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c00102ef0 0x7f1c0019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 -- 172.21.15.64:0/337317537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0019d3d0 con 0x7f1c00102570 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c00198960 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bfdffb640 1 -- 172.21.15.64:0/337317537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be803f030 con 0x7f1c00102570 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bfdffb640 1 -- 172.21.15.64:0/337317537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1be8003aa0 con 0x7f1c00102570 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0019d660 con 0x7f1c00102570 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1bfdffb640 1 -- 172.21.15.64:0/337317537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8035070 con 0x7f1c00102570 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.430+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001a4fa0 con 0x7f1c00102570 2024-09-19T05:09:47.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.433+0000 7f1bfdffb640 1 -- 172.21.15.64:0/337317537 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1be8042030 con 0x7f1c00102570 2024-09-19T05:09:47.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.434+0000 7f1c0891e740 1 Processor -- start 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.434+0000 7f1c0891e740 1 -- start start 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.434+0000 7f1bfdffb640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab67686090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67686a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab67697b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c0891e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ab675ec2f0 con 0x55ab67687b40 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c0891e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ab67593cf0 con 0x55ab676865d0 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c0891e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab675e82b0 con 0x55ab67695780 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab67686090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfdffb640 1 -- 172.21.15.64:0/337317537 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1be80f0320 con 0x7f1c00102570 2024-09-19T05:09:47.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab67686090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52308/0 (socket says 172.21.15.64:52308) 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 -- 172.21.15.64:0/405274055 learned_addr learned my addr 172.21.15.64:0/405274055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab67697b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67686a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 -- 172.21.15.64:0/405274055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 msgr2=0x55ab67697b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab67697b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 -- 172.21.15.64:0/405274055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 msgr2=0x55ab67686a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67686a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1bfffff640 1 -- 172.21.15.64:0/405274055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab675783a0 con 0x55ab67687b40 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.435+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1bec0a3430 0x7f1bec0a58f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009fd0 tx=0x7f1bf8013b40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.436+0000 7f1bfffff640 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab67686090 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005f10 tx=0x7f1bf00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71b07d5c2d69577 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.436+0000 7f1bf67fc640 1 -- 172.21.15.64:0/405274055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0023460 con 0x55ab67687b40 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.436+0000 7f1bf67fc640 1 -- 172.21.15.64:0/405274055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bf0023ab0 con 0x55ab67687b40 2024-09-19T05:09:47.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.436+0000 7f1bf67fc640 1 -- 172.21.15.64:0/405274055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000fb30 con 0x55ab67687b40 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 -- 172.21.15.64:0/405274055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 msgr2=0x55ab67686090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab67686090 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005f10 tx=0x7f1bf00117b0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 -- 172.21.15.64:0/405274055 shutdown_connections 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab67697b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67686a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 --2- 172.21.15.64:0/405274055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab67686090 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 -- 172.21.15.64:0/405274055 >> 172.21.15.64:0/405274055 conn(0x55ab6764f310 msgr2=0x55ab67682c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 -- 172.21.15.64:0/405274055 shutdown_connections 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 -- 172.21.15.64:0/405274055 wait complete. 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.437+0000 7f1c0891e740 1 -- start start 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 Processor -- start 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab6768e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6768f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ab67689550 con 0x55ab67687b40 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ab675a7060 con 0x55ab676865d0 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab676896d0 con 0x55ab67695780 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c0891e740 10 client.0 _ll_register_callbacks cb 0x55ab672fa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab6768e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab6768e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:52298/0 (socket says 172.21.15.64:52298) 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1bfffff640 1 -- 172.21.15.64:0/2898049371 learned_addr learned my addr 172.21.15.64:0/2898049371 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.438+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6768f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bfffff640 1 -- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 msgr2=0x55ab6768f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bfffff640 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6768f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bfffff640 1 -- 172.21.15.64:0/2898049371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 msgr2=0x55ab6768ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bfffff640 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bfffff640 1 -- 172.21.15.64:0/2898049371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6735d3c0 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bfffff640 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab6768e670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005f60 tx=0x7f1bf0009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90f33cf45eaf37f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/2898049371 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0023460 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/2898049371 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1bf0023600 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf0023ab0 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/2898049371 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bf0042050 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1c0891e740 10 client.26006 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.439+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab6769fac0 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/2898049371 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bf0039070 con 0x55ab676865d0 2024-09-19T05:09:47.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/2898049371 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1bf000fa30 con 0x55ab676865d0 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1c0891e740 10 client.26006 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1c0891e740 10 client.26006 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1c0891e740 20 client.26006 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab676a0e60 0x55ab676a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1c0891e740 4 client.26006 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1bf0023ab0 con 0x55ab676a0e60 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.440+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab676a0e60 0x55ab676a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab676a0e60 0x55ab676a3250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e9aed91a05ac340 server_cookie=fa8cc341f49f0a37 in_seq=0 out_seq=0 2024-09-19T05:09:47.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1bdf7fe640 10 client.26006 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:47.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T05:09:47.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/2898049371 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1bf0023ab0 con 0x55ab676a0e60 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1bdf7fe640 10 client.26006 handle_command_reply: tid=0 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1c0891e740 1 client.26006 shutdown 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 msgr2=0x55ab6768e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.441+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab6768e670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005f60 tx=0x7f1bf0009df0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 shutdown_connections 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab676a0e60 0x55ab676a3250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6768f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1bdf7fe640 0 client.26006 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2898049371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab6768e670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 >> 172.21.15.64:0/2898049371 conn(0x55ab6764f310 msgr2=0x55ab67688960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 shutdown_connections 2024-09-19T05:09:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 1 -- 172.21.15.64:0/2898049371 wait complete. 2024-09-19T05:09:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.442+0000 7f1c0891e740 20 client.26006 trim_cache size 0 max 16384 2024-09-19T05:09:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 Processor -- start 2024-09-19T05:09:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 -- start start 2024-09-19T05:09:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67691ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab676a8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001adde0 con 0x55ab67687b40 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001ab6b0 con 0x55ab676865d0 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c0891e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a9520 con 0x55ab67695780 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67691ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1bf7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52318/0 (socket says 172.21.15.64:52318) 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1bf7fff640 1 -- 172.21.15.64:0/2433300565 learned_addr learned my addr 172.21.15.64:0/2433300565 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.455+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab676a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf7fff640 1 -- 172.21.15.64:0/2433300565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 msgr2=0x55ab676a8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab676a8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf7fff640 1 -- 172.21.15.64:0/2433300565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 msgr2=0x55ab67691ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67691ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf7fff640 1 -- 172.21.15.64:0/2433300565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab67683960 con 0x55ab67687b40 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bfffff640 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67691ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ef40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005ab0 tx=0x7f1be8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe539070472bba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf5ffb640 1 -- 172.21.15.64:0/2433300565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8049040 con 0x55ab67687b40 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf5ffb640 1 -- 172.21.15.64:0/2433300565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1be80bc600 con 0x55ab67687b40 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.456+0000 7f1bf5ffb640 1 -- 172.21.15.64:0/2433300565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be80b8040 con 0x55ab67687b40 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 -- 172.21.15.64:0/2433300565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 msgr2=0x55ab6768ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ef40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005ab0 tx=0x7f1be8005c90 comp rx=0 tx=0).stop 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 -- 172.21.15.64:0/2433300565 shutdown_connections 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab676a8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab67687b40 0x55ab6768ef40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 --2- 172.21.15.64:0/2433300565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab676865d0 0x55ab67691ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 -- 172.21.15.64:0/2433300565 >> 172.21.15.64:0/2433300565 conn(0x55ab6764f310 msgr2=0x55ab6768d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 -- 172.21.15.64:0/2433300565 shutdown_connections 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 -- 172.21.15.64:0/2433300565 wait complete. 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.457+0000 7f1c0891e740 1 -- start start 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 Processor -- start 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 0x55ab6769a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab67687b40 0x55ab6769a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6769aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001a9b70 con 0x55ab676865d0 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001aafe0 con 0x55ab67687b40 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001aa240 con 0x55ab67695780 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c0891e740 10 client.0 _ll_register_callbacks cb 0x7f1bf8009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:47.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 0x55ab6769a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1c04ae2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6769aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1bfffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 0x55ab6769a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52322/0 (socket says 172.21.15.64:52322) 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1bfffff640 1 -- 172.21.15.64:0/316464851 learned_addr learned my addr 172.21.15.64:0/316464851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.458+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab67687b40 0x55ab6769a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bfffff640 1 -- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 msgr2=0x55ab6769aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bfffff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6769aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bfffff640 1 -- 172.21.15.64:0/316464851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab67687b40 msgr2=0x55ab6769a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bfffff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab67687b40 0x55ab6769a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bfffff640 1 -- 172.21.15.64:0/316464851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6770a980 con 0x55ab676865d0 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab67687b40 0x55ab6769a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1c04ae2640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6769aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:47.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bfffff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 0x55ab6769a050 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000fe00 tx=0x7f1bf0004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f6b2dcd6d78589 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/316464851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000fba0 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be80bc600 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/316464851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1bf0023460 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/316464851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bf0055050 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1c0891e740 10 client.16383 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.459+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab6770bc80 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/316464851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1be80bc600 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/316464851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1bf003e540 con 0x55ab676865d0 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1c0891e740 10 client.16383 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1c0891e740 10 client.16383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1c0891e740 20 client.16383 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:47.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1c0891e740 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab6770c370 0x55ab6770e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:47.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1c0891e740 4 client.16383 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:47.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab675f9860 con 0x55ab6770c370 2024-09-19T05:09:47.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab6770c370 0x55ab6770e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1bdf7fe640 10 client.16383 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.460+0000 7f1bf7fff640 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab6770c370 0x55ab6770e790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29dfc10a1a769924 server_cookie=59f32ba4efa16f8f in_seq=0 out_seq=0 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1bdf7fe640 1 -- 172.21.15.64:0/316464851 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab675f9860 con 0x55ab6770c370 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1bdf7fe640 10 client.16383 handle_command_reply: tid=0 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 client.16383 shutdown 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 msgr2=0x55ab6769a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 0x55ab6769a050 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000fe00 tx=0x7f1bf0004940 comp rx=0 tx=0).stop 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 shutdown_connections 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ab6770c370 0x55ab6770e790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab67695780 0x55ab6769aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab67687b40 0x55ab6769a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 --2- 172.21.15.64:0/316464851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab676865d0 0x55ab6769a050 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1bdf7fe640 0 client.16383 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:09:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 >> 172.21.15.64:0/316464851 conn(0x55ab6764f310 msgr2=0x55ab676814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 shutdown_connections 2024-09-19T05:09:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 1 -- 172.21.15.64:0/316464851 wait complete. 2024-09-19T05:09:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.461+0000 7f1c0891e740 20 client.16383 trim_cache size 0 max 16384 2024-09-19T05:09:47.468 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1bec0a3430 msgr2=0x7f1bec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1bec0a3430 0x7f1bec0a58f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009fd0 tx=0x7f1bf8013b40 comp rx=0 tx=0).stop 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 msgr2=0x7f1c00198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c00198960 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be8004270 comp rx=0 tx=0).stop 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 shutdown_connections 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c00102ef0 0x7f1c0019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c00102570 0x7f1c00198960 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 --2- 172.21.15.64:0/337317537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c00100e20 0x7f1c00198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:47.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 >> 172.21.15.64:0/337317537 conn(0x7f1c000fc7d0 msgr2=0x7f1c0006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 shutdown_connections 2024-09-19T05:09:47.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:47.462+0000 7f1c06545640 1 -- 172.21.15.64:0/337317537 wait complete. 2024-09-19T05:09:47.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:47 smithi103 ceph-mon[34446]: pgmap v119: 97 pgs: 97 active+clean; 386 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 470 op/s 2024-09-19T05:09:47.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:47 smithi174 ceph-mon[34488]: pgmap v119: 97 pgs: 97 active+clean; 386 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 470 op/s 2024-09-19T05:09:48.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:09:48.564 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:49.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 -- 172.21.15.64:0/4158054753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c101de0 msgr2=0x7f037c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 --2- 172.21.15.64:0/4158054753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c101de0 0x7f037c1021c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c02a410 comp rx=0 tx=0).stop 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 -- 172.21.15.64:0/4158054753 shutdown_connections 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 --2- 172.21.15.64:0/4158054753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f037c106f10 0x7f037c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 --2- 172.21.15.64:0/4158054753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f037c102790 0x7f037c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 --2- 172.21.15.64:0/4158054753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c101de0 0x7f037c1021c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 -- 172.21.15.64:0/4158054753 >> 172.21.15.64:0/4158054753 conn(0x7f037c0fd8c0 msgr2=0x7f037c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 -- 172.21.15.64:0/4158054753 shutdown_connections 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.009+0000 7f03808f0640 1 -- 172.21.15.64:0/4158054753 wait complete. 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.010+0000 7f03808f0640 1 Processor -- start 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.010+0000 7f03808f0640 1 -- start start 2024-09-19T05:09:49.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f03808f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f037c101de0 0x7f037c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f03808f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f037c102790 0x7f037c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f03808f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 0x7f037c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f03808f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f037c111e20 con 0x7f037c106f10 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f03808f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f037c111ca0 con 0x7f037c101de0 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f03808f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f037c111fa0 con 0x7f037c102790 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 0x7f037c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 0x7f037c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52336/0 (socket says 172.21.15.64:52336) 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 -- 172.21.15.64:0/23782258 learned_addr learned my addr 172.21.15.64:0/23782258 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037a59c640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f037c101de0 0x7f037c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f0379d9b640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f037c102790 0x7f037c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 -- 172.21.15.64:0/23782258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f037c102790 msgr2=0x7f037c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f037c102790 0x7f037c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 -- 172.21.15.64:0/23782258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f037c101de0 msgr2=0x7f037c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f037c101de0 0x7f037c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 -- 172.21.15.64:0/23782258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c19d310 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.011+0000 7f037ad9d640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 0x7f037c19cc30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f037400eaa0 tx=0x7f037400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.012+0000 7f036b7fe640 1 -- 172.21.15.64:0/23782258 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037400cdb0 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.012+0000 7f036b7fe640 1 -- 172.21.15.64:0/23782258 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0374005020 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.012+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037c19d600 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.012+0000 7f036b7fe640 1 -- 172.21.15.64:0/23782258 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03740116d0 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.012+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037c1a4ee0 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.014+0000 7f036b7fe640 1 -- 172.21.15.64:0/23782258 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f03740051d0 con 0x7f037c106f10 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.014+0000 7f0382cc9740 1 Processor -- start 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 -- start start 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b07162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b0716f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555b070c82f0 con 0x555b071625d0 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b0706fcf0 con 0x555b07163b40 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0382cc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b070c42b0 con 0x555b0716d370 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f037a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b07162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f037a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b07162a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52350/0 (socket says 172.21.15.64:52350) 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f037a59c640 1 -- 172.21.15.64:0/698761034 learned_addr learned my addr 172.21.15.64:0/698761034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0379d9b640 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f036b7fe640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f03640a33b0 0x7f03640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.015+0000 7f0379d9b640 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b0716f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037a59c640 1 -- 172.21.15.64:0/698761034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 msgr2=0x555b0716f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037a59c640 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b0716f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037a59c640 1 -- 172.21.15.64:0/698761034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 msgr2=0x555b07162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037ad9d640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f03640a33b0 0x7f03640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037a59c640 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037a59c640 1 -- 172.21.15.64:0/698761034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b070543a0 con 0x555b071625d0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f036b7fe640 1 -- 172.21.15.64:0/23782258 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f03740cfee0 con 0x7f037c106f10 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037a59c640 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b07162a10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f6ea46bd42c934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f0368ff9640 1 -- 172.21.15.64:0/698761034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c040070 con 0x555b071625d0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f0368ff9640 1 -- 172.21.15.64:0/698761034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f036c0316a0 con 0x555b071625d0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f037ad9d640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f03640a33b0 0x7f03640a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f037409be60 tx=0x7f037404b040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.016+0000 7f0368ff9640 1 -- 172.21.15.64:0/698761034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c035040 con 0x555b071625d0 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 -- 172.21.15.64:0/698761034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 msgr2=0x555b07162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b07162a10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c032e40 comp rx=0 tx=0).stop 2024-09-19T05:09:49.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 -- 172.21.15.64:0/698761034 shutdown_connections 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b0716f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b07162a10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 --2- 172.21.15.64:0/698761034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 -- 172.21.15.64:0/698761034 >> 172.21.15.64:0/698761034 conn(0x555b0712b310 msgr2=0x555b0715ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 -- 172.21.15.64:0/698761034 shutdown_connections 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 -- 172.21.15.64:0/698761034 wait complete. 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.017+0000 7f0382cc9740 1 -- start start 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 Processor -- start 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b0716cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b07163b40 0x555b07169ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 0x555b0716a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555b07170010 con 0x555b0716d370 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b07083060 con 0x555b071625d0 2024-09-19T05:09:49.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b07170190 con 0x555b07163b40 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0382cc9740 10 client.0 _ll_register_callbacks cb 0x555b06dd68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 0x555b0716a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 0x555b0716a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52364/0 (socket says 172.21.15.64:52364) 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 -- 172.21.15.64:0/4173993771 learned_addr learned my addr 172.21.15.64:0/4173993771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b0716cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f037a59c640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b07163b40 0x555b07169ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 -- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b07163b40 msgr2=0x555b07169ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b07163b40 0x555b07169ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 -- 172.21.15.64:0/4173993771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 msgr2=0x555b0716cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b0716cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.018+0000 7f0379d9b640 1 -- 172.21.15.64:0/4173993771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b06e393c0 con 0x555b0716d370 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0379d9b640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 0x555b0716a5e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0370012fa0 tx=0x7f0370004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c771e12bed27f7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:49.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0361ffb640 1 -- 172.21.15.64:0/4173993771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037000ce10 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0361ffb640 1 -- 172.21.15.64:0/4173993771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0370004020 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c0316a0 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0361ffb640 1 -- 172.21.15.64:0/4173993771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0370004220 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 10 client.16395 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b0717bae0 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0361ffb640 1 -- 172.21.15.64:0/4173993771 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f036c0316a0 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0361ffb640 1 -- 172.21.15.64:0/4173993771 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f03700035f0 con 0x555b0716d370 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 10 client.16395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 20 client.16395 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:49.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b0717cff0 0x555b0717f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 4 client.16395 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.019+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b070d5860 con 0x555b0717cff0 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.020+0000 7f037a59c640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b0717cff0 0x555b0717f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.020+0000 7f037a59c640 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b0717cff0 0x555b0717f450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fb505f74c62c8d0 server_cookie=52dc7de2c828e836 in_seq=0 out_seq=0 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.020+0000 7f0361ffb640 10 client.16395 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0361ffb640 1 -- 172.21.15.64:0/4173993771 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b070d5860 con 0x555b0717cff0 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0361ffb640 10 client.16395 handle_command_reply: tid=0 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 client.16395 shutdown 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 msgr2=0x555b0716a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 0x555b0716a5e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0370012fa0 tx=0x7f0370004730 comp rx=0 tx=0).stop 2024-09-19T05:09:49.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 shutdown_connections 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b0717cff0 0x555b0717f450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b0716d370 0x555b0716a5e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b07163b40 0x555b07169ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0361ffb640 0 client.16395 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 --2- 172.21.15.64:0/4173993771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b0716cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 >> 172.21.15.64:0/4173993771 conn(0x555b0712b310 msgr2=0x555b07164930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 shutdown_connections 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 1 -- 172.21.15.64:0/4173993771 wait complete. 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.021+0000 7f0382cc9740 20 client.16395 trim_cache size 0 max 16384 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.034+0000 7f0382cc9740 1 Processor -- start 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.034+0000 7f0382cc9740 1 -- start start 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b0716bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0382cc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1b01c0 con 0x555b071625d0 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0382cc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1ada90 con 0x555b07163b40 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0382cc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1ab8e0 con 0x555b0716d370 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b0716bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b0716bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52368/0 (socket says 172.21.15.64:52368) 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 -- 172.21.15.64:0/1953567538 learned_addr learned my addr 172.21.15.64:0/1953567538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f037a59c640 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 -- 172.21.15.64:0/1953567538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 msgr2=0x555b07165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07165660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 -- 172.21.15.64:0/1953567538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 msgr2=0x555b07165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 -- 172.21.15.64:0/1953567538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0716abe0 con 0x555b071625d0 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0379d9b640 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b0716bf60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0370005ab0 tx=0x7f0370013fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8994efb17bb7767a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0363fff640 1 -- 172.21.15.64:0/1953567538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037000ce10 con 0x555b071625d0 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.035+0000 7f0363fff640 1 -- 172.21.15.64:0/1953567538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0370004060 con 0x555b071625d0 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0363fff640 1 -- 172.21.15.64:0/1953567538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03700121a0 con 0x555b071625d0 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 -- 172.21.15.64:0/1953567538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 msgr2=0x555b0716bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b0716bf60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0370005ab0 tx=0x7f0370013fe0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 -- 172.21.15.64:0/1953567538 shutdown_connections 2024-09-19T05:09:49.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b07163b40 0x555b07165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 --2- 172.21.15.64:0/1953567538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b071625d0 0x555b0716bf60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 -- 172.21.15.64:0/1953567538 >> 172.21.15.64:0/1953567538 conn(0x555b0712b310 msgr2=0x555b0716b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 -- 172.21.15.64:0/1953567538 shutdown_connections 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.036+0000 7f0382cc9740 1 -- 172.21.15.64:0/1953567538 wait complete. 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 -- start start 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 Processor -- start 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b07167070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 0x555b071675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07167af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1abf50 con 0x555b07163b40 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1ad3c0 con 0x555b071625d0 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1ac620 con 0x555b0716d370 2024-09-19T05:09:49.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0382cc9740 10 client.0 _ll_register_callbacks cb 0x7f037000ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 0x555b071675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 0x555b071675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52382/0 (socket says 172.21.15.64:52382) 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 -- 172.21.15.64:0/386117388 learned_addr learned my addr 172.21.15.64:0/386117388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0379d9b640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b07167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f0379d9b640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 -- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 msgr2=0x555b07167af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07167af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 -- 172.21.15.64:0/386117388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 msgr2=0x555b07167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b07167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.037+0000 7f037a59c640 1 -- 172.21.15.64:0/386117388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b071e6d40 con 0x555b07163b40 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f037a59c640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 0x555b071675b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f036c005340 tx=0x7f036c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ace1a1d96fdc43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:49.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0361ffb640 1 -- 172.21.15.64:0/386117388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c040070 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0361ffb640 1 -- 172.21.15.64:0/386117388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f036c031660 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b070cfd70 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0361ffb640 1 -- 172.21.15.64:0/386117388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f036c035040 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 10 client.16401 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b071e8020 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0361ffb640 1 -- 172.21.15.64:0/386117388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555b070cfd70 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0361ffb640 1 -- 172.21.15.64:0/386117388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f036c04dac0 con 0x555b07163b40 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 10 client.16401 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 10 client.16401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 20 client.16401 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b071e8410 0x555b071ea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:49.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 4 client.16401 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.038+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b0704c170 con 0x555b071e8410 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0379d9b640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b071e8410 0x555b071ea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0379d9b640 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b071e8410 0x555b071ea800 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58d959f8e7dfceb6 server_cookie=7ecbd8a676cb9def in_seq=0 out_seq=0 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0361ffb640 10 client.16401 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0361ffb640 1 -- 172.21.15.64:0/386117388 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b0704c170 con 0x555b071e8410 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0361ffb640 10 client.16401 handle_command_reply: tid=0 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 client.16401 shutdown 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 msgr2=0x555b071675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 0x555b071675b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f036c005340 tx=0x7f036c031400 comp rx=0 tx=0).stop 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 shutdown_connections 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b0716d370 0x555b07167af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x555b071e8410 0x555b071ea800 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555b07163b40 0x555b071675b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 --2- 172.21.15.64:0/386117388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b071625d0 0x555b07167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 >> 172.21.15.64:0/386117388 conn(0x555b0712b310 msgr2=0x555b0715dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0361ffb640 0 client.16401 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.039+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 shutdown_connections 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f0382cc9740 1 -- 172.21.15.64:0/386117388 wait complete. 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:49.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f0382cc9740 20 client.16401 trim_cache size 0 max 16384 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f03640a33b0 msgr2=0x7f03640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f03640a33b0 0x7f03640a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f037409be60 tx=0x7f037404b040 comp rx=0 tx=0).stop 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 msgr2=0x7f037c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 0x7f037c19cc30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f037400eaa0 tx=0x7f037400ef60 comp rx=0 tx=0).stop 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 shutdown_connections 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f03640a33b0 0x7f03640a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f037c106f10 0x7f037c19cc30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f037c102790 0x7f037c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 --2- 172.21.15.64:0/23782258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f037c101de0 0x7f037c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 >> 172.21.15.64:0/23782258 conn(0x7f037c0fd8c0 msgr2=0x7f037c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 shutdown_connections 2024-09-19T05:09:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:49.040+0000 7f03808f0640 1 -- 172.21.15.64:0/23782258 wait complete. 2024-09-19T05:09:49.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:49 smithi103 ceph-mon[34446]: pgmap v120: 97 pgs: 97 active+clean; 386 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-09-19T05:09:49.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:49 smithi174 ceph-mon[34488]: pgmap v120: 97 pgs: 97 active+clean; 386 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-09-19T05:09:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:49 smithi064 ceph-mon[30960]: pgmap v120: 97 pgs: 97 active+clean; 386 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-09-19T05:09:49.999 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:09:50.263 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:50.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 -- 172.21.15.64:0/915448787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763410c180 msgr2=0x7f763410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 --2- 172.21.15.64:0/915448787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763410c180 0x7f763410e610 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f762c009900 tx=0x7f762c02a480 comp rx=0 tx=0).stop 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 -- 172.21.15.64:0/915448787 shutdown_connections 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 --2- 172.21.15.64:0/915448787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f763410c180 0x7f763410e610 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 --2- 172.21.15.64:0/915448787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7634103700 0x7f763410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 --2- 172.21.15.64:0/915448787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634102de0 0x7f76341031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 -- 172.21.15.64:0/915448787 >> 172.21.15.64:0/915448787 conn(0x7f76340fc7a0 msgr2=0x7f76340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 -- 172.21.15.64:0/915448787 shutdown_connections 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.734+0000 7f763bdfe640 1 -- 172.21.15.64:0/915448787 wait complete. 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 Processor -- start 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 -- start start 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 0x7f76341982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634103700 0x7f76341987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f763410c180 0x7f763419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111de0 con 0x7f7634102de0 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111c60 con 0x7f763410c180 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763bdfe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111f60 con 0x7f7634103700 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 0x7f76341982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 0x7f76341982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52388/0 (socket says 172.21.15.64:52388) 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 -- 172.21.15.64:0/2310075723 learned_addr learned my addr 172.21.15.64:0/2310075723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f763a39b640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f763410c180 0x7f763419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639399640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634103700 0x7f76341987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 -- 172.21.15.64:0/2310075723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634103700 msgr2=0x7f76341987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634103700 0x7f76341987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 -- 172.21.15.64:0/2310075723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f763410c180 msgr2=0x7f763419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f763410c180 0x7f763419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639b9a640 1 -- 172.21.15.64:0/2310075723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763419d250 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.735+0000 7f7639399640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634103700 0x7f76341987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.736+0000 7f7639b9a640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 0x7f76341982a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f762400cce0 tx=0x7f762400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.736+0000 7f7632ffd640 1 -- 172.21.15.64:0/2310075723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400f650 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.736+0000 7f7632ffd640 1 -- 172.21.15.64:0/2310075723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f762400fca0 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.736+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763419d540 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.736+0000 7f7632ffd640 1 -- 172.21.15.64:0/2310075723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624018b90 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.736+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a4e20 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.738+0000 7f7632ffd640 1 -- 172.21.15.64:0/2310075723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f762400f7f0 con 0x7f7634102de0 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.738+0000 7f763e1d7740 1 Processor -- start 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 -- start start 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f86b40 0x5590f0f85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f90370 0x5590f0f92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5590f0eeb2f0 con 0x5590f0f855d0 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5590f0e92cf0 con 0x5590f0f86b40 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763e1d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590f0ee72b0 con 0x5590f0f90370 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f85a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52400/0 (socket says 172.21.15.64:52400) 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 -- 172.21.15.64:0/3073808340 learned_addr learned my addr 172.21.15.64:0/3073808340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639399640 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f90370 0x5590f0f92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7632ffd640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f761c0a33b0 0x7f761c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763a39b640 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f86b40 0x5590f0f85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f763a39b640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f761c0a33b0 0x7f761c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7632ffd640 1 -- 172.21.15.64:0/2310075723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f76240d0210 con 0x7f7634102de0 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 -- 172.21.15.64:0/3073808340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f90370 msgr2=0x5590f0f92760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f90370 0x5590f0f92760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 -- 172.21.15.64:0/3073808340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f86b40 msgr2=0x5590f0f85090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f86b40 0x5590f0f85090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639b9a640 1 -- 172.21.15.64:0/3073808340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f0e773a0 con 0x5590f0f855d0 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.739+0000 7f7639399640 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f90370 0x5590f0f92760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.740+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f85a10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f762409c1f0 tx=0x7f7624018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8a03614ba248bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.740+0000 7f763a39b640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f761c0a33b0 0x7f761c0a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f762c006010 tx=0x7f762c0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.740+0000 7f761bfff640 1 -- 172.21.15.64:0/3073808340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762409c030 con 0x5590f0f855d0 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.740+0000 7f761bfff640 1 -- 172.21.15.64:0/3073808340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f762400fae0 con 0x5590f0f855d0 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.740+0000 7f761bfff640 1 -- 172.21.15.64:0/3073808340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762409be50 con 0x5590f0f855d0 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 -- 172.21.15.64:0/3073808340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 msgr2=0x5590f0f85a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f85a10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f762409c1f0 tx=0x7f7624018400 comp rx=0 tx=0).stop 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 -- 172.21.15.64:0/3073808340 shutdown_connections 2024-09-19T05:09:50.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f90370 0x5590f0f92760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f85a10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3073808340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f86b40 0x5590f0f85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 -- 172.21.15.64:0/3073808340 >> 172.21.15.64:0/3073808340 conn(0x5590f0f4e310 msgr2=0x5590f0f81c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 -- 172.21.15.64:0/3073808340 shutdown_connections 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 -- 172.21.15.64:0/3073808340 wait complete. 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 -- start start 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.741+0000 7f763e1d7740 1 Processor -- start 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f8fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f8d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0f8d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5590f0f93010 con 0x5590f0f855d0 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5590f0ea6060 con 0x5590f0f90370 2024-09-19T05:09:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590f0f93190 con 0x5590f0f86b40 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f763e1d7740 10 client.0 _ll_register_callbacks cb 0x5590f0bf98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f8fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f8fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52402/0 (socket says 172.21.15.64:52402) 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 -- 172.21.15.64:0/4223064055 learned_addr learned my addr 172.21.15.64:0/4223064055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639399640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f8d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 -- 172.21.15.64:0/4223064055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 msgr2=0x5590f0f8d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f8d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 -- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 msgr2=0x5590f0f8d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0f8d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 -- 172.21.15.64:0/4223064055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f0c5c3c0 con 0x5590f0f855d0 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639399640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f8d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f7639b9a640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f8fe00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f76240cda10 tx=0x7f7624018f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf502c7273eed5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f76197fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:50.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f76197fa640 1 -- 172.21.15.64:0/4223064055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762409be50 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.742+0000 7f76197fa640 1 -- 172.21.15.64:0/4223064055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f76240cdbe0 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590f0ef8860 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f76197fa640 1 -- 172.21.15.64:0/4223064055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f762409c030 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 10 client.16413 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590f0f93010 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f76197fa640 1 -- 172.21.15.64:0/4223064055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5590f0ef8860 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f76197fa640 1 -- 172.21.15.64:0/4223064055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76240cd520 con 0x5590f0f855d0 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 20 client.16413 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f0f9fce0 0x5590f0fa20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 4 client.16413 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:50.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590f0ef2d70 con 0x5590f0f9fce0 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.743+0000 7f7639399640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f0f9fce0 0x5590f0fa20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.744+0000 7f76197fa640 10 client.16413 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.744+0000 7f7639399640 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f0f9fce0 0x5590f0fa20d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f8790b7fc43803 server_cookie=d5813f085aa0fd73 in_seq=0 out_seq=0 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.744+0000 7f76197fa640 1 -- 172.21.15.64:0/4223064055 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590f0ef2d70 con 0x5590f0f9fce0 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.744+0000 7f76197fa640 10 client.16413 handle_command_reply: tid=0 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.744+0000 7f763e1d7740 1 client.16413 shutdown 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 msgr2=0x5590f0f8fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f8fe00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f76240cda10 tx=0x7f7624018f60 comp rx=0 tx=0).stop 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 shutdown_connections 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0f8d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f8d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f0f9fce0 0x5590f0fa20d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 --2- 172.21.15.64:0/4223064055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f8fe00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f76197fa640 0 client.16413 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 >> 172.21.15.64:0/4223064055 conn(0x5590f0f4e310 msgr2=0x5590f0f87990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 shutdown_connections 2024-09-19T05:09:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 1 -- 172.21.15.64:0/4223064055 wait complete. 2024-09-19T05:09:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.745+0000 7f763e1d7740 20 client.16413 trim_cache size 0 max 16384 2024-09-19T05:09:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.758+0000 7f763e1d7740 1 Processor -- start 2024-09-19T05:09:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.758+0000 7f763e1d7740 1 -- start start 2024-09-19T05:09:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f91760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f91ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0fa7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763e1d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76341b0080 con 0x5590f0f855d0 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763e1d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ad950 con 0x5590f0f90370 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763e1d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ab7a0 con 0x5590f0f86b40 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f91760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f91760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52404/0 (socket says 172.21.15.64:52404) 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 -- 172.21.15.64:0/3997056891 learned_addr learned my addr 172.21.15.64:0/3997056891 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763a39b640 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0fa7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 -- 172.21.15.64:0/3997056891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 msgr2=0x5590f0f91ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f91ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 -- 172.21.15.64:0/3997056891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 msgr2=0x5590f0fa7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0fa7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 -- 172.21.15.64:0/3997056891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f0f8e010 con 0x5590f0f855d0 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f763a39b640 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0fa7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f91760 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f762400afd0 tx=0x7f7624018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b224309803900a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.759+0000 7f761b7fe640 1 -- 172.21.15.64:0/3997056891 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762409be50 con 0x5590f0f855d0 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f761b7fe640 1 -- 172.21.15.64:0/3997056891 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7624066020 con 0x5590f0f855d0 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f761b7fe640 1 -- 172.21.15.64:0/3997056891 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624098790 con 0x5590f0f855d0 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 -- 172.21.15.64:0/3997056891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 msgr2=0x5590f0f91760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f91760 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f762400afd0 tx=0x7f7624018400 comp rx=0 tx=0).stop 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 -- 172.21.15.64:0/3997056891 shutdown_connections 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f90370 0x5590f0fa7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f91ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3997056891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f855d0 0x5590f0f91760 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 -- 172.21.15.64:0/3997056891 >> 172.21.15.64:0/3997056891 conn(0x5590f0f4e310 msgr2=0x5590f0f8c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 -- 172.21.15.64:0/3997056891 shutdown_connections 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.760+0000 7f763e1d7740 1 -- 172.21.15.64:0/3997056891 wait complete. 2024-09-19T05:09:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 -- start start 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 Processor -- start 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f855d0 0x5590f0f98430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f98970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 0x5590f0f933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76341abe10 con 0x5590f0f90370 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ad280 con 0x5590f0f855d0 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ac4e0 con 0x5590f0f86b40 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763e1d7740 10 client.0 _ll_register_callbacks cb 0x7f762c032c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763a39b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 0x5590f0f933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763a39b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 0x5590f0f933e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52418/0 (socket says 172.21.15.64:52418) 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f763a39b640 1 -- 172.21.15.64:0/3579201457 learned_addr learned my addr 172.21.15.64:0/3579201457 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f855d0 0x5590f0f98430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.761+0000 7f7639399640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f98970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763a39b640 1 -- 172.21.15.64:0/3579201457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 msgr2=0x5590f0f98970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763a39b640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f98970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763a39b640 1 -- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f855d0 msgr2=0x5590f0f98430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763a39b640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f855d0 0x5590f0f98430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763a39b640 1 -- 172.21.15.64:0/3579201457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f0f93920 con 0x5590f0f90370 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763a39b640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 0x5590f0f933e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f762c02a9a0 tx=0x7f762c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680e0edc45ac9292 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f76197fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f76197fa640 1 -- 172.21.15.64:0/3579201457 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c04f030 con 0x5590f0f90370 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f76197fa640 1 -- 172.21.15.64:0/3579201457 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f762c005020 con 0x5590f0f90370 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7624066020 con 0x5590f0f90370 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f76197fa640 1 -- 172.21.15.64:0/3579201457 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f762c059050 con 0x5590f0f90370 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763e1d7740 10 client.16419 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590f100f1c0 con 0x5590f0f90370 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.762+0000 7f76197fa640 1 -- 172.21.15.64:0/3579201457 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7624066020 con 0x5590f0f90370 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f76197fa640 1 -- 172.21.15.64:0/3579201457 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f762c003c60 con 0x5590f0f90370 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f763e1d7740 10 client.16419 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f763e1d7740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f763e1d7740 20 client.16419 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f100f990 0x5590f1011db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f763e1d7740 4 client.16419 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590f0e6f170 con 0x5590f100f990 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f100f990 0x5590f1011db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f7639b9a640 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f100f990 0x5590f1011db0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42664d270b0e8cfb server_cookie=d4e552cde3acc46a in_seq=0 out_seq=0 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.763+0000 7f76197fa640 10 client.16419 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f76197fa640 1 -- 172.21.15.64:0/3579201457 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590f0e6f170 con 0x5590f100f990 2024-09-19T05:09:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f76197fa640 10 client.16419 handle_command_reply: tid=0 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 client.16419 shutdown 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 msgr2=0x5590f0f933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 0x5590f0f933e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f762c02a9a0 tx=0x7f762c005db0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 shutdown_connections 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590f100f990 0x5590f1011db0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590f0f90370 0x5590f0f933e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f0f86b40 0x5590f0f98970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 --2- 172.21.15.64:0/3579201457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590f0f855d0 0x5590f0f98430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f76197fa640 0 client.16419 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 >> 172.21.15.64:0/3579201457 conn(0x5590f0f4e310 msgr2=0x5590f0f87110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 shutdown_connections 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 1 -- 172.21.15.64:0/3579201457 wait complete. 2024-09-19T05:09:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.764+0000 7f763e1d7740 20 client.16419 trim_cache size 0 max 16384 2024-09-19T05:09:50.773 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f761c0a33b0 msgr2=0x7f761c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f761c0a33b0 0x7f761c0a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f762c006010 tx=0x7f762c0058d0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 msgr2=0x7f76341982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 0x7f76341982a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f762400cce0 tx=0x7f762400a620 comp rx=0 tx=0).stop 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 shutdown_connections 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f761c0a33b0 0x7f761c0a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f763410c180 0x7f763419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7634103700 0x7f76341987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 --2- 172.21.15.64:0/2310075723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7634102de0 0x7f76341982a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 >> 172.21.15.64:0/2310075723 conn(0x7f76340fc7a0 msgr2=0x7f76340fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 shutdown_connections 2024-09-19T05:09:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:50.765+0000 7f763bdfe640 1 -- 172.21.15.64:0/2310075723 wait complete. 2024-09-19T05:09:51.716 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:51 smithi064 ceph-mon[30960]: pgmap v121: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 578 op/s 2024-09-19T05:09:51.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:51 smithi103 ceph-mon[34446]: pgmap v121: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 578 op/s 2024-09-19T05:09:51.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:51 smithi174 ceph-mon[34488]: pgmap v121: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 578 op/s 2024-09-19T05:09:52.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:09:52.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:09:53.033 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 -- 172.21.15.64:0/1631210541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 msgr2=0x7ffb40107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 --2- 172.21.15.64:0/1631210541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40107b90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009930 tx=0x7ffb3002a420 comp rx=0 tx=0).stop 2024-09-19T05:09:53.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 -- 172.21.15.64:0/1631210541 shutdown_connections 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 --2- 172.21.15.64:0/1631210541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb401082a0 0x7ffb4010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 --2- 172.21.15.64:0/1631210541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40107b90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 --2- 172.21.15.64:0/1631210541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb40102d30 0x7ffb40103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.491+0000 7ffb4822c640 1 -- 172.21.15.64:0/1631210541 >> 172.21.15.64:0/1631210541 conn(0x7ffb400fc7d0 msgr2=0x7ffb400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 -- 172.21.15.64:0/1631210541 shutdown_connections 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 -- 172.21.15.64:0/1631210541 wait complete. 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 Processor -- start 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 -- start start 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb40102d30 0x7ffb40100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb401082a0 0x7ffb40101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40111f60 con 0x7ffb401036e0 2024-09-19T05:09:53.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40111de0 con 0x7ffb40102d30 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.492+0000 7ffb4822c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401120e0 con 0x7ffb401082a0 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52440/0 (socket says 172.21.15.64:52440) 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 -- 172.21.15.64:0/3960359795 learned_addr learned my addr 172.21.15.64:0/3960359795 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb40102d30 0x7ffb40100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb467c9640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb401082a0 0x7ffb40101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 -- 172.21.15.64:0/3960359795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb401082a0 msgr2=0x7ffb40101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb401082a0 0x7ffb40101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 -- 172.21.15.64:0/3960359795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb40102d30 msgr2=0x7ffb40100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb40102d30 0x7ffb40100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 -- 172.21.15.64:0/3960359795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb401a6a80 con 0x7ffb401036e0 2024-09-19T05:09:53.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb457c7640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40100ed0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb30031440 tx=0x7ffb30003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb36ffd640 1 -- 172.21.15.64:0/3960359795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3003f070 con 0x7ffb401036e0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb401a6d10 con 0x7ffb401036e0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb36ffd640 1 -- 172.21.15.64:0/3960359795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb30031b50 con 0x7ffb401036e0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb36ffd640 1 -- 172.21.15.64:0/3960359795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30035030 con 0x7ffb401036e0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.493+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401a71c0 con 0x7ffb401036e0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 Processor -- start 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 -- start start 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb36ffd640 1 -- 172.21.15.64:0/3960359795 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ffb30042070 con 0x7ffb401036e0 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x5639604faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639604fb480 0x563960505800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 0x563960508130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563960460280 con 0x563960505d40 2024-09-19T05:09:53.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563960407b10 con 0x5639604fab60 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.496+0000 7ffb4a605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56396045c240 con 0x5639604fb480 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 0x563960508130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 0x563960508130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52454/0 (socket says 172.21.15.64:52454) 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 -- 172.21.15.64:0/3028392679 learned_addr learned my addr 172.21.15.64:0/3028392679 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639604fb480 0x563960505800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb467c9640 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x5639604faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 -- 172.21.15.64:0/3028392679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639604fb480 msgr2=0x563960505800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639604fb480 0x563960505800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 -- 172.21.15.64:0/3028392679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 msgr2=0x5639604faf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x5639604faf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 -- 172.21.15.64:0/3028392679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639604141f0 con 0x563960505d40 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb457c7640 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 0x563960508130 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffb300ed920 tx=0x7ffb300f8580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb8e2fa3d9ba36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:53.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb36ffd640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb2c0a33b0 0x7ffb2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.497+0000 7ffb2bfff640 1 -- 172.21.15.64:0/3028392679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300b8890 con 0x563960505d40 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb467c9640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb2c0a33b0 0x7ffb2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb2bfff640 1 -- 172.21.15.64:0/3028392679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb300fd030 con 0x563960505d40 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb2bfff640 1 -- 172.21.15.64:0/3028392679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300bbd70 con 0x563960505d40 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb36ffd640 1 -- 172.21.15.64:0/3960359795 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ffb300f0260 con 0x7ffb401036e0 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 -- 172.21.15.64:0/3028392679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 msgr2=0x563960508130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb467c9640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb2c0a33b0 0x7ffb2c0a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005fd0 tx=0x7ffb38009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 0x563960508130 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffb300ed920 tx=0x7ffb300f8580 comp rx=0 tx=0).stop 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 -- 172.21.15.64:0/3028392679 shutdown_connections 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563960505d40 0x563960508130 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639604fb480 0x563960505800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 --2- 172.21.15.64:0/3028392679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x5639604faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.498+0000 7ffb4a605740 1 -- 172.21.15.64:0/3028392679 >> 172.21.15.64:0/3028392679 conn(0x5639604f70f0 msgr2=0x5639604f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:53.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 -- 172.21.15.64:0/3028392679 shutdown_connections 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 -- 172.21.15.64:0/3028392679 wait complete. 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 -- start start 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 Processor -- start 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960502b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x5639605030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960503600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639605086d0 con 0x5639604fb480 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56396041af10 con 0x5639604fab60 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563960508850 con 0x563960505d40 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.499+0000 7ffb4a605740 10 client.0 _ll_register_callbacks cb 0x5639603e3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x5639605030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x5639605030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52462/0 (socket says 172.21.15.64:52462) 2024-09-19T05:09:53.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 -- 172.21.15.64:0/1464710269 learned_addr learned my addr 172.21.15.64:0/1464710269 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960502b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb467c9640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960503600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 -- 172.21.15.64:0/1464710269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 msgr2=0x563960503600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960503600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 -- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 msgr2=0x563960502b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960502b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 -- 172.21.15.64:0/1464710269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639601d16a0 con 0x5639604fb480 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb467c9640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960503600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb457c7640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x5639605030c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ffb300c3050 tx=0x7ffb3002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db3e8f6c6382cee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb297fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb297fa640 1 -- 172.21.15.64:0/1464710269 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30049070 con 0x5639604fb480 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb297fa640 1 -- 172.21.15.64:0/1464710269 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ffb300ba8f0 con 0x5639604fb480 2024-09-19T05:09:53.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb300fd030 con 0x5639604fb480 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb297fa640 1 -- 172.21.15.64:0/1464710269 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb300bbf50 con 0x5639604fb480 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb4a605740 10 client.16431 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.500+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639605086d0 con 0x5639604fb480 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb297fa640 1 -- 172.21.15.64:0/1464710269 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb300fd030 con 0x5639604fb480 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb297fa640 1 -- 172.21.15.64:0/1464710269 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffb300b8c60 con 0x5639604fb480 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb4a605740 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb4a605740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb4a605740 20 client.16431 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb4a605740 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563960515380 0x563960517770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb4a605740 4 client.16431 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56396046d7f0 con 0x563960515380 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.501+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563960515380 0x563960517770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563960515380 0x563960517770 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1dcd091489236ba server_cookie=ffbb764c59bd3011 in_seq=0 out_seq=0 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb297fa640 10 client.16431 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb297fa640 1 -- 172.21.15.64:0/1464710269 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56396046d7f0 con 0x563960515380 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb297fa640 10 client.16431 handle_command_reply: tid=0 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb4a605740 1 client.16431 shutdown 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 msgr2=0x5639605030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.502+0000 7ffb4a605740 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x5639605030c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ffb300c3050 tx=0x7ffb3002a420 comp rx=0 tx=0).stop 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 shutdown_connections 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960503600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563960515380 0x563960517770 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x5639605030c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 --2- 172.21.15.64:0/1464710269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960502b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb297fa640 0 client.16431 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:53.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 >> 172.21.15.64:0/1464710269 conn(0x5639604f70f0 msgr2=0x5639604fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:53.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 shutdown_connections 2024-09-19T05:09:53.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 1 -- 172.21.15.64:0/1464710269 wait complete. 2024-09-19T05:09:53.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.503+0000 7ffb4a605740 20 client.16431 trim_cache size 0 max 16384 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.520+0000 7ffb4a605740 1 Processor -- start 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.520+0000 7ffb4a605740 1 -- start start 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960501e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x563960503450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x56396051cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb4a605740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401b21b0 con 0x5639604fb480 2024-09-19T05:09:53.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb4a605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401afa80 con 0x5639604fab60 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb4a605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401ad8d0 con 0x563960505d40 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x563960503450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x563960503450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52468/0 (socket says 172.21.15.64:52468) 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 -- 172.21.15.64:0/470101673 learned_addr learned my addr 172.21.15.64:0/470101673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960501e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb467c9640 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x56396051cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 -- 172.21.15.64:0/470101673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 msgr2=0x56396051cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x56396051cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 -- 172.21.15.64:0/470101673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 msgr2=0x563960501e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960501e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 -- 172.21.15.64:0/470101673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639604fb2b0 con 0x5639604fb480 2024-09-19T05:09:53.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960501e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.521+0000 7ffb457c7640 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x563960503450 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffb300b8a50 tx=0x7ffb300f0530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8486c30f44c762d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.522+0000 7ffb2b7fe640 1 -- 172.21.15.64:0/470101673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30049040 con 0x5639604fb480 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.522+0000 7ffb2b7fe640 1 -- 172.21.15.64:0/470101673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb300fd070 con 0x5639604fb480 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.522+0000 7ffb2b7fe640 1 -- 172.21.15.64:0/470101673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300bc130 con 0x5639604fb480 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 -- 172.21.15.64:0/470101673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 msgr2=0x563960503450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x563960503450 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffb300b8a50 tx=0x7ffb300f0530 comp rx=0 tx=0).stop 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 -- 172.21.15.64:0/470101673 shutdown_connections 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x56396051cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x563960503450 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 --2- 172.21.15.64:0/470101673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x563960501e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 -- 172.21.15.64:0/470101673 >> 172.21.15.64:0/470101673 conn(0x5639604f70f0 msgr2=0x563960504af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 -- 172.21.15.64:0/470101673 shutdown_connections 2024-09-19T05:09:53.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 -- 172.21.15.64:0/470101673 wait complete. 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 -- start start 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.523+0000 7ffb4a605740 1 Processor -- start 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x56396050dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x56396050dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960508a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401adf40 con 0x5639604fb480 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401af3b0 con 0x5639604fab60 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x56396050dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401ae610 con 0x563960505d40 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x56396050dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52474/0 (socket says 172.21.15.64:52474) 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 -- 172.21.15.64:0/2724555389 learned_addr learned my addr 172.21.15.64:0/2724555389 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb4a605740 10 client.0 _ll_register_callbacks cb 0x7ffb38009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:53.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x56396050dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb467c9640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960508a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 -- 172.21.15.64:0/2724555389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 msgr2=0x563960508a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960508a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 -- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 msgr2=0x56396050dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x56396050dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb457c7640 1 -- 172.21.15.64:0/2724555389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563960508fa0 con 0x5639604fb480 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.524+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x56396050dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb457c7640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x56396050dff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffb300c4c80 tx=0x7ffb30005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d9baefa99bee09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb297fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb297fa640 1 -- 172.21.15.64:0/2724555389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30049070 con 0x5639604fb480 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb297fa640 1 -- 172.21.15.64:0/2724555389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ffb30031d00 con 0x5639604fb480 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb300fd070 con 0x5639604fb480 2024-09-19T05:09:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb297fa640 1 -- 172.21.15.64:0/2724555389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb300bc3c0 con 0x5639604fb480 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 10 client.16437 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563960508fa0 con 0x5639604fb480 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb297fa640 1 -- 172.21.15.64:0/2724555389 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb300fd070 con 0x5639604fb480 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb297fa640 1 -- 172.21.15.64:0/2724555389 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffb300edc30 con 0x5639604fb480 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 10 client.16437 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 20 client.16437 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.525+0000 7ffb4a605740 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5639605808d0 0x563960582cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.526+0000 7ffb4a605740 4 client.16437 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.526+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563960467d00 con 0x5639605808d0 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.526+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5639605808d0 0x563960582cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.526+0000 7ffb45fc8640 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5639605808d0 0x563960582cf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91e8495c0ae3280 server_cookie=8f3ff3952224068d in_seq=0 out_seq=0 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.526+0000 7ffb297fa640 10 client.16437 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:53.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.527+0000 7ffb297fa640 1 -- 172.21.15.64:0/2724555389 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563960467d00 con 0x5639605808d0 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.527+0000 7ffb297fa640 10 client.16437 handle_command_reply: tid=0 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.527+0000 7ffb4a605740 1 client.16437 shutdown 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 msgr2=0x56396050dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x56396050dff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffb300c4c80 tx=0x7ffb30005c90 comp rx=0 tx=0).stop 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 shutdown_connections 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563960505d40 0x563960508a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5639605808d0 0x563960582cf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639604fb480 0x56396050dff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 --2- 172.21.15.64:0/2724555389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639604fab60 0x56396050dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 >> 172.21.15.64:0/2724555389 conn(0x5639604f70f0 msgr2=0x563960510920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb297fa640 0 client.16437 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 shutdown_connections 2024-09-19T05:09:53.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 1 -- 172.21.15.64:0/2724555389 wait complete. 2024-09-19T05:09:53.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4a605740 20 client.16437 trim_cache size 0 max 16384 2024-09-19T05:09:53.541 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:09:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "a6b97d9d-d9e5-4fbe-aa1a-6f4da77a7837", 2024-09-19T05:09:53.541 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:09:53.541 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb2c0a33b0 msgr2=0x7ffb2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4822c640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb2c0a33b0 0x7ffb2c0a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005fd0 tx=0x7ffb38009500 comp rx=0 tx=0).stop 2024-09-19T05:09:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 msgr2=0x7ffb40100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.528+0000 7ffb4822c640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40100ed0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb30031440 tx=0x7ffb30003b90 comp rx=0 tx=0).stop 2024-09-19T05:09:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 shutdown_connections 2024-09-19T05:09:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb2c0a33b0 0x7ffb2c0a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb401082a0 0x7ffb40101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb401036e0 0x7ffb40100ed0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 --2- 172.21.15.64:0/3960359795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb40102d30 0x7ffb40100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 >> 172.21.15.64:0/3960359795 conn(0x7ffb400fc7d0 msgr2=0x7ffb40105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 shutdown_connections 2024-09-19T05:09:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:53.529+0000 7ffb4822c640 1 -- 172.21.15.64:0/3960359795 wait complete. 2024-09-19T05:09:53.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:53 smithi103 ceph-mon[34446]: pgmap v122: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 336 op/s 2024-09-19T05:09:53.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:53 smithi174 ceph-mon[34488]: pgmap v122: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 336 op/s 2024-09-19T05:09:53.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:53 smithi064 ceph-mon[30960]: pgmap v122: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 336 op/s 2024-09-19T05:09:54.490 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:54.748 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:55.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 -- 172.21.15.64:0/2647309993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 msgr2=0x7fac3c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 --2- 172.21.15.64:0/2647309993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c103120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009900 tx=0x7fac2c02a480 comp rx=0 tx=0).stop 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 -- 172.21.15.64:0/2647309993 shutdown_connections 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 --2- 172.21.15.64:0/2647309993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 0x7fac3c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 --2- 172.21.15.64:0/2647309993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 0x7fac3c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 --2- 172.21.15.64:0/2647309993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c103120 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 -- 172.21.15.64:0/2647309993 >> 172.21.15.64:0/2647309993 conn(0x7fac3c0fc7a0 msgr2=0x7fac3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 -- 172.21.15.64:0/2647309993 shutdown_connections 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 -- 172.21.15.64:0/2647309993 wait complete. 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.196+0000 7fac42158640 1 Processor -- start 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 -- start start 2024-09-19T05:09:55.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 0x7fac3c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 0x7fac3c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111e20 con 0x7fac3c102d40 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111ca0 con 0x7fac3c10c170 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac42158640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c111fa0 con 0x7fac3c103660 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56764/0 (socket says 172.21.15.64:56764) 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 -- 172.21.15.64:0/2612557029 learned_addr learned my addr 172.21.15.64:0/2612557029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3affd640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 0x7fac3c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3bfff640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 0x7fac3c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 -- 172.21.15.64:0/2612557029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 msgr2=0x7fac3c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 0x7fac3c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 -- 172.21.15.64:0/2612557029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 msgr2=0x7fac3c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 0x7fac3c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3b7fe640 1 -- 172.21.15.64:0/2612557029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c1a68f0 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3bfff640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 0x7fac3c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.197+0000 7fac3affd640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 0x7fac3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.198+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c1009a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005940 tx=0x7fac2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.198+0000 7fac38ff9640 1 -- 172.21.15.64:0/2612557029 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c03f030 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.198+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c1a6b80 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.198+0000 7fac38ff9640 1 -- 172.21.15.64:0/2612557029 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c003aa0 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.198+0000 7fac38ff9640 1 -- 172.21.15.64:0/2612557029 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c035070 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.198+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1a70b0 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.200+0000 7fac38ff9640 1 -- 172.21.15.64:0/2612557029 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fac2c042030 con 0x7fac3c102d40 2024-09-19T05:09:55.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.200+0000 7fac44531740 1 Processor -- start 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.200+0000 7fac44531740 1 -- start start 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac44531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d94a0 0x55c1554d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac44531740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac44531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac44531740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c15543f2e0 con 0x55c1554d9dc0 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac44531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c1553e6ce0 con 0x55c1554e4680 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac44531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c15543b2a0 con 0x55c1554d94a0 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56768/0 (socket says 172.21.15.64:56768) 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 -- 172.21.15.64:0/161838104 learned_addr learned my addr 172.21.15.64:0/161838104 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3bfff640 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d94a0 0x55c1554d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3affd640 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 -- 172.21.15.64:0/161838104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d94a0 msgr2=0x55c1554d9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d94a0 0x55c1554d9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 -- 172.21.15.64:0/161838104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 msgr2=0x55c1554e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac38ff9640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fac280a33b0 0x7fac280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3b7fe640 1 -- 172.21.15.64:0/161838104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1553cb440 con 0x55c1554d9dc0 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3affd640 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.201+0000 7fac3bfff640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fac280a33b0 0x7fac280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.202+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e4140 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0edbe0 tx=0x7fac2c0b8b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11daf29c33a978b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.202+0000 7fac38ff9640 1 -- 172.21.15.64:0/2612557029 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fac2c0f05b0 con 0x7fac3c102d40 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.202+0000 7fac31ffb640 1 -- 172.21.15.64:0/161838104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0bc600 con 0x55c1554d9dc0 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.202+0000 7fac31ffb640 1 -- 172.21.15.64:0/161838104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c03b460 con 0x55c1554d9dc0 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.202+0000 7fac31ffb640 1 -- 172.21.15.64:0/161838104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0bc190 con 0x55c1554d9dc0 2024-09-19T05:09:55.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 -- 172.21.15.64:0/161838104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 msgr2=0x55c1554e4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e4140 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0edbe0 tx=0x7fac2c0b8b90 comp rx=0 tx=0).stop 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac3bfff640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fac280a33b0 0x7fac280a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fac34009370 tx=0x7fac34009720 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 -- 172.21.15.64:0/161838104 shutdown_connections 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e4140 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 --2- 172.21.15.64:0/161838104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d94a0 0x55c1554d9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 -- 172.21.15.64:0/161838104 >> 172.21.15.64:0/161838104 conn(0x55c1554d5a30 msgr2=0x55c1554d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 -- 172.21.15.64:0/161838104 shutdown_connections 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 -- 172.21.15.64:0/161838104 wait complete. 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 -- start start 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.203+0000 7fac44531740 1 Processor -- start 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d9dc0 0x55c1554e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c1554e7010 con 0x55c1554d94a0 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c1553fa050 con 0x55c1554e4680 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1554e7190 con 0x55c1554d9dc0 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56776/0 (socket says 172.21.15.64:56776) 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 -- 172.21.15.64:0/4162524869 learned_addr learned my addr 172.21.15.64:0/4162524869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 10 client.0 _ll_register_callbacks cb 0x55c15543a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3affd640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d9dc0 0x55c1554e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3bfff640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 -- 172.21.15.64:0/4162524869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d9dc0 msgr2=0x55c1554e11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d9dc0 0x55c1554e11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 -- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 msgr2=0x55c1554e16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 -- 172.21.15.64:0/4162524869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1551b03c0 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3bfff640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554e3f00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0c4f70 tx=0x7fac2c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c36f3496665bad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac1affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac1affd640 1 -- 172.21.15.64:0/4162524869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c049040 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac1affd640 1 -- 172.21.15.64:0/4162524869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fac2c0c3110 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.204+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c15544c850 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac1affd640 1 -- 172.21.15.64:0/4162524869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac2c0bc370 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 10 client.16449 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1554e7010 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac1affd640 1 -- 172.21.15.64:0/4162524869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c15544c850 con 0x55c1554d94a0 2024-09-19T05:09:55.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac1affd640 1 -- 172.21.15.64:0/4162524869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fac2c0b8ea0 con 0x55c1554d94a0 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 20 client.16449 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c1554f3ae0 0x55c1554f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 4 client.16449 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c155446d60 con 0x55c1554f3ae0 2024-09-19T05:09:55.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.205+0000 7fac3affd640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c1554f3ae0 0x55c1554f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.206+0000 7fac3affd640 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c1554f3ae0 0x55c1554f5f20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a44aeaf3d7a69f server_cookie=6320f7d3865a8b90 in_seq=0 out_seq=0 2024-09-19T05:09:55.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.206+0000 7fac1affd640 10 client.16449 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac1affd640 1 -- 172.21.15.64:0/4162524869 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c155446d60 con 0x55c1554f3ae0 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac1affd640 10 client.16449 handle_command_reply: tid=0 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 client.16449 shutdown 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 msgr2=0x55c1554e3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554e3f00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0c4f70 tx=0x7fac2c02a480 comp rx=0 tx=0).stop 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 shutdown_connections 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554e4680 0x55c1554e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554d9dc0 0x55c1554e11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c1554f3ae0 0x55c1554f5f20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 --2- 172.21.15.64:0/4162524869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554e3f00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 >> 172.21.15.64:0/4162524869 conn(0x55c1554d5a30 msgr2=0x55c1554db710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:55.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac1affd640 0 client.16449 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:55.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 shutdown_connections 2024-09-19T05:09:55.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 1 -- 172.21.15.64:0/4162524869 wait complete. 2024-09-19T05:09:55.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.207+0000 7fac44531740 20 client.16449 trim_cache size 0 max 16384 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.224+0000 7fac44531740 1 Processor -- start 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.224+0000 7fac44531740 1 -- start start 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac44531740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac44531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d9dc0 0x55c1554fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac44531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac44531740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1b2380 con 0x55c1554d94a0 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac44531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1afc50 con 0x55c1554d9dc0 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac44531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1adaa0 con 0x55c1554e4680 2024-09-19T05:09:55.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554f01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56778/0 (socket says 172.21.15.64:56778) 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 -- 172.21.15.64:0/3165229696 learned_addr learned my addr 172.21.15.64:0/3165229696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 -- 172.21.15.64:0/3165229696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 msgr2=0x55c1554fbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3affd640 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d9dc0 0x55c1554fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 -- 172.21.15.64:0/3165229696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d9dc0 msgr2=0x55c1554fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d9dc0 0x55c1554fb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 -- 172.21.15.64:0/3165229696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1554d5860 con 0x55c1554d94a0 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554f01e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac2c03bb80 tx=0x7fac2c0b8460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae3db8d3f3d123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac317fa640 1 -- 172.21.15.64:0/3165229696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c049020 con 0x55c1554d94a0 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac317fa640 1 -- 172.21.15.64:0/3165229696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c0c0be0 con 0x55c1554d94a0 2024-09-19T05:09:55.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.225+0000 7fac317fa640 1 -- 172.21.15.64:0/3165229696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c003c50 con 0x55c1554d94a0 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 -- 172.21.15.64:0/3165229696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 msgr2=0x55c1554f01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554f01e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac2c03bb80 tx=0x7fac2c0b8460 comp rx=0 tx=0).stop 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 -- 172.21.15.64:0/3165229696 shutdown_connections 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d9dc0 0x55c1554fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 --2- 172.21.15.64:0/3165229696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d94a0 0x55c1554f01e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 -- 172.21.15.64:0/3165229696 >> 172.21.15.64:0/3165229696 conn(0x55c1554d5a30 msgr2=0x55c1554dfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 -- 172.21.15.64:0/3165229696 shutdown_connections 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 -- 172.21.15.64:0/3165229696 wait complete. 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.226+0000 7fac44531740 1 -- start start 2024-09-19T05:09:55.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 Processor -- start 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d94a0 0x55c1554ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1ae110 con 0x55c1554d9dc0 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1af580 con 0x55c1554d94a0 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1ae7e0 con 0x55c1554e4680 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac44531740 10 client.0 _ll_register_callbacks cb 0x7fac340095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56782/0 (socket says 172.21.15.64:56782) 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 -- 172.21.15.64:0/2100517794 learned_addr learned my addr 172.21.15.64:0/2100517794 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3bfff640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d94a0 0x55c1554ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 -- 172.21.15.64:0/2100517794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 msgr2=0x55c1554e83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554e83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 -- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d94a0 msgr2=0x55c1554ecb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d94a0 0x55c1554ecb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.227+0000 7fac3affd640 1 -- 172.21.15.64:0/2100517794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1554e8be0 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac3affd640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e7e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55c1554d6db0 tx=0x7fac24014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac9b545b6175a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac1affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac1affd640 1 -- 172.21.15.64:0/2100517794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24004220 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac1affd640 1 -- 172.21.15.64:0/2100517794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fac240043c0 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac2c0c0be0 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac1affd640 1 -- 172.21.15.64:0/2100517794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac2402a050 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c15555e870 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac1affd640 1 -- 172.21.15.64:0/2100517794 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac2c0c0be0 con 0x55c1554d9dc0 2024-09-19T05:09:55.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac1affd640 1 -- 172.21.15.64:0/2100517794 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fac2401b530 con 0x55c1554d9dc0 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 20 client.16455 populate_metadata read hostname 'smithi064' 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.228+0000 7fac44531740 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c15555f020 0x55c155561410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac44531740 4 client.16455 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1554e1a80 con 0x55c15555f020 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c15555f020 0x55c155561410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac3b7fe640 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c15555f020 0x55c155561410 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5a54c0377d18f5 server_cookie=a256d540dc3c59da in_seq=0 out_seq=0 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac1affd640 10 client.16455 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac1affd640 1 -- 172.21.15.64:0/2100517794 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c1554e1a80 con 0x55c15555f020 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac1affd640 10 client.16455 handle_command_reply: tid=0 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac44531740 1 client.16455 shutdown 2024-09-19T05:09:55.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 msgr2=0x55c1554e7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.229+0000 7fac44531740 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e7e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55c1554d6db0 tx=0x7fac24014900 comp rx=0 tx=0).stop 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 shutdown_connections 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1554e4680 0x55c1554e83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c15555f020 0x55c155561410 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1554d9dc0 0x55c1554e7e50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 --2- 172.21.15.64:0/2100517794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c1554d94a0 0x55c1554ecb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac1affd640 0 client.16455 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 >> 172.21.15.64:0/2100517794 conn(0x55c1554d5a30 msgr2=0x55c1554d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 shutdown_connections 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 1 -- 172.21.15.64:0/2100517794 wait complete. 2024-09-19T05:09:55.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac44531740 20 client.16455 trim_cache size 0 max 16384 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: "a6b97d9d-d9e5-4fbe-aa1a-6f4da77a7837": { 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "a6b97d9d-d9e5-4fbe-aa1a-6f4da77a7837", 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:09:55.237 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fac280a33b0 msgr2=0x7fac280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fac280a33b0 0x7fac280a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fac34009370 tx=0x7fac34009720 comp rx=0 tx=0).stop 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 msgr2=0x7fac3c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c1009a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005940 tx=0x7fac2c004270 comp rx=0 tx=0).stop 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 shutdown_connections 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fac280a33b0 0x7fac280a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac3c10c170 0x7fac3c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac3c103660 0x7fac3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.230+0000 7fac42158640 1 --2- 172.21.15.64:0/2612557029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac3c102d40 0x7fac3c1009a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.231+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 >> 172.21.15.64:0/2612557029 conn(0x7fac3c0fc7a0 msgr2=0x7fac3c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.231+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 shutdown_connections 2024-09-19T05:09:55.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:09:55.231+0000 7fac42158640 1 -- 172.21.15.64:0/2612557029 wait complete. 2024-09-19T05:09:55.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:55 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:09:55.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:55 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:55.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:55 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:09:55.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:55 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:55.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:55 smithi064 ceph-mon[30960]: pgmap v123: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 336 op/s 2024-09-19T05:09:55.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:55 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:09:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:55 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:55 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:09:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:55 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:55 smithi103 ceph-mon[34446]: pgmap v123: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 336 op/s 2024-09-19T05:09:55.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:55 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:09:55.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:55 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:55.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:55 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:09:55.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:55 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:55.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:55 smithi174 ceph-mon[34488]: pgmap v123: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 336 op/s 2024-09-19T05:09:56.126 INFO:tasks.cephfs.filesystem:scrub status for tag:a6b97d9d-d9e5-4fbe-aa1a-6f4da77a7837 - {'path': '/', 'tag': 'a6b97d9d-d9e5-4fbe-aa1a-6f4da77a7837', 'options': 'recursive,force'} 2024-09-19T05:09:57.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:57 smithi103 ceph-mon[34446]: pgmap v124: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-09-19T05:09:57.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:57 smithi174 ceph-mon[34488]: pgmap v124: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-09-19T05:09:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:57 smithi064 ceph-mon[30960]: pgmap v124: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-09-19T05:09:59.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:59 smithi103 ceph-mon[34446]: pgmap v125: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 410 KiB/s wr, 117 op/s 2024-09-19T05:09:59.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:09:59 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:09:59.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:59 smithi174 ceph-mon[34488]: pgmap v125: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 410 KiB/s wr, 117 op/s 2024-09-19T05:09:59.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:09:59 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:09:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:59 smithi064 ceph-mon[30960]: pgmap v125: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 410 KiB/s wr, 117 op/s 2024-09-19T05:09:59.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:09:59 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:10:00.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:00 smithi103 ceph-mon[34446]: overall HEALTH_OK 2024-09-19T05:10:00.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:00 smithi174 ceph-mon[34488]: overall HEALTH_OK 2024-09-19T05:10:00.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:00 smithi064 ceph-mon[30960]: overall HEALTH_OK 2024-09-19T05:10:01.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:01 smithi103 ceph-mon[34446]: pgmap v126: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 410 KiB/s wr, 120 op/s 2024-09-19T05:10:01.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:01 smithi174 ceph-mon[34488]: pgmap v126: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 410 KiB/s wr, 120 op/s 2024-09-19T05:10:01.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:01 smithi064 ceph-mon[30960]: pgmap v126: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 410 KiB/s wr, 120 op/s 2024-09-19T05:10:03.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:03 smithi103 ceph-mon[34446]: pgmap v127: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T05:10:03.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:03 smithi174 ceph-mon[34488]: pgmap v127: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T05:10:03.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:03 smithi064 ceph-mon[30960]: pgmap v127: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T05:10:05.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:05 smithi103 ceph-mon[34446]: pgmap v128: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T05:10:05.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:05 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:10:05.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:05 smithi174 ceph-mon[34488]: pgmap v128: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T05:10:05.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:05 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:10:05.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:05 smithi064 ceph-mon[30960]: pgmap v128: 97 pgs: 97 active+clean; 25 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T05:10:05.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:05 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:10:06.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:10:06.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:10:06.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:10:06.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:10:06.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:10:06.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:10:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:10:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:10:06.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:10:07.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:07 smithi103 ceph-mon[34446]: pgmap v129: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-19T05:10:07.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:07 smithi174 ceph-mon[34488]: pgmap v129: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-19T05:10:07.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:07 smithi064 ceph-mon[30960]: pgmap v129: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-19T05:10:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:09 smithi174 ceph-mon[34488]: pgmap v130: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:09.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:09 smithi064 ceph-mon[30960]: pgmap v130: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:09.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:09 smithi103 ceph-mon[34446]: pgmap v130: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:11.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:11 smithi174 ceph-mon[34488]: pgmap v131: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-19T05:10:11.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:11 smithi064 ceph-mon[30960]: pgmap v131: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-19T05:10:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:11 smithi103 ceph-mon[34446]: pgmap v131: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-19T05:10:13.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:13 smithi174 ceph-mon[34488]: pgmap v132: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:13.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:13 smithi064 ceph-mon[30960]: pgmap v132: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:13.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:13 smithi103 ceph-mon[34446]: pgmap v132: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:15.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:15 smithi174 ceph-mon[34488]: pgmap v133: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:15.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:15 smithi064 ceph-mon[30960]: pgmap v133: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:15.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:15 smithi103 ceph-mon[34446]: pgmap v133: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T05:10:17.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:17 smithi174 ceph-mon[34488]: pgmap v134: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-19T05:10:17.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:17 smithi064 ceph-mon[30960]: pgmap v134: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-19T05:10:17.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:17 smithi103 ceph-mon[34446]: pgmap v134: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-19T05:10:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:19 smithi174 ceph-mon[34488]: pgmap v135: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:10:19.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:19 smithi064 ceph-mon[30960]: pgmap v135: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:10:19.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:19 smithi103 ceph-mon[34446]: pgmap v135: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:10:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:21 smithi174 ceph-mon[34488]: pgmap v136: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:10:21.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:21 smithi064 ceph-mon[30960]: pgmap v136: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:10:21.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:21 smithi103 ceph-mon[34446]: pgmap v136: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:10:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:23 smithi174 ceph-mon[34488]: pgmap v137: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:23.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:23 smithi064 ceph-mon[30960]: pgmap v137: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:23.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:23 smithi103 ceph-mon[34446]: pgmap v137: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:25 smithi174 ceph-mon[34488]: pgmap v138: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:25.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:25 smithi064 ceph-mon[30960]: pgmap v138: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:25.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:25 smithi103 ceph-mon[34446]: pgmap v138: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:26.127 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:10:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:26.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1573886933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0101de0 msgr2=0x7fe1c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1573886933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0101de0 0x7fe1c01021c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009900 tx=0x7fe1b402a410 comp rx=0 tx=0).stop 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1573886933 shutdown_connections 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1573886933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0106f10 0x7fe1c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1573886933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0102790 0x7fe1c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1573886933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0101de0 0x7fe1c01021c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1573886933 >> 172.21.15.64:0/1573886933 conn(0x7fe1c00fd8c0 msgr2=0x7fe1c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1573886933 shutdown_connections 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.989+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1573886933 wait complete. 2024-09-19T05:10:26.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 Processor -- start 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 -- start start 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0101de0 0x7fe1c01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0102790 0x7fe1c01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 0x7fe1c019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0111e40 con 0x7fe1c0106f10 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0111cc0 con 0x7fe1c0101de0 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c7fc5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0111fc0 con 0x7fe1c0102790 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c6562640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 0x7fe1c019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0101de0 0x7fe1c01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c6562640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 0x7fe1c019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55960/0 (socket says 172.21.15.64:55960) 2024-09-19T05:10:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.990+0000 7fe1c6562640 1 -- 172.21.15.64:0/1281633341 learned_addr learned my addr 172.21.15.64:0/1281633341 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0102790 0x7fe1c01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c6562640 1 -- 172.21.15.64:0/1281633341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0102790 msgr2=0x7fe1c01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c6562640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0102790 0x7fe1c01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c6562640 1 -- 172.21.15.64:0/1281633341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0101de0 msgr2=0x7fe1c01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c6562640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0101de0 0x7fe1c01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c6562640 1 -- 172.21.15.64:0/1281633341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c019d460 con 0x7fe1c0106f10 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0102790 0x7fe1c01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0101de0 0x7fe1c01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c6562640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 0x7fe1c019cd80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800b970 tx=0x7fe1b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1281633341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8019070 con 0x7fe1c0106f10 2024-09-19T05:10:26.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c019d750 con 0x7fe1c0106f10 2024-09-19T05:10:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1281633341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b8014440 con 0x7fe1c0106f10 2024-09-19T05:10:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c01a5030 con 0x7fe1c0106f10 2024-09-19T05:10:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.991+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1281633341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80075d0 con 0x7fe1c0106f10 2024-09-19T05:10:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1281633341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe1b801c070 con 0x7fe1c0106f10 2024-09-19T05:10:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 Processor -- start 2024-09-19T05:10:27.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 -- start start 2024-09-19T05:10:27.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c194a0 0x559ac8c19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c19dc0 0x559ac8c24140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 0x559ac8c26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559ac8b7f2e0 con 0x559ac8c24680 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ac8b26ce0 con 0x559ac8c194a0 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ac8b7b2a0 con 0x559ac8c19dc0 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1c5560640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 0x559ac8c26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1c5560640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 0x559ac8c26a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55976/0 (socket says 172.21.15.64:55976) 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.994+0000 7fe1c5560640 1 -- 172.21.15.64:0/625366627 learned_addr learned my addr 172.21.15.64:0/625366627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c194a0 0x559ac8c19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c19dc0 0x559ac8c24140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1b2ffd640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe1ac0a33b0 0x7fe1ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 -- 172.21.15.64:0/625366627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c19dc0 msgr2=0x559ac8c24140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c19dc0 0x559ac8c24140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 -- 172.21.15.64:0/625366627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c194a0 msgr2=0x559ac8c19880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c6562640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe1ac0a33b0 0x7fe1ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c194a0 0x559ac8c19880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 -- 172.21.15.64:0/625366627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac8b0b440 con 0x559ac8c24680 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1b2ffd640 1 -- 172.21.15.64:0/1281633341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe1b80cf210 con 0x7fe1c0106f10 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c5560640 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 0x559ac8c26a70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc01b910 tx=0x7fe1bc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e04acafb1b97253 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1abfff640 1 -- 172.21.15.64:0/625366627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc01b240 con 0x559ac8c24680 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1abfff640 1 -- 172.21.15.64:0/625366627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1bc021070 con 0x559ac8c24680 2024-09-19T05:10:27.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1abfff640 1 -- 172.21.15.64:0/625366627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc01a8c0 con 0x559ac8c24680 2024-09-19T05:10:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.995+0000 7fe1c6562640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe1ac0a33b0 0x7fe1ac0a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80ccf60 tx=0x7fe1b8004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 -- 172.21.15.64:0/625366627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 msgr2=0x559ac8c26a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 0x559ac8c26a70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc01b910 tx=0x7fe1bc00b420 comp rx=0 tx=0).stop 2024-09-19T05:10:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 -- 172.21.15.64:0/625366627 shutdown_connections 2024-09-19T05:10:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c24680 0x559ac8c26a70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c19dc0 0x559ac8c24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/625366627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c194a0 0x559ac8c19880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 -- 172.21.15.64:0/625366627 >> 172.21.15.64:0/625366627 conn(0x559ac8c15a30 msgr2=0x559ac8c15e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 -- 172.21.15.64:0/625366627 shutdown_connections 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.996+0000 7fe1ca39e740 1 -- 172.21.15.64:0/625366627 wait complete. 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 -- start start 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 Processor -- start 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c24010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559ac8c27250 con 0x559ac8c19dc0 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ac8b3a050 con 0x559ac8c24680 2024-09-19T05:10:27.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ac8c273d0 con 0x559ac8c194a0 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1ca39e740 10 client.0 _ll_register_callbacks cb 0x559ac8b7a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55992/0 (socket says 172.21.15.64:55992) 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1c5d61640 1 -- 172.21.15.64:0/1511375880 learned_addr learned my addr 172.21.15.64:0/1511375880 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.997+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c24010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c21840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5d61640 1 -- 172.21.15.64:0/1511375880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 msgr2=0x559ac8c24010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c24010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5d61640 1 -- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 msgr2=0x559ac8c21840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c21840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5d61640 1 -- 172.21.15.64:0/1511375880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac88f03c0 con 0x559ac8c19dc0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c21840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c212c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe1b402a930 tx=0x7fe1b40319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32bc62ab112d35a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1a97fa640 1 -- 172.21.15.64:0/1511375880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4035040 con 0x559ac8c19dc0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1bc021070 con 0x559ac8c19dc0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1a97fa640 1 -- 172.21.15.64:0/1511375880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe1b4004400 con 0x559ac8c19dc0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1a97fa640 1 -- 172.21.15.64:0/1511375880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe1b4005030 con 0x559ac8c19dc0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1ca39e740 10 client.16467 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.998+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ac8c32c90 con 0x559ac8c19dc0 2024-09-19T05:10:27.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1a97fa640 1 -- 172.21.15.64:0/1511375880 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1bc021070 con 0x559ac8c19dc0 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1a97fa640 1 -- 172.21.15.64:0/1511375880 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe1b40035f0 con 0x559ac8c19dc0 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1ca39e740 10 client.16467 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1ca39e740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1ca39e740 20 client.16467 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c340a0 0x559ac8c36500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1ca39e740 4 client.16467 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ac8b8c850 con 0x559ac8c340a0 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:26.999+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c340a0 0x559ac8c36500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1a97fa640 10 client.16467 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:27.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1c5560640 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c340a0 0x559ac8c36500 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9691acce13e11 server_cookie=9186605cd429d2b3 in_seq=0 out_seq=0 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1a97fa640 1 -- 172.21.15.64:0/1511375880 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ac8b8c850 con 0x559ac8c340a0 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1a97fa640 10 client.16467 handle_command_reply: tid=0 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1ca39e740 1 client.16467 shutdown 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 msgr2=0x559ac8c212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.000+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c212c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe1b402a930 tx=0x7fe1b40319e0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 shutdown_connections 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c21840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c340a0 0x559ac8c36500 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c212c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/1511375880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c24010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1a97fa640 0 client.16467 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 >> 172.21.15.64:0/1511375880 conn(0x559ac8c15a30 msgr2=0x559ac8c1b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 shutdown_connections 2024-09-19T05:10:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 1 -- 172.21.15.64:0/1511375880 wait complete. 2024-09-19T05:10:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.001+0000 7fe1ca39e740 20 client.16467 trim_cache size 0 max 16384 2024-09-19T05:10:27.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 Processor -- start 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 -- start start 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c3bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c3c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01adfa0 con 0x559ac8c19dc0 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01ab870 con 0x559ac8c24680 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01a96c0 con 0x559ac8c194a0 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c3bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c3bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56008/0 (socket says 172.21.15.64:56008) 2024-09-19T05:10:27.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.018+0000 7fe1c5d61640 1 -- 172.21.15.64:0/3217759250 learned_addr learned my addr 172.21.15.64:0/3217759250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5560640 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5560640 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c3c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5d61640 1 -- 172.21.15.64:0/3217759250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 msgr2=0x559ac8c304b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c304b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5d61640 1 -- 172.21.15.64:0/3217759250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 msgr2=0x559ac8c3c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c3c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5d61640 1 -- 172.21.15.64:0/3217759250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac8c19bf0 con 0x559ac8c19dc0 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5560640 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c3bdd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4006010 tx=0x7fe1b402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4da6f074e11bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1ab7fe640 1 -- 172.21.15.64:0/3217759250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4035030 con 0x559ac8c19dc0 2024-09-19T05:10:27.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1ab7fe640 1 -- 172.21.15.64:0/3217759250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b40037f0 con 0x559ac8c19dc0 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.019+0000 7fe1ab7fe640 1 -- 172.21.15.64:0/3217759250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4053070 con 0x559ac8c19dc0 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 -- 172.21.15.64:0/3217759250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 msgr2=0x559ac8c3bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c3bdd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4006010 tx=0x7fe1b402a410 comp rx=0 tx=0).stop 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 -- 172.21.15.64:0/3217759250 shutdown_connections 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c3bdd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/3217759250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 -- 172.21.15.64:0/3217759250 >> 172.21.15.64:0/3217759250 conn(0x559ac8c15a30 msgr2=0x559ac8c1c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:27.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 -- 172.21.15.64:0/3217759250 shutdown_connections 2024-09-19T05:10:27.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 -- 172.21.15.64:0/3217759250 wait complete. 2024-09-19T05:10:27.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 -- start start 2024-09-19T05:10:27.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.020+0000 7fe1ca39e740 1 Processor -- start 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c2ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c28160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c28700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01a9d30 con 0x559ac8c19dc0 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01ab1a0 con 0x559ac8c24680 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01aa400 con 0x559ac8c194a0 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 10 client.0 _ll_register_callbacks cb 0x7fe1b4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c28160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c28160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56022/0 (socket says 172.21.15.64:56022) 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 -- 172.21.15.64:0/566557645 learned_addr learned my addr 172.21.15.64:0/566557645 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5560640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c2ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5560640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c28700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 -- 172.21.15.64:0/566557645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 msgr2=0x559ac8c2ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c2ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 -- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 msgr2=0x559ac8c28700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c28700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 -- 172.21.15.64:0/566557645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac8c28f30 con 0x559ac8c19dc0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1c5d61640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c28160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40058c0 tx=0x7fe1b4032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf70e2615ab3ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1a97fa640 1 -- 172.21.15.64:0/566557645 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4053070 con 0x559ac8c19dc0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1a97fa640 1 -- 172.21.15.64:0/566557645 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe1b4004480 con 0x559ac8c19dc0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.021+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b40037f0 con 0x559ac8c19dc0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1a97fa640 1 -- 172.21.15.64:0/566557645 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe1b4035040 con 0x559ac8c19dc0 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 10 client.16473 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:27.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ac8c9edf0 con 0x559ac8c19dc0 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1a97fa640 1 -- 172.21.15.64:0/566557645 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b40037f0 con 0x559ac8c19dc0 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1a97fa640 1 -- 172.21.15.64:0/566557645 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe1b405e030 con 0x559ac8c19dc0 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 20 client.16473 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c9f3b0 0x559ac8ca17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 4 client.16473 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ac8b86d60 con 0x559ac8c9f3b0 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.022+0000 7fe1c5560640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c9f3b0 0x559ac8ca17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1c5560640 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c9f3b0 0x559ac8ca17d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa00a18376fef25 server_cookie=a699a2b1c6ac93e3 in_seq=0 out_seq=0 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1a97fa640 10 client.16473 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1a97fa640 1 -- 172.21.15.64:0/566557645 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ac8b86d60 con 0x559ac8c9f3b0 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1a97fa640 10 client.16473 handle_command_reply: tid=0 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 client.16473 shutdown 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 msgr2=0x559ac8c28160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c28160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40058c0 tx=0x7fe1b4032f60 comp rx=0 tx=0).stop 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 shutdown_connections 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ac8c24680 0x559ac8c28700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559ac8c9f3b0 0x559ac8ca17d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559ac8c19dc0 0x559ac8c28160 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 --2- 172.21.15.64:0/566557645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ac8c194a0 0x559ac8c2ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 >> 172.21.15.64:0/566557645 conn(0x559ac8c15a30 msgr2=0x559ac8c22b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1a97fa640 0 client.16473 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 shutdown_connections 2024-09-19T05:10:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 1 -- 172.21.15.64:0/566557645 wait complete. 2024-09-19T05:10:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.023+0000 7fe1ca39e740 20 client.16473 trim_cache size 0 max 16384 2024-09-19T05:10:27.034 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:10:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:10:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:10:27.034 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe1ac0a33b0 msgr2=0x7fe1ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe1ac0a33b0 0x7fe1ac0a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80ccf60 tx=0x7fe1b8004a40 comp rx=0 tx=0).stop 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 msgr2=0x7fe1c019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 0x7fe1c019cd80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800b970 tx=0x7fe1b800be30 comp rx=0 tx=0).stop 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 shutdown_connections 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe1ac0a33b0 0x7fe1ac0a5870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1c0106f10 0x7fe1c019cd80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1c0102790 0x7fe1c01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 --2- 172.21.15.64:0/1281633341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c0101de0 0x7fe1c01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 >> 172.21.15.64:0/1281633341 conn(0x7fe1c00fd8c0 msgr2=0x7fe1c00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 shutdown_connections 2024-09-19T05:10:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:27.024+0000 7fe1c7fc5640 1 -- 172.21.15.64:0/1281633341 wait complete. 2024-09-19T05:10:27.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:27 smithi064 ceph-mon[30960]: pgmap v139: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:27.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:27 smithi174 ceph-mon[34488]: pgmap v139: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:27.891 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:10:27.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:10:27.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:27 smithi103 ceph-mon[34446]: pgmap v139: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:28.154 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:28.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 -- 172.21.15.64:0/2624263637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe81000d0 msgr2=0x7fcfe8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 --2- 172.21.15.64:0/2624263637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe81000d0 0x7fcfe8100530 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009900 tx=0x7fcfdc02a410 comp rx=0 tx=0).stop 2024-09-19T05:10:28.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 -- 172.21.15.64:0/2624263637 shutdown_connections 2024-09-19T05:10:28.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 --2- 172.21.15.64:0/2624263637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe8100a70 0x7fcfe810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 --2- 172.21.15.64:0/2624263637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe81000d0 0x7fcfe8100530 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 --2- 172.21.15.64:0/2624263637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 -- 172.21.15.64:0/2624263637 >> 172.21.15.64:0/2624263637 conn(0x7fcfe8077210 msgr2=0x7fcfe80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 -- 172.21.15.64:0/2624263637 shutdown_connections 2024-09-19T05:10:28.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.618+0000 7fcfecebf640 1 -- 172.21.15.64:0/2624263637 wait complete. 2024-09-19T05:10:28.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 Processor -- start 2024-09-19T05:10:28.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 -- start start 2024-09-19T05:10:28.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe81000d0 0x7fcfe81012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe8100a70 0x7fcfe81017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8111f50 con 0x7fcfe8106ba0 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8111dd0 con 0x7fcfe81000d0 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfecebf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81120d0 con 0x7fcfe8100a70 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.619+0000 7fcfe6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56040/0 (socket says 172.21.15.64:56040) 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 -- 172.21.15.64:0/3063305519 learned_addr learned my addr 172.21.15.64:0/3063305519 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe5d9b640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe8100a70 0x7fcfe81017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe659c640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe81000d0 0x7fcfe81012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 -- 172.21.15.64:0/3063305519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe8100a70 msgr2=0x7fcfe81017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe8100a70 0x7fcfe81017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 -- 172.21.15.64:0/3063305519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe81000d0 msgr2=0x7fcfe81012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe81000d0 0x7fcfe81012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 -- 172.21.15.64:0/3063305519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe81a68e0 con 0x7fcfe8106ba0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe659c640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe81000d0 0x7fcfe81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfe6d9d640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8101d30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000eaa0 tx=0x7fcfe000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000cdb0 con 0x7fcfe8106ba0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfe0005020 con 0x7fcfe8106ba0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81a6bd0 con 0x7fcfe8106ba0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00116d0 con 0x7fcfe8106ba0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.620+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe81a7090 con 0x7fcfe8106ba0 2024-09-19T05:10:28.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.621+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8100530 con 0x7fcfe8106ba0 2024-09-19T05:10:28.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.622+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fcfe00051d0 con 0x7fcfe8106ba0 2024-09-19T05:10:28.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.623+0000 7fcfd77fe640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcfd00a33b0 0x7fcfd00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.623+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fcfe00cff40 con 0x7fcfe8106ba0 2024-09-19T05:10:28.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.623+0000 7fcfe659c640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcfd00a33b0 0x7fcfd00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.624+0000 7fcfe659c640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcfd00a33b0 0x7fcfd00a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005fd0 tx=0x7fcfd800b5d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.627+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcfe0016030 con 0x7fcfe8106ba0 2024-09-19T05:10:28.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.839+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfe810a090 con 0x7fcfe8106ba0 2024-09-19T05:10:28.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.840+0000 7fcfd77fe640 1 -- 172.21.15.64:0/3063305519 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fcfe009c920 con 0x7fcfe8106ba0 2024-09-19T05:10:28.844 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:10:28.844 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:10:28.844 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.843+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcfd00a33b0 msgr2=0x7fcfd00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.843+0000 7fcfecebf640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcfd00a33b0 0x7fcfd00a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005fd0 tx=0x7fcfd800b5d0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.843+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 msgr2=0x7fcfe8101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.843+0000 7fcfecebf640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8101d30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000eaa0 tx=0x7fcfe000ef60 comp rx=0 tx=0).stop 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 shutdown_connections 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fcfd00a33b0 0x7fcfd00a5870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfe8106ba0 0x7fcfe8101d30 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe8100a70 0x7fcfe81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 --2- 172.21.15.64:0/3063305519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfe81000d0 0x7fcfe81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 >> 172.21.15.64:0/3063305519 conn(0x7fcfe8077210 msgr2=0x7fcfe8105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 shutdown_connections 2024-09-19T05:10:28.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:28.844+0000 7fcfecebf640 1 -- 172.21.15.64:0/3063305519 wait complete. 2024-09-19T05:10:29.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:10:29.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:29 smithi174 ceph-mon[34488]: pgmap v140: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:29.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:29 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3063305519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:29.833 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:29.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:29 smithi064 ceph-mon[30960]: pgmap v140: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:29.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:29 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3063305519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:29.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:29 smithi103 ceph-mon[34446]: pgmap v140: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:29.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:29 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3063305519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:30.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 -- 172.21.15.64:0/285252734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0102790 msgr2=0x7f4db01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/285252734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0102790 0x7f4db01067e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009900 tx=0x7f4da402a410 comp rx=0 tx=0).stop 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 -- 172.21.15.64:0/285252734 shutdown_connections 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/285252734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db0106f10 0x7f4db010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/285252734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0102790 0x7f4db01067e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/285252734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0101de0 0x7f4db01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 -- 172.21.15.64:0/285252734 >> 172.21.15.64:0/285252734 conn(0x7f4db00fd8c0 msgr2=0x7f4db00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 -- 172.21.15.64:0/285252734 shutdown_connections 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.310+0000 7f4db5e6b640 1 -- 172.21.15.64:0/285252734 wait complete. 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.311+0000 7f4db5e6b640 1 Processor -- start 2024-09-19T05:10:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.311+0000 7f4db5e6b640 1 -- start start 2024-09-19T05:10:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.311+0000 7f4db5e6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db0101de0 0x7f4db019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.311+0000 7f4db5e6b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0102790 0x7f4db019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.311+0000 7f4db5e6b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 0x7f4db019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4db5e6b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0111e20 con 0x7f4db0106f10 2024-09-19T05:10:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4db5e6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0111ca0 con 0x7f4db0101de0 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4db5e6b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0111fa0 con 0x7f4db0102790 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 0x7f4db019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 0x7f4db019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56058/0 (socket says 172.21.15.64:56058) 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 -- 172.21.15.64:0/3060230542 learned_addr learned my addr 172.21.15.64:0/3060230542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db0101de0 0x7f4db019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0102790 0x7f4db019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 -- 172.21.15.64:0/3060230542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0102790 msgr2=0x7f4db019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0102790 0x7f4db019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 -- 172.21.15.64:0/3060230542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db0101de0 msgr2=0x7f4db019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db0101de0 0x7f4db019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 -- 172.21.15.64:0/3060230542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db019f550 con 0x7f4db0106f10 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0102790 0x7f4db019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:30.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4daffff640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 0x7f4db019ee70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4da800cce0 tx=0x7f4da800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4dacff9640 1 -- 172.21.15.64:0/3060230542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da800f450 con 0x7f4db0106f10 2024-09-19T05:10:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4dacff9640 1 -- 172.21.15.64:0/3060230542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da800faa0 con 0x7f4db0106f10 2024-09-19T05:10:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db019f840 con 0x7f4db0106f10 2024-09-19T05:10:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.312+0000 7f4dacff9640 1 -- 172.21.15.64:0/3060230542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8018b70 con 0x7f4db0106f10 2024-09-19T05:10:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.313+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a7120 con 0x7f4db0106f10 2024-09-19T05:10:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4dacff9640 1 -- 172.21.15.64:0/3060230542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4da8018d10 con 0x7f4db0106f10 2024-09-19T05:10:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 Processor -- start 2024-09-19T05:10:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 -- start start 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 0x55ef527c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 0x55ef527d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ef5272a2f0 con 0x55ef527c5b40 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef526d1cf0 con 0x55ef527cf370 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.315+0000 7f4db8244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef527262b0 con 0x55ef527c45d0 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 0x55ef527c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 0x55ef527c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56062/0 (socket says 172.21.15.64:56062) 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 -- 172.21.15.64:0/3452513491 learned_addr learned my addr 172.21.15.64:0/3452513491 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4dacff9640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4d980a3430 0x7f4d980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daffff640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4d980a3430 0x7f4d980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 -- 172.21.15.64:0/3452513491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 msgr2=0x55ef527c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 -- 172.21.15.64:0/3452513491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 msgr2=0x55ef527d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 0x55ef527d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 -- 172.21.15.64:0/3452513491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef526b63a0 con 0x55ef527c5b40 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4dacff9640 1 -- 172.21.15.64:0/3060230542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4da80d0070 con 0x7f4db0106f10 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 0x55ef527c4090 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005940 tx=0x7f4da4031ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b305fcc6d017ab59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4d95ffb640 1 -- 172.21.15.64:0/3452513491 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403f070 con 0x55ef527c5b40 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.316+0000 7f4daffff640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4d980a3430 0x7f4d980a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4da80cddc0 tx=0x7f4da80baf90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4d95ffb640 1 -- 172.21.15.64:0/3452513491 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da4031760 con 0x55ef527c5b40 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4d95ffb640 1 -- 172.21.15.64:0/3452513491 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4035070 con 0x55ef527c5b40 2024-09-19T05:10:30.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 -- 172.21.15.64:0/3452513491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 msgr2=0x55ef527c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 0x55ef527c4090 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005940 tx=0x7f4da4031ed0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 -- 172.21.15.64:0/3452513491 shutdown_connections 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 0x55ef527d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 --2- 172.21.15.64:0/3452513491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c5b40 0x55ef527c4090 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 -- 172.21.15.64:0/3452513491 >> 172.21.15.64:0/3452513491 conn(0x55ef5278d310 msgr2=0x55ef527c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 -- 172.21.15.64:0/3452513491 shutdown_connections 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.317+0000 7f4db8244740 1 -- 172.21.15.64:0/3452513491 wait complete. 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 -- start start 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 Processor -- start 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c45d0 0x55ef527ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef527cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ef527d2010 con 0x55ef527cf370 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef526e5060 con 0x55ef527c45d0 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef527d2190 con 0x55ef527c5b40 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4db8244740 10 client.0 _ll_register_callbacks cb 0x55ef524388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef527cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef527cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56078/0 (socket says 172.21.15.64:56078) 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4daeffd640 1 -- 172.21.15.64:0/3040137031 learned_addr learned my addr 172.21.15.64:0/3040137031 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c45d0 0x55ef527ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.318+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daeffd640 1 -- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 msgr2=0x55ef527cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daeffd640 1 -- 172.21.15.64:0/3040137031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c45d0 msgr2=0x55ef527ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c45d0 0x55ef527ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daeffd640 1 -- 172.21.15.64:0/3040137031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef5249b3c0 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4daeffd640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef527cc5d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4da4031a70 tx=0x7f4da40054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c085ba3018a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4d8effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4d8effd640 1 -- 172.21.15.64:0/3040137031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403f070 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4d8effd640 1 -- 172.21.15.64:0/3040137031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4da4004260 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da4031760 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4d8effd640 1 -- 172.21.15.64:0/3040137031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4da4057050 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4db8244740 10 client.16491 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef527dda90 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.319+0000 7f4d8effd640 1 -- 172.21.15.64:0/3040137031 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da4031760 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4d8effd640 1 -- 172.21.15.64:0/3040137031 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4da40037f0 con 0x55ef527cf370 2024-09-19T05:10:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4db8244740 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4db8244740 10 client.16491 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:10:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4db8244740 20 client.16491 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4db8244740 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef527dede0 0x55ef527e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4db8244740 4 client.16491 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef52737860 con 0x55ef527dede0 2024-09-19T05:10:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef527dede0 0x55ef527e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef527dede0 0x55ef527e11d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b1a3fb244302525 server_cookie=2893ab09cb35fa2a in_seq=0 out_seq=0 2024-09-19T05:10:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.320+0000 7f4d8effd640 10 client.16491 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:10:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4d8effd640 1 -- 172.21.15.64:0/3040137031 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef52737860 con 0x55ef527dede0 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4d8effd640 10 client.16491 handle_command_reply: tid=0 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4db8244740 1 client.16491 shutdown 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 msgr2=0x55ef527cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4db8244740 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef527cc5d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4da4031a70 tx=0x7f4da40054e0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 shutdown_connections 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.321+0000 7f4db8244740 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef527dede0 0x55ef527e11d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef527cc5d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 1 --2- 172.21.15.64:0/3040137031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c45d0 0x55ef527ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 >> 172.21.15.64:0/3040137031 conn(0x55ef5278d310 msgr2=0x55ef527c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4d8effd640 0 client.16491 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:10:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 shutdown_connections 2024-09-19T05:10:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 1 -- 172.21.15.64:0/3040137031 wait complete. 2024-09-19T05:10:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.322+0000 7f4db8244740 20 client.16491 trim_cache size 0 max 16384 2024-09-19T05:10:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 Processor -- start 2024-09-19T05:10:30.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 -- start start 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 0x55ef527e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 0x55ef527c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b2160 con 0x55ef527c45d0 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01afa30 con 0x55ef527cf370 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4db8244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ad880 con 0x55ef527c5b40 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 0x55ef527e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 0x55ef527e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56082/0 (socket says 172.21.15.64:56082) 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 -- 172.21.15.64:0/1055975459 learned_addr learned my addr 172.21.15.64:0/1055975459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 -- 172.21.15.64:0/1055975459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 msgr2=0x55ef527e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527e4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 -- 172.21.15.64:0/1055975459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 msgr2=0x55ef527c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 0x55ef527c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.335+0000 7f4daeffd640 1 -- 172.21.15.64:0/1055975459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef527ce970 con 0x55ef527c45d0 2024-09-19T05:10:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4daeffd640 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 0x55ef527e3e10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4da403b4c0 tx=0x7f4da4005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f1adca7bcd8f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4d957fa640 1 -- 172.21.15.64:0/1055975459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403f030 con 0x55ef527c45d0 2024-09-19T05:10:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4d957fa640 1 -- 172.21.15.64:0/1055975459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da4003de0 con 0x55ef527c45d0 2024-09-19T05:10:30.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4d957fa640 1 -- 172.21.15.64:0/1055975459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4035040 con 0x55ef527c45d0 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 -- 172.21.15.64:0/1055975459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 msgr2=0x55ef527e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 0x55ef527e3e10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4da403b4c0 tx=0x7f4da4005910 comp rx=0 tx=0).stop 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 -- 172.21.15.64:0/1055975459 shutdown_connections 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527cf370 0x55ef527c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c5b40 0x55ef527e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 --2- 172.21.15.64:0/1055975459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527c45d0 0x55ef527e3e10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 -- 172.21.15.64:0/1055975459 >> 172.21.15.64:0/1055975459 conn(0x55ef5278d310 msgr2=0x55ef527cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:30.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 -- 172.21.15.64:0/1055975459 shutdown_connections 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.336+0000 7f4db8244740 1 -- 172.21.15.64:0/1055975459 wait complete. 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 -- start start 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 Processor -- start 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 0x55ef527e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef52847710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01adef0 con 0x55ef527cf370 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01af360 con 0x55ef527c5b40 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ae5c0 con 0x55ef527c45d0 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4db8244740 10 client.0 _ll_register_callbacks cb 0x7f4da4031e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4daf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 0x55ef527e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef52847710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4daf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 0x55ef527e1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:45264/0 (socket says 172.21.15.64:45264) 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.337+0000 7f4daf7fe640 1 -- 172.21.15.64:0/750048593 learned_addr learned my addr 172.21.15.64:0/750048593 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daeffd640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daf7fe640 1 -- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 msgr2=0x55ef527e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daf7fe640 1 -- 172.21.15.64:0/750048593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 msgr2=0x55ef52847710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef52847710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daf7fe640 1 -- 172.21.15.64:0/750048593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef52847c50 con 0x55ef527c5b40 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daeffd640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef52847710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4daf7fe640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 0x55ef527e1780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ef527c1ac0 tx=0x7f4da0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ef18e0fadaba67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4d8effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4d8effd640 1 -- 172.21.15.64:0/750048593 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da001d880 con 0x55ef527c5b40 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da4003de0 con 0x55ef527c5b40 2024-09-19T05:10:30.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4d8effd640 1 -- 172.21.15.64:0/750048593 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f4da001da20 con 0x55ef527c5b40 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4d8effd640 1 -- 172.21.15.64:0/750048593 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4da002c050 con 0x55ef527c5b40 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4db8244740 10 client.26090 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.338+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef52848f40 con 0x55ef527c5b40 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4d8effd640 1 -- 172.21.15.64:0/750048593 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da4003de0 con 0x55ef527c5b40 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4d8effd640 1 -- 172.21.15.64:0/750048593 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4da000f420 con 0x55ef527c5b40 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4db8244740 10 client.26090 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4db8244740 10 client.26090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4db8244740 20 client.26090 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4db8244740 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef528496f0 0x55ef5284bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4db8244740 4 client.26090 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef52731d70 con 0x55ef528496f0 2024-09-19T05:10:30.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.339+0000 7f4daeffd640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef528496f0 0x55ef5284bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4d8effd640 10 client.26090 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4daeffd640 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef528496f0 0x55ef5284bb10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2990b3ca0fd50453 server_cookie=6f4184152f88a76f in_seq=0 out_seq=0 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4d8effd640 1 -- 172.21.15.64:0/750048593 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef52731d70 con 0x55ef528496f0 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4d8effd640 10 client.26090 handle_command_reply: tid=0 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 client.26090 shutdown 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 msgr2=0x55ef527e1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 0x55ef527e1780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ef527c1ac0 tx=0x7f4da0011810 comp rx=0 tx=0).stop 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 shutdown_connections 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55ef528496f0 0x55ef5284bb10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ef527cf370 0x55ef52847710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef527c5b40 0x55ef527e1780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4d8effd640 0 client.26090 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 --2- 172.21.15.64:0/750048593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef527c45d0 0x55ef527e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 >> 172.21.15.64:0/750048593 conn(0x55ef5278d310 msgr2=0x55ef527cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 shutdown_connections 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 1 -- 172.21.15.64:0/750048593 wait complete. 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.340+0000 7f4db8244740 20 client.26090 trim_cache size 0 max 16384 2024-09-19T05:10:30.349 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:10:30.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4d980a3430 msgr2=0x7f4d980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4d980a3430 0x7f4d980a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4da80cddc0 tx=0x7f4da80baf90 comp rx=0 tx=0).stop 2024-09-19T05:10:30.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 msgr2=0x7f4db019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:30.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 0x7f4db019ee70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4da800cce0 tx=0x7f4da800a620 comp rx=0 tx=0).stop 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 shutdown_connections 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f4d980a3430 0x7f4d980a58f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4db0106f10 0x7f4db019ee70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4db0102790 0x7f4db019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 --2- 172.21.15.64:0/3060230542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4db0101de0 0x7f4db019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 >> 172.21.15.64:0/3060230542 conn(0x7f4db00fd8c0 msgr2=0x7f4db00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 shutdown_connections 2024-09-19T05:10:30.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:30.341+0000 7f4db5e6b640 1 -- 172.21.15.64:0/3060230542 wait complete. 2024-09-19T05:10:31.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:10:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:31.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:31 smithi064 ceph-mon[30960]: pgmap v141: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:31.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:31 smithi174 ceph-mon[34488]: pgmap v141: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 -- 172.21.15.64:0/963089402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444100ca0 msgr2=0x7f8444106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 --2- 172.21.15.64:0/963089402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444100ca0 0x7f8444106840 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8438009930 tx=0x7f843802a420 comp rx=0 tx=0).stop 2024-09-19T05:10:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 -- 172.21.15.64:0/963089402 shutdown_connections 2024-09-19T05:10:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 --2- 172.21.15.64:0/963089402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 0x7f844410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 --2- 172.21.15.64:0/963089402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8444100ca0 0x7f8444106840 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 --2- 172.21.15.64:0/963089402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84441002f0 0x7f84441006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 -- 172.21.15.64:0/963089402 >> 172.21.15.64:0/963089402 conn(0x7f8444077210 msgr2=0x7f84440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 -- 172.21.15.64:0/963089402 shutdown_connections 2024-09-19T05:10:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.818+0000 7f844a99a640 1 -- 172.21.15.64:0/963089402 wait complete. 2024-09-19T05:10:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 Processor -- start 2024-09-19T05:10:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 -- start start 2024-09-19T05:10:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 0x7f8444101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8444100ca0 0x7f84441017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 0x7f8444101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8444111e50 con 0x7f84441002f0 2024-09-19T05:10:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8444111cd0 con 0x7f8444106f50 2024-09-19T05:10:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f844a99a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8444111fd0 con 0x7f8444100ca0 2024-09-19T05:10:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.819+0000 7f8443fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 0x7f8444101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 0x7f8444101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56110/0 (socket says 172.21.15.64:56110) 2024-09-19T05:10:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 -- 172.21.15.64:0/2887890184 learned_addr learned my addr 172.21.15.64:0/2887890184 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f84437fe640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8444100ca0 0x7f84441017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8448f37640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 0x7f8444101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 -- 172.21.15.64:0/2887890184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8444100ca0 msgr2=0x7f84441017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8444100ca0 0x7f84441017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 -- 172.21.15.64:0/2887890184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 msgr2=0x7f8444101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 0x7f8444101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 -- 172.21.15.64:0/2887890184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441a68b0 con 0x7f84441002f0 2024-09-19T05:10:31.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8448f37640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 0x7f8444101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:31.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f8443fff640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 0x7f8444101270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f843400c970 tx=0x7f843400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f84417fa640 1 -- 172.21.15.64:0/2887890184 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434007bb0 con 0x7f84441002f0 2024-09-19T05:10:31.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f84417fa640 1 -- 172.21.15.64:0/2887890184 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8434007d50 con 0x7f84441002f0 2024-09-19T05:10:31.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84441a6ba0 con 0x7f84441002f0 2024-09-19T05:10:31.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f84417fa640 1 -- 172.21.15.64:0/2887890184 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400f450 con 0x7f84441002f0 2024-09-19T05:10:31.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.820+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84441a7060 con 0x7f84441002f0 2024-09-19T05:10:31.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.823+0000 7f84417fa640 1 -- 172.21.15.64:0/2887890184 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8434004f20 con 0x7f84441002f0 2024-09-19T05:10:31.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.823+0000 7f844cd73740 1 Processor -- start 2024-09-19T05:10:31.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.823+0000 7f844cd73740 1 -- start start 2024-09-19T05:10:31.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f844cd73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856eb40 0x55bdc856d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f844cd73740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 0x55bdc856da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f844cd73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc857a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f844cd73740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bdc84d32f0 con 0x55bdc856d5d0 2024-09-19T05:10:31.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f844cd73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bdc847acf0 con 0x55bdc8578370 2024-09-19T05:10:31.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f844cd73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bdc84cf2b0 con 0x55bdc856eb40 2024-09-19T05:10:31.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 0x55bdc856da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 0x55bdc856da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56112/0 (socket says 172.21.15.64:56112) 2024-09-19T05:10:31.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 -- 172.21.15.64:0/4155770453 learned_addr learned my addr 172.21.15.64:0/4155770453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:31.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f84437fe640 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc857a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 -- 172.21.15.64:0/4155770453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856eb40 msgr2=0x55bdc856d090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:10:31.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856eb40 0x55bdc856d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 -- 172.21.15.64:0/4155770453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 msgr2=0x55bdc857a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc857a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 -- 172.21.15.64:0/4155770453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc845f3a0 con 0x55bdc856d5d0 2024-09-19T05:10:31.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f84417fa640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f842c0a33b0 0x7f842c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8448f37640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f842c0a33b0 0x7f842c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.824+0000 7f8443fff640 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 0x55bdc856da10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f84340cd170 tx=0x7f84340d85f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134f3da6a1745b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f84417fa640 1 -- 172.21.15.64:0/2887890184 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f84340cfab0 con 0x7f84441002f0 2024-09-19T05:10:31.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f842a7fc640 1 -- 172.21.15.64:0/4155770453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843401d020 con 0x55bdc856d5d0 2024-09-19T05:10:31.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f842a7fc640 1 -- 172.21.15.64:0/4155770453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84340d5070 con 0x55bdc856d5d0 2024-09-19T05:10:31.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f842a7fc640 1 -- 172.21.15.64:0/4155770453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843409b8d0 con 0x55bdc856d5d0 2024-09-19T05:10:31.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f844cd73740 1 -- 172.21.15.64:0/4155770453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 msgr2=0x55bdc856da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f8448f37640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f842c0a33b0 0x7f842c0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f843c008c60 tx=0x7f843c008230 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.825+0000 7f844cd73740 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 0x55bdc856da10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f84340cd170 tx=0x7f84340d85f0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 -- 172.21.15.64:0/4155770453 shutdown_connections 2024-09-19T05:10:31.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc857a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856d5d0 0x55bdc856da10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 --2- 172.21.15.64:0/4155770453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856eb40 0x55bdc856d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 -- 172.21.15.64:0/4155770453 >> 172.21.15.64:0/4155770453 conn(0x55bdc8536310 msgr2=0x55bdc8569c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 -- 172.21.15.64:0/4155770453 shutdown_connections 2024-09-19T05:10:31.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 -- 172.21.15.64:0/4155770453 wait complete. 2024-09-19T05:10:31.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 -- start start 2024-09-19T05:10:31.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 Processor -- start 2024-09-19T05:10:31.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8577d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.826+0000 7f844cd73740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8575010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f844cd73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8575600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f844cd73740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bdc857b070 con 0x55bdc856eb40 2024-09-19T05:10:31.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f844cd73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bdc848e060 con 0x55bdc856d5d0 2024-09-19T05:10:31.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f844cd73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bdc857b1f0 con 0x55bdc8578370 2024-09-19T05:10:31.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f844cd73740 10 client.0 _ll_register_callbacks cb 0x55bdc81e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:31.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8575010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f8443fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8577d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8575010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56116/0 (socket says 172.21.15.64:56116) 2024-09-19T05:10:31.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 -- 172.21.15.64:0/2234242686 learned_addr learned my addr 172.21.15.64:0/2234242686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:31.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f8443fff640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8577d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:45302/0 (socket says 172.21.15.64:45302) 2024-09-19T05:10:31.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 -- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 msgr2=0x55bdc8575600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:10:31.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8575600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 -- 172.21.15.64:0/2234242686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 msgr2=0x55bdc8577d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8577d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 -- 172.21.15.64:0/2234242686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc82443c0 con 0x55bdc856eb40 2024-09-19T05:10:31.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f8443fff640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8577d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:31.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84437fe640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8575010 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8438009930 tx=0x7f84380316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0f46cd59a702f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84237fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:31.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.827+0000 7f84237fe640 1 -- 172.21.15.64:0/2234242686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438035040 con 0x55bdc856eb40 2024-09-19T05:10:31.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84340d5070 con 0x55bdc856eb40 2024-09-19T05:10:31.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f84237fe640 1 -- 172.21.15.64:0/2234242686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f8438004240 con 0x55bdc856eb40 2024-09-19T05:10:31.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f84237fe640 1 -- 172.21.15.64:0/2234242686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8438004440 con 0x55bdc856eb40 2024-09-19T05:10:31.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 10 client.16509 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:31.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdc8586b90 con 0x55bdc856eb40 2024-09-19T05:10:31.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f84237fe640 1 -- 172.21.15.64:0/2234242686 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84340d5070 con 0x55bdc856eb40 2024-09-19T05:10:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f84237fe640 1 -- 172.21.15.64:0/2234242686 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8438004cd0 con 0x55bdc856eb40 2024-09-19T05:10:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 10 client.16509 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 10 client.16509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:10:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 20 client.16509 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc8587fa0 0x55bdc858a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 4 client.16509 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.828+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdc84e0860 con 0x55bdc8587fa0 2024-09-19T05:10:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.829+0000 7f8443fff640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc8587fa0 0x55bdc858a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.829+0000 7f84237fe640 10 client.16509 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:10:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.829+0000 7f8443fff640 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc8587fa0 0x55bdc858a3f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca8cbea612416e3d server_cookie=10fa08685a943ac6 in_seq=0 out_seq=0 2024-09-19T05:10:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f84237fe640 1 -- 172.21.15.64:0/2234242686 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdc84e0860 con 0x55bdc8587fa0 2024-09-19T05:10:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f84237fe640 10 client.16509 handle_command_reply: tid=0 2024-09-19T05:10:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 client.16509 shutdown 2024-09-19T05:10:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 msgr2=0x55bdc8575010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8575010 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8438009930 tx=0x7f84380316d0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 shutdown_connections 2024-09-19T05:10:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8575600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc8587fa0 0x55bdc858a3f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8575010 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 --2- 172.21.15.64:0/2234242686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8577d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 >> 172.21.15.64:0/2234242686 conn(0x55bdc8536310 msgr2=0x55bdc856f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f84237fe640 0 client.16509 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:10:31.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 shutdown_connections 2024-09-19T05:10:31.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 1 -- 172.21.15.64:0/2234242686 wait complete. 2024-09-19T05:10:31.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.830+0000 7f844cd73740 20 client.16509 trim_cache size 0 max 16384 2024-09-19T05:10:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 Processor -- start 2024-09-19T05:10:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 -- start start 2024-09-19T05:10:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856d5d0 0x55bdc8584340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8570120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc8570660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84441b2380 con 0x55bdc856eb40 2024-09-19T05:10:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84441afc50 con 0x55bdc8578370 2024-09-19T05:10:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.843+0000 7f844cd73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84441adaa0 con 0x55bdc856d5d0 2024-09-19T05:10:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8570120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8570120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56130/0 (socket says 172.21.15.64:56130) 2024-09-19T05:10:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 -- 172.21.15.64:0/2851190844 learned_addr learned my addr 172.21.15.64:0/2851190844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8448f37640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc8570660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8443fff640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856d5d0 0x55bdc8584340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 -- 172.21.15.64:0/2851190844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856d5d0 msgr2=0x55bdc8584340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856d5d0 0x55bdc8584340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 -- 172.21.15.64:0/2851190844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 msgr2=0x55bdc8570660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc8570660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 -- 172.21.15.64:0/2851190844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc8573560 con 0x55bdc856eb40 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8443fff640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856d5d0 0x55bdc8584340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8448f37640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc8570660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f84437fe640 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8570120 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8438031a70 tx=0x7f84380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccd12ee83fa534b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8429ffb640 1 -- 172.21.15.64:0/2851190844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438035070 con 0x55bdc856eb40 2024-09-19T05:10:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8429ffb640 1 -- 172.21.15.64:0/2851190844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8438004150 con 0x55bdc856eb40 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.844+0000 7f8429ffb640 1 -- 172.21.15.64:0/2851190844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380047a0 con 0x55bdc856eb40 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 -- 172.21.15.64:0/2851190844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 msgr2=0x55bdc8570120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8570120 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8438031a70 tx=0x7f84380045e0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 -- 172.21.15.64:0/2851190844 shutdown_connections 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc8578370 0x55bdc8570660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8570120 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 --2- 172.21.15.64:0/2851190844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc856d5d0 0x55bdc8584340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 -- 172.21.15.64:0/2851190844 >> 172.21.15.64:0/2851190844 conn(0x55bdc8536310 msgr2=0x55bdc8574100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 -- 172.21.15.64:0/2851190844 shutdown_connections 2024-09-19T05:10:31.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 -- 172.21.15.64:0/2851190844 wait complete. 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 -- start start 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.845+0000 7f844cd73740 1 Processor -- start 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8580bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8571ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8572220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84441ae110 con 0x55bdc856eb40 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84441af580 con 0x55bdc856d5d0 2024-09-19T05:10:31.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84441ae7e0 con 0x55bdc8578370 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f844cd73740 10 client.0 _ll_register_callbacks cb 0x7f8438032b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8571ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8571ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56146/0 (socket says 172.21.15.64:56146) 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 -- 172.21.15.64:0/2262014868 learned_addr learned my addr 172.21.15.64:0/2262014868 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f8443fff640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8580bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f8448f37640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8572220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 -- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 msgr2=0x55bdc8572220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8572220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 -- 172.21.15.64:0/2262014868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 msgr2=0x55bdc8580bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8580bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 -- 172.21.15.64:0/2262014868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc8572a50 con 0x55bdc856eb40 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84437fe640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8571ce0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8438009850 tx=0x7f8438000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35f46e43ad482cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.846+0000 7f84237fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f84237fe640 1 -- 172.21.15.64:0/2262014868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438035040 con 0x55bdc856eb40 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f84237fe640 1 -- 172.21.15.64:0/2262014868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f84380047a0 con 0x55bdc856eb40 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8438004150 con 0x55bdc856eb40 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f84237fe640 1 -- 172.21.15.64:0/2262014868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84380049a0 con 0x55bdc856eb40 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 10 client.16515 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdc85f2b70 con 0x55bdc856eb40 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f84237fe640 1 -- 172.21.15.64:0/2262014868 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8438004150 con 0x55bdc856eb40 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f84237fe640 1 -- 172.21.15.64:0/2262014868 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8438003e20 con 0x55bdc856eb40 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 10 client.16515 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 10 client.16515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 20 client.16515 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc85f31b0 0x55bdc85f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 4 client.16515 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.847+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdc84dad70 con 0x55bdc85f31b0 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f8443fff640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc85f31b0 0x55bdc85f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f84237fe640 10 client.16515 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f8443fff640 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc85f31b0 0x55bdc85f55d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f2e69d20393190d server_cookie=241fc0d6b0dc29db in_seq=0 out_seq=0 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f84237fe640 1 -- 172.21.15.64:0/2262014868 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdc84dad70 con 0x55bdc85f31b0 2024-09-19T05:10:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f84237fe640 10 client.16515 handle_command_reply: tid=0 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 client.16515 shutdown 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 msgr2=0x55bdc8571ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8571ce0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8438009850 tx=0x7f8438000bc0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 shutdown_connections 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bdc8578370 0x55bdc8572220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55bdc85f31b0 0x55bdc85f55d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc856eb40 0x55bdc8571ce0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 --2- 172.21.15.64:0/2262014868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdc856d5d0 0x55bdc8580bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 >> 172.21.15.64:0/2262014868 conn(0x55bdc8536310 msgr2=0x55bdc8576870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.848+0000 7f84237fe640 0 client.16515 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 shutdown_connections 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844cd73740 1 -- 172.21.15.64:0/2262014868 wait complete. 2024-09-19T05:10:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844cd73740 20 client.16515 trim_cache size 0 max 16384 2024-09-19T05:10:31.880 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f842c0a33b0 msgr2=0x7f842c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f842c0a33b0 0x7f842c0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f843c008c60 tx=0x7f843c008230 comp rx=0 tx=0).stop 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 msgr2=0x7f8444101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 0x7f8444101270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f843400c970 tx=0x7f843400ce30 comp rx=0 tx=0).stop 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 shutdown_connections 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f842c0a33b0 0x7f842c0a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8444106f50 0x7f8444101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8444100ca0 0x7f84441017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 --2- 172.21.15.64:0/2887890184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84441002f0 0x7f8444101270 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 >> 172.21.15.64:0/2887890184 conn(0x7f8444077210 msgr2=0x7f8444105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 shutdown_connections 2024-09-19T05:10:31.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:31.849+0000 7f844a99a640 1 -- 172.21.15.64:0/2887890184 wait complete. 2024-09-19T05:10:31.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:31 smithi103 ceph-mon[34446]: pgmap v141: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:32.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:10:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 -- 172.21.15.64:0/3221777964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc7810c180 msgr2=0x7fbc7810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/3221777964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc7810c180 0x7fbc7810e600 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009ca0 tx=0x7fbc7002a950 comp rx=0 tx=0).stop 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 -- 172.21.15.64:0/3221777964 shutdown_connections 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/3221777964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc7810c180 0x7fbc7810e600 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/3221777964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc781036c0 0x7fbc7810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/3221777964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc78102da0 0x7fbc78103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 -- 172.21.15.64:0/3221777964 >> 172.21.15.64:0/3221777964 conn(0x7fbc780fc7a0 msgr2=0x7fbc780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 -- 172.21.15.64:0/3221777964 shutdown_connections 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.602+0000 7fbc7ef11640 1 -- 172.21.15.64:0/3221777964 wait complete. 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 Processor -- start 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 -- start start 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 0x7fbc78100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc781036c0 0x7fbc78100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc7810c180 0x7fbc78101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111dd0 con 0x7fbc78102da0 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111c50 con 0x7fbc7810c180 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ef11640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111f50 con 0x7fbc781036c0 2024-09-19T05:10:33.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 0x7fbc78100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 0x7fbc78100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56158/0 (socket says 172.21.15.64:56158) 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 -- 172.21.15.64:0/2041405802 learned_addr learned my addr 172.21.15.64:0/2041405802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc7810c180 0x7fbc78101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 -- 172.21.15.64:0/2041405802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc781036c0 msgr2=0x7fbc78100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc781036c0 0x7fbc78100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 -- 172.21.15.64:0/2041405802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc7810c180 msgr2=0x7fbc78101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc7810c180 0x7fbc78101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.603+0000 7fbc7ccad640 1 -- 172.21.15.64:0/2041405802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc781a66b0 con 0x7fbc78102da0 2024-09-19T05:10:33.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.604+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 0x7fbc78100a00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c920 tx=0x7fbc6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.604+0000 7fbc75ffb640 1 -- 172.21.15.64:0/2041405802 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c007c70 con 0x7fbc78102da0 2024-09-19T05:10:33.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.604+0000 7fbc75ffb640 1 -- 172.21.15.64:0/2041405802 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc6c007e10 con 0x7fbc78102da0 2024-09-19T05:10:33.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.604+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc781a69a0 con 0x7fbc78102da0 2024-09-19T05:10:33.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.604+0000 7fbc75ffb640 1 -- 172.21.15.64:0/2041405802 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c018820 con 0x7fbc78102da0 2024-09-19T05:10:33.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.604+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781a6ee0 con 0x7fbc78102da0 2024-09-19T05:10:33.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.607+0000 7fbc75ffb640 1 -- 172.21.15.64:0/2041405802 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fbc6c0189c0 con 0x7fbc78102da0 2024-09-19T05:10:33.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.607+0000 7fbc812ea740 1 Processor -- start 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.607+0000 7fbc812ea740 1 -- start start 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b08b40 0x565247b07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc812ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565247a6d2f0 con 0x565247b12370 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc812ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565247a14cf0 con 0x565247b075d0 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc812ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565247a692b0 con 0x565247b08b40 2024-09-19T05:10:33.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc75ffb640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc680a33b0 0x7fbc680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b14760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56168/0 (socket says 172.21.15.64:56168) 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 -- 172.21.15.64:0/3949931461 learned_addr learned my addr 172.21.15.64:0/3949931461 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc75ffb640 1 -- 172.21.15.64:0/2041405802 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fbc6c0cff20 con 0x7fbc78102da0 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc680a33b0 0x7fbc680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b08b40 0x565247b07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 -- 172.21.15.64:0/3949931461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b08b40 msgr2=0x565247b07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b08b40 0x565247b07090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 -- 172.21.15.64:0/3949931461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 msgr2=0x565247b07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 -- 172.21.15.64:0/3949931461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652479f93a0 con 0x565247b12370 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.608+0000 7fbc77fff640 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b14760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400a9e0 tx=0x7fbc6400aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275d8644bcbe88fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc680a33b0 0x7fbc680a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009a60 tx=0x7fbc70009890 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc62ffd640 1 -- 172.21.15.64:0/3949931461 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400cdb0 con 0x565247b12370 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc62ffd640 1 -- 172.21.15.64:0/3949931461 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc640035f0 con 0x565247b12370 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc62ffd640 1 -- 172.21.15.64:0/3949931461 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400f6b0 con 0x565247b12370 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 -- 172.21.15.64:0/3949931461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 msgr2=0x565247b14760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b14760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400a9e0 tx=0x7fbc6400aea0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 -- 172.21.15.64:0/3949931461 shutdown_connections 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b14760 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3949931461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b08b40 0x565247b07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.609+0000 7fbc812ea740 1 -- 172.21.15.64:0/3949931461 >> 172.21.15.64:0/3949931461 conn(0x565247ad0310 msgr2=0x565247b03c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 -- 172.21.15.64:0/3949931461 shutdown_connections 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 -- 172.21.15.64:0/3949931461 wait complete. 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 -- start start 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 Processor -- start 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b11d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b08b40 0x565247b0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b0f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565247b15070 con 0x565247b08b40 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565247a28060 con 0x565247b075d0 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565247b151f0 con 0x565247b12370 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.610+0000 7fbc812ea740 10 client.0 _ll_register_callbacks cb 0x56524777b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b11d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b08b40 0x565247b0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b11d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:45354/0 (socket says 172.21.15.64:45354) 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3658391707 learned_addr learned my addr 172.21.15.64:0/3658391707 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:33.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b0f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3658391707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 msgr2=0x565247b0f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b0f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3658391707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b08b40 msgr2=0x565247b0efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b08b40 0x565247b0efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3658391707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652477de3c0 con 0x565247b075d0 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc77fff640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b08b40 0x565247b0efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b11d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c8f0 tx=0x7fbc6c09a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be2a1d578533120b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc57fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc57fff640 1 -- 172.21.15.64:0/3658391707 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0b4030 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc57fff640 1 -- 172.21.15.64:0/3658391707 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fbc6c09c460 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.611+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc640035f0 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc57fff640 1 -- 172.21.15.64:0/3658391707 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c09c200 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 10 client.26108 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565247b20b80 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc57fff640 1 -- 172.21.15.64:0/3658391707 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc640035f0 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc57fff640 1 -- 172.21.15.64:0/3658391707 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc6c0a49e0 con 0x565247b075d0 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 10 client.26108 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 20 client.26108 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b22040 0x565247b24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 4 client.26108 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565247a7a860 con 0x565247b22040 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.612+0000 7fbc77fff640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b22040 0x565247b24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.613+0000 7fbc57fff640 10 client.26108 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:33.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.613+0000 7fbc77fff640 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b22040 0x565247b24430 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da7a652fc9445b7 server_cookie=4f2da87fc06d4b2b in_seq=0 out_seq=0 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.613+0000 7fbc57fff640 1 -- 172.21.15.64:0/3658391707 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565247a7a860 con 0x565247b22040 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.613+0000 7fbc57fff640 10 client.26108 handle_command_reply: tid=0 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.613+0000 7fbc812ea740 1 client.26108 shutdown 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 msgr2=0x565247b11d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b11d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c8f0 tx=0x7fbc6c09a710 comp rx=0 tx=0).stop 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 shutdown_connections 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b0f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b22040 0x565247b24430 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b08b40 0x565247b0efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3658391707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b075d0 0x565247b11d30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc57fff640 0 client.26108 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 >> 172.21.15.64:0/3658391707 conn(0x565247ad0310 msgr2=0x565247b09990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 shutdown_connections 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 1 -- 172.21.15.64:0/3658391707 wait complete. 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.614+0000 7fbc812ea740 20 client.26108 trim_cache size 0 max 16384 2024-09-19T05:10:33.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 Processor -- start 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 -- start start 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 0x565247b03e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b29c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781b2220 con 0x565247b075d0 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781afaf0 con 0x565247b08b40 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc812ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ad940 con 0x565247b12370 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc7ccad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 0x565247b03e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc7ccad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 0x565247b03e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56176/0 (socket says 172.21.15.64:56176) 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.627+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3089219691 learned_addr learned my addr 172.21.15.64:0/3089219691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3089219691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 msgr2=0x565247b29c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b29c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b29c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3089219691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 msgr2=0x565247b1e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7ccad640 1 -- 172.21.15.64:0/3089219691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565247b0f940 con 0x565247b075d0 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 0x565247b03e10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0bc010 tx=0x7fbc6c00fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c36a3111d579e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc627fc640 1 -- 172.21.15.64:0/3089219691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0b4070 con 0x565247b075d0 2024-09-19T05:10:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc627fc640 1 -- 172.21.15.64:0/3089219691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc6c00fb00 con 0x565247b075d0 2024-09-19T05:10:33.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.628+0000 7fbc627fc640 1 -- 172.21.15.64:0/3089219691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0a4e70 con 0x565247b075d0 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 -- 172.21.15.64:0/3089219691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 msgr2=0x565247b03e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 0x565247b03e10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0bc010 tx=0x7fbc6c00fe00 comp rx=0 tx=0).stop 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 -- 172.21.15.64:0/3089219691 shutdown_connections 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b12370 0x565247b29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b1e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 --2- 172.21.15.64:0/3089219691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b075d0 0x565247b03e10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 -- 172.21.15.64:0/3089219691 >> 172.21.15.64:0/3089219691 conn(0x565247ad0310 msgr2=0x565247b03700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 -- 172.21.15.64:0/3089219691 shutdown_connections 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 -- 172.21.15.64:0/3089219691 wait complete. 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 -- start start 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 Processor -- start 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b075d0 0x565247b161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.629+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b16700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc812ea740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b16c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc812ea740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781adfb0 con 0x565247b12370 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc812ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781af420 con 0x565247b08b40 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc812ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ae680 con 0x565247b075d0 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc812ea740 10 client.0 _ll_register_callbacks cb 0x7fbc6c09bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b16c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b16c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56186/0 (socket says 172.21.15.64:56186) 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 -- 172.21.15.64:0/1465614699 learned_addr learned my addr 172.21.15.64:0/1465614699 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b075d0 0x565247b161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc77fff640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b16700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 -- 172.21.15.64:0/1465614699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b075d0 msgr2=0x565247b161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b075d0 0x565247b161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 -- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 msgr2=0x565247b16700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b16700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 -- 172.21.15.64:0/1465614699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565247b8bc10 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b075d0 0x565247b161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc77fff640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b16700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc7d4ae640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b16c40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbc7002f810 tx=0x7fbc70009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d637c8cff7e5e859 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc57fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc57fff640 1 -- 172.21.15.64:0/1465614699 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70034040 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc57fff640 1 -- 172.21.15.64:0/1465614699 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fbc700047c0 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.630+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565247a74d70 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc57fff640 1 -- 172.21.15.64:0/1465614699 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc70030440 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 10 client.16533 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565247b8ce00 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc57fff640 1 -- 172.21.15.64:0/1465614699 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x565247a74d70 con 0x565247b12370 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc57fff640 1 -- 172.21.15.64:0/1465614699 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc7005b6f0 con 0x565247b12370 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 10 client.16533 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 20 client.16533 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b8d550 0x565247b8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 4 client.16533 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652479f1170 con 0x565247b8d550 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.631+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b8d550 0x565247b8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc7ccad640 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b8d550 0x565247b8f970 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf783c60dfa9e8a1 server_cookie=8b513454d342fa05 in_seq=0 out_seq=0 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc57fff640 10 client.16533 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc57fff640 1 -- 172.21.15.64:0/1465614699 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652479f1170 con 0x565247b8d550 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc57fff640 10 client.16533 handle_command_reply: tid=0 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 client.16533 shutdown 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 msgr2=0x565247b16c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b16c40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbc7002f810 tx=0x7fbc70009820 comp rx=0 tx=0).stop 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 shutdown_connections 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x565247b8d550 0x565247b8f970 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565247b12370 0x565247b16c40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565247b08b40 0x565247b16700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc57fff640 0 client.16533 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 --2- 172.21.15.64:0/1465614699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565247b075d0 0x565247b161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 >> 172.21.15.64:0/1465614699 conn(0x565247ad0310 msgr2=0x565247b09110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 shutdown_connections 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 1 -- 172.21.15.64:0/1465614699 wait complete. 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.632+0000 7fbc812ea740 20 client.16533 trim_cache size 0 max 16384 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc680a33b0 msgr2=0x7fbc680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc680a33b0 0x7fbc680a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009a60 tx=0x7fbc70009890 comp rx=0 tx=0).stop 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 msgr2=0x7fbc78100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 0x7fbc78100a00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c920 tx=0x7fbc6c00cde0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 shutdown_connections 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbc680a33b0 0x7fbc680a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc7810c180 0x7fbc78101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc781036c0 0x7fbc78100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 --2- 172.21.15.64:0/2041405802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc78102da0 0x7fbc78100a00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 >> 172.21.15.64:0/2041405802 conn(0x7fbc780fc7a0 msgr2=0x7fbc780fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 shutdown_connections 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:33.633+0000 7fbc7ef11640 1 -- 172.21.15.64:0/2041405802 wait complete. 2024-09-19T05:10:33.771 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:33 smithi103 ceph-mon[34446]: pgmap v142: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:33.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:33 smithi174 ceph-mon[34488]: pgmap v142: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:33.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:33 smithi064 ceph-mon[30960]: pgmap v142: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:35.446 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:10:35.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:10:35.720 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:35.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:35 smithi174 ceph-mon[34488]: pgmap v143: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:35.841 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:35 smithi064 ceph-mon[30960]: pgmap v143: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:35.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:35 smithi103 ceph-mon[34446]: pgmap v143: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 -- 172.21.15.64:0/1599683014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8102ef0 msgr2=0x7ff5e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 --2- 172.21.15.64:0/1599683014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8102ef0 0x7ff5e810da40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009ca0 tx=0x7ff5e002a950 comp rx=0 tx=0).stop 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 -- 172.21.15.64:0/1599683014 shutdown_connections 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 --2- 172.21.15.64:0/1599683014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8102ef0 0x7ff5e810da40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 --2- 172.21.15.64:0/1599683014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8102570 0x7ff5e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 --2- 172.21.15.64:0/1599683014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5e8100e20 0x7ff5e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 -- 172.21.15.64:0/1599683014 >> 172.21.15.64:0/1599683014 conn(0x7ff5e80fc7d0 msgr2=0x7ff5e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 -- 172.21.15.64:0/1599683014 shutdown_connections 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 -- 172.21.15.64:0/1599683014 wait complete. 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 Processor -- start 2024-09-19T05:10:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.206+0000 7ff5ee901640 1 -- start start 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 0x7ff5e80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8102570 0x7ff5e80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5e8102ef0 0x7ff5e80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8110e80 con 0x7ff5e8100e20 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8110d00 con 0x7ff5e8102570 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8111000 con 0x7ff5e8102ef0 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 0x7ff5e80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 0x7ff5e80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49884/0 (socket says 172.21.15.64:49884) 2024-09-19T05:10:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 -- 172.21.15.64:0/2388955008 learned_addr learned my addr 172.21.15.64:0/2388955008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5dffff640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8102570 0x7ff5e80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 -- 172.21.15.64:0/2388955008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5e8102ef0 msgr2=0x7ff5e80fd700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5e8102ef0 0x7ff5e80fd700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 -- 172.21.15.64:0/2388955008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8102570 msgr2=0x7ff5e80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8102570 0x7ff5e80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 -- 172.21.15.64:0/2388955008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e81a67b0 con 0x7ff5e8100e20 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 0x7ff5e80fcc80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800d770 tx=0x7ff5d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e5ffb640 1 -- 172.21.15.64:0/2388955008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8017070 con 0x7ff5e8100e20 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e5ffb640 1 -- 172.21.15.64:0/2388955008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5d80054d0 con 0x7ff5e8100e20 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e81a6aa0 con 0x7ff5e8100e20 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.207+0000 7ff5e5ffb640 1 -- 172.21.15.64:0/2388955008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8004cf0 con 0x7ff5e8100e20 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.208+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81a6fe0 con 0x7ff5e8100e20 2024-09-19T05:10:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5e5ffb640 1 -- 172.21.15.64:0/2388955008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff5d8004e90 con 0x7ff5e8100e20 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5f0cda740 1 Processor -- start 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5f0cda740 1 -- start start 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 0x562918789f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56291878a480 0x562918794800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5e5ffb640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5d40a3430 0x7ff5d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918797130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.210+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5629186ef280 con 0x562918794d40 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562918696b10 con 0x56291878a480 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629186eb240 con 0x562918789b60 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918797130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5dffff640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5d40a3430 0x7ff5d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918797130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49900/0 (socket says 172.21.15.64:49900) 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 -- 172.21.15.64:0/1562557005 learned_addr learned my addr 172.21.15.64:0/1562557005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5e5ffb640 1 -- 172.21.15.64:0/2388955008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff5d80cf7c0 con 0x7ff5e8100e20 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56291878a480 0x562918794800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 -- 172.21.15.64:0/1562557005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 msgr2=0x562918789f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 0x562918789f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 -- 172.21.15.64:0/1562557005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56291878a480 msgr2=0x562918794800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56291878a480 0x562918794800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 -- 172.21.15.64:0/1562557005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629186a31f0 con 0x562918794d40 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5dffff640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5d40a3430 0x7ff5d40a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80fe710 tx=0x7ff5d000c4a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918797130 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0004af0 tx=0x7ff5e002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bfea05c83e2481 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5de7fc640 1 -- 172.21.15.64:0/1562557005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0034030 con 0x562918794d40 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5de7fc640 1 -- 172.21.15.64:0/1562557005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5e002f540 con 0x562918794d40 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.211+0000 7ff5de7fc640 1 -- 172.21.15.64:0/1562557005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00042b0 con 0x562918794d40 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 -- 172.21.15.64:0/1562557005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 msgr2=0x562918797130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918797130 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0004af0 tx=0x7ff5e002f450 comp rx=0 tx=0).stop 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 -- 172.21.15.64:0/1562557005 shutdown_connections 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918797130 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56291878a480 0x562918794800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/1562557005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 0x562918789f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 -- 172.21.15.64:0/1562557005 >> 172.21.15.64:0/1562557005 conn(0x5629187860f0 msgr2=0x5629187864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 -- 172.21.15.64:0/1562557005 shutdown_connections 2024-09-19T05:10:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 -- 172.21.15.64:0/1562557005 wait complete. 2024-09-19T05:10:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.212+0000 7ff5f0cda740 1 -- start start 2024-09-19T05:10:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 Processor -- start 2024-09-19T05:10:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 0x562918791c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 0x562918792180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918794d40 0x5629187926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562918797910 con 0x56291878a480 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629186a9f10 con 0x562918794d40 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562918797a90 con 0x562918789b60 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5f0cda740 10 client.0 _ll_register_callbacks cb 0x562918672a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 0x562918792180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 0x562918792180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49914/0 (socket says 172.21.15.64:49914) 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 -- 172.21.15.64:0/3554252790 learned_addr learned my addr 172.21.15.64:0/3554252790 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918794d40 0x5629187926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 -- 172.21.15.64:0/3554252790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 msgr2=0x562918791c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 0x562918791c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 -- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918794d40 msgr2=0x5629187926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918794d40 0x5629187926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.213+0000 7ff5dffff640 1 -- 172.21.15.64:0/3554252790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629184606a0 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5dffff640 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 0x562918792180 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00109b0 tx=0x7ff5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0addecb6b4da49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5c77fe640 1 -- 172.21.15.64:0/3554252790 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0021070 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5c77fe640 1 -- 172.21.15.64:0/3554252790 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff5d000fcd0 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e002f540 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5c77fe640 1 -- 172.21.15.64:0/3554252790 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5d002d050 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 10 client.16545 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629187a35f0 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5c77fe640 1 -- 172.21.15.64:0/3554252790 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5e002f540 con 0x56291878a480 2024-09-19T05:10:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5c77fe640 1 -- 172.21.15.64:0/3554252790 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5d0025440 con 0x56291878a480 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 10 client.16545 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 20 client.16545 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5629187a4aa0 0x5629187a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 4 client.16545 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.214+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629186fc7f0 con 0x5629187a4aa0 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.215+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5629187a4aa0 0x5629187a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.215+0000 7ff5c77fe640 10 client.16545 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.215+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5629187a4aa0 0x5629187a6f00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a5d6ee12ee417b server_cookie=eb89267ac45992a5 in_seq=0 out_seq=0 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5c77fe640 1 -- 172.21.15.64:0/3554252790 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629186fc7f0 con 0x5629187a4aa0 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5c77fe640 10 client.16545 handle_command_reply: tid=0 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 client.16545 shutdown 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 msgr2=0x562918792180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 0x562918792180 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00109b0 tx=0x7ff5d00046c0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 shutdown_connections 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918794d40 0x5629187926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5629187a4aa0 0x5629187a6f00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56291878a480 0x562918792180 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/3554252790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562918789b60 0x562918791c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5c77fe640 0 client.16545 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 >> 172.21.15.64:0/3554252790 conn(0x5629187860f0 msgr2=0x56291878bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 shutdown_connections 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 1 -- 172.21.15.64:0/3554252790 wait complete. 2024-09-19T05:10:36.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.216+0000 7ff5f0cda740 20 client.16545 trim_cache size 0 max 16384 2024-09-19T05:10:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.234+0000 7ff5f0cda740 1 Processor -- start 2024-09-19T05:10:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.234+0000 7ff5f0cda740 1 -- start start 2024-09-19T05:10:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918793aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x5629187acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b2140 con 0x562918794d40 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81afa10 con 0x562918789b60 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81ad860 con 0x56291878a480 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x5629187acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x5629187acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49924/0 (socket says 172.21.15.64:49924) 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2762252 learned_addr learned my addr 172.21.15.64:0/2762252 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5dffff640 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918793aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2762252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 msgr2=0x5629187ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2762252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 msgr2=0x562918793aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918793aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2762252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562918787280 con 0x562918794d40 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.235+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x5629187acd10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0003d90 tx=0x7ff5e0003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8436bbe63d7907 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5ddffb640 1 -- 172.21.15.64:0/2762252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0034050 con 0x562918794d40 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5ddffb640 1 -- 172.21.15.64:0/2762252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5e002f540 con 0x562918794d40 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5ddffb640 1 -- 172.21.15.64:0/2762252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0030510 con 0x562918794d40 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2762252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 msgr2=0x5629187acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x5629187acd10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0003d90 tx=0x7ff5e0003d20 comp rx=0 tx=0).stop 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2762252 shutdown_connections 2024-09-19T05:10:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x5629187acd10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2762252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918793aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2762252 >> 172.21.15.64:0/2762252 conn(0x5629187860f0 msgr2=0x562918793390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2762252 shutdown_connections 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.236+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2762252 wait complete. 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 -- start start 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 Processor -- start 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918799370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187998b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918808770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81aded0 con 0x562918794d40 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81af340 con 0x562918789b60 2024-09-19T05:10:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81ae5a0 con 0x56291878a480 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5f0cda740 10 client.0 _ll_register_callbacks cb 0x7ff5d0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5ece9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918808770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918799370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5ece9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918808770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49936/0 (socket says 172.21.15.64:49936) 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918799370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:33774/0 (socket says 172.21.15.64:33774) 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.237+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2513571244 learned_addr learned my addr 172.21.15.64:0/2513571244 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5dffff640 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2513571244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 msgr2=0x5629187998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 msgr2=0x562918799370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918799370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5ece9e640 1 -- 172.21.15.64:0/2513571244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562918808e10 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5ece9e640 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918808770 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0005e30 tx=0x7ff5e0005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd60f7bf5cd16b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5c77fe640 1 -- 172.21.15.64:0/2513571244 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0052070 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5c77fe640 1 -- 172.21.15.64:0/2513571244 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff5e00304b0 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e002f540 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5c77fe640 1 -- 172.21.15.64:0/2513571244 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5e005b050 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5f0cda740 10 client.16551 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.238+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562918810200 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5c77fe640 1 -- 172.21.15.64:0/2513571244 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5e002f540 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5c77fe640 1 -- 172.21.15.64:0/2513571244 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5e0055020 con 0x562918794d40 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5f0cda740 10 client.16551 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5f0cda740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5f0cda740 20 client.16551 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562918810430 0x562918812820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5f0cda740 4 client.16551 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629186f6d00 con 0x562918810430 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562918810430 0x562918812820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5c77fe640 10 client.16551 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.239+0000 7ff5e7fff640 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562918810430 0x562918812820 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73f3b03d37deae8 server_cookie=84987b81db494b7c in_seq=0 out_seq=0 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.240+0000 7ff5c77fe640 1 -- 172.21.15.64:0/2513571244 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629186f6d00 con 0x562918810430 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.240+0000 7ff5c77fe640 10 client.16551 handle_command_reply: tid=0 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.240+0000 7ff5f0cda740 1 client.16551 shutdown 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 msgr2=0x562918808770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918808770 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0005e30 tx=0x7ff5e0005cf0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 shutdown_connections 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x562918810430 0x562918812820 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562918794d40 0x562918808770 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56291878a480 0x5629187998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5c77fe640 0 client.16551 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 --2- 172.21.15.64:0/2513571244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562918789b60 0x562918799370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 >> 172.21.15.64:0/2513571244 conn(0x5629187860f0 msgr2=0x562918785270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 shutdown_connections 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 1 -- 172.21.15.64:0/2513571244 wait complete. 2024-09-19T05:10:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.241+0000 7ff5f0cda740 20 client.16551 trim_cache size 0 max 16384 2024-09-19T05:10:36.259 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:10:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:10:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "4f135230-2cf6-4cca-9e33-c32b4df6b186", 2024-09-19T05:10:36.259 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:10:36.259 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5d40a3430 msgr2=0x7ff5d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5d40a3430 0x7ff5d40a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80fe710 tx=0x7ff5d000c4a0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 msgr2=0x7ff5e80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 0x7ff5e80fcc80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800d770 tx=0x7ff5d800dc30 comp rx=0 tx=0).stop 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 shutdown_connections 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5d40a3430 0x7ff5d40a58f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5e8102ef0 0x7ff5e80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8102570 0x7ff5e80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 --2- 172.21.15.64:0/2388955008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5e8100e20 0x7ff5e80fcc80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 >> 172.21.15.64:0/2388955008 conn(0x7ff5e80fc7d0 msgr2=0x7ff5e806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 shutdown_connections 2024-09-19T05:10:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:36.242+0000 7ff5ee901640 1 -- 172.21.15.64:0/2388955008 wait complete. 2024-09-19T05:10:37.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:10:37.322 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:37.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:37 smithi064 ceph-mon[30960]: pgmap v144: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:37.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:37 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:10:37.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:37 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:37.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:37 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:10:37.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:37 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:37 smithi174 ceph-mon[34488]: pgmap v144: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:37 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:10:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:37 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:37 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:10:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:37 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:37.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 -- 172.21.15.64:0/73452543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4106ed0 msgr2=0x7f7da410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 --2- 172.21.15.64:0/73452543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4106ed0 0x7f7da410eaa0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009ca0 tx=0x7f7d9c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 -- 172.21.15.64:0/73452543 shutdown_connections 2024-09-19T05:10:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 --2- 172.21.15.64:0/73452543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4106ed0 0x7f7da410eaa0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 --2- 172.21.15.64:0/73452543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4100eb0 0x7f7da41067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 --2- 172.21.15.64:0/73452543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 0x7f7da41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 -- 172.21.15.64:0/73452543 >> 172.21.15.64:0/73452543 conn(0x7f7da40fc5c0 msgr2=0x7f7da40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 -- 172.21.15.64:0/73452543 shutdown_connections 2024-09-19T05:10:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.779+0000 7f7dabf01640 1 -- 172.21.15.64:0/73452543 wait complete. 2024-09-19T05:10:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 Processor -- start 2024-09-19T05:10:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 -- start start 2024-09-19T05:10:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 0x7f7da4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 0x7f7da4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4106ed0 0x7f7da419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111e30 con 0x7f7da4100eb0 2024-09-19T05:10:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111cb0 con 0x7f7da4100500 2024-09-19T05:10:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7dabf01640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111fb0 con 0x7f7da4106ed0 2024-09-19T05:10:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 0x7f7da4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 0x7f7da4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49952/0 (socket says 172.21.15.64:49952) 2024-09-19T05:10:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 -- 172.21.15.64:0/3477015769 learned_addr learned my addr 172.21.15.64:0/3477015769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4106ed0 0x7f7da419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 0x7f7da4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 -- 172.21.15.64:0/3477015769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4106ed0 msgr2=0x7f7da419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4106ed0 0x7f7da419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 -- 172.21.15.64:0/3477015769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 msgr2=0x7f7da4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 0x7f7da4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 -- 172.21.15.64:0/3477015769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da419d2c0 con 0x7f7da4100eb0 2024-09-19T05:10:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4106ed0 0x7f7da419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 0x7f7da4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.780+0000 7f7da949c640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 0x7f7da4198850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400d950 tx=0x7f7d9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.781+0000 7f7d9affd640 1 -- 172.21.15.64:0/3477015769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94017030 con 0x7f7da4100eb0 2024-09-19T05:10:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.781+0000 7f7d9affd640 1 -- 172.21.15.64:0/3477015769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d94004020 con 0x7f7da4100eb0 2024-09-19T05:10:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.781+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da419d5b0 con 0x7f7da4100eb0 2024-09-19T05:10:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.781+0000 7f7d9affd640 1 -- 172.21.15.64:0/3477015769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940047a0 con 0x7f7da4100eb0 2024-09-19T05:10:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.781+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a4e90 con 0x7f7da4100eb0 2024-09-19T05:10:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.783+0000 7f7d9affd640 1 -- 172.21.15.64:0/3477015769 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7d940041d0 con 0x7f7da4100eb0 2024-09-19T05:10:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.783+0000 7f7dae2da740 1 Processor -- start 2024-09-19T05:10:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 -- start start 2024-09-19T05:10:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6360 0x55bb649c6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7d9affd640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7d900a3430 0x7f7d900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bb6492c2e0 con 0x55bb649c6c80 2024-09-19T05:10:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb648d3ce0 con 0x55bb649c6360 2024-09-19T05:10:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7dae2da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb649282a0 con 0x55bb649d5a90 2024-09-19T05:10:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649d5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49960/0 (socket says 172.21.15.64:49960) 2024-09-19T05:10:37.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3709697189 learned_addr learned my addr 172.21.15.64:0/3709697189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:37.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da949c640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7d900a3430 0x7f7d900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7d9affd640 1 -- 172.21.15.64:0/3477015769 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7d940cf740 con 0x7f7da4100eb0 2024-09-19T05:10:37.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6360 0x55bb649c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3709697189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 msgr2=0x55bb649d7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3709697189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6360 msgr2=0x55bb649c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6360 0x55bb649c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.784+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3709697189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb648b8440 con 0x55bb649c6c80 2024-09-19T05:10:37.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.785+0000 7f7da949c640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7d900a3430 0x7f7d900a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d940ccf40 tx=0x7f7d94086040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:37.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.785+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649d5550 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009c50 tx=0x7f7da000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c5d12fbd3b104b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:37.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.785+0000 7f7d8ffff640 1 -- 172.21.15.64:0/3709697189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0016070 con 0x55bb649c6c80 2024-09-19T05:10:37.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.785+0000 7f7d8ffff640 1 -- 172.21.15.64:0/3709697189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7da000ee70 con 0x55bb649c6c80 2024-09-19T05:10:37.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.785+0000 7f7d8ffff640 1 -- 172.21.15.64:0/3709697189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0011470 con 0x55bb649c6c80 2024-09-19T05:10:37.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 -- 172.21.15.64:0/3709697189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 msgr2=0x55bb649d5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649d5550 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009c50 tx=0x7f7da000e6c0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 -- 172.21.15.64:0/3709697189 shutdown_connections 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649d5550 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3709697189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6360 0x55bb649c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 -- 172.21.15.64:0/3709697189 >> 172.21.15.64:0/3709697189 conn(0x55bb649c2a30 msgr2=0x55bb649c2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 -- 172.21.15.64:0/3709697189 shutdown_connections 2024-09-19T05:10:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 -- 172.21.15.64:0/3709697189 wait complete. 2024-09-19T05:10:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 -- start start 2024-09-19T05:10:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 Processor -- start 2024-09-19T05:10:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.786+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bb649d8660 con 0x55bb649c6360 2024-09-19T05:10:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb648e7050 con 0x55bb649c6c80 2024-09-19T05:10:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb649d87e0 con 0x55bb649d5a90 2024-09-19T05:10:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 10 client.0 _ll_register_callbacks cb 0x55bb64927160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49962/0 (socket says 172.21.15.64:49962) 2024-09-19T05:10:37.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 -- 172.21.15.64:0/3068100290 learned_addr learned my addr 172.21.15.64:0/3068100290 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 -- 172.21.15.64:0/3068100290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 msgr2=0x55bb649d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 -- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 msgr2=0x55bb649d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649d2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 -- 172.21.15.64:0/3068100290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb6469d3c0 con 0x55bb649c6360 2024-09-19T05:10:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d53a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c02ade0 tx=0x7f7d9c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663e5ce57791fbf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7d8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3068100290 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c003b80 con 0x55bb649c6360 2024-09-19T05:10:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da000ee70 con 0x55bb649c6360 2024-09-19T05:10:37.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.787+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3068100290 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7d9c003d20 con 0x55bb649c6360 2024-09-19T05:10:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3068100290 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c004040 con 0x55bb649c6360 2024-09-19T05:10:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 10 client.16563 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb649e4020 con 0x55bb649c6360 2024-09-19T05:10:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3068100290 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7da000ee70 con 0x55bb649c6360 2024-09-19T05:10:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3068100290 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d9c030970 con 0x55bb649c6360 2024-09-19T05:10:37.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 20 client.16563 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649e53e0 0x55bb649e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 4 client.16563 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb64939850 con 0x55bb649e53e0 2024-09-19T05:10:37.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.788+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649e53e0 0x55bb649e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.789+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649e53e0 0x55bb649e7830 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e374e2028f7457ac server_cookie=2881b16e9349824a in_seq=0 out_seq=0 2024-09-19T05:10:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.789+0000 7f7d8d7fa640 10 client.16563 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.789+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3068100290 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb64939850 con 0x55bb649e53e0 2024-09-19T05:10:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.789+0000 7f7d8d7fa640 10 client.16563 handle_command_reply: tid=0 2024-09-19T05:10:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.789+0000 7f7dae2da740 1 client.16563 shutdown 2024-09-19T05:10:37.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 msgr2=0x55bb649d53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d53a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c02ade0 tx=0x7f7d9c009900 comp rx=0 tx=0).stop 2024-09-19T05:10:37.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 shutdown_connections 2024-09-19T05:10:37.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649d5a90 0x55bb649d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649e53e0 0x55bb649e7830 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3068100290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d53a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7d8d7fa640 0 client.16563 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 >> 172.21.15.64:0/3068100290 conn(0x55bb649c2a30 msgr2=0x55bb649c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 shutdown_connections 2024-09-19T05:10:37.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 1 -- 172.21.15.64:0/3068100290 wait complete. 2024-09-19T05:10:37.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.790+0000 7f7dae2da740 20 client.16563 trim_cache size 0 max 16384 2024-09-19T05:10:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 Processor -- start 2024-09-19T05:10:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 -- start start 2024-09-19T05:10:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c2a30 0x55bb649e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41adf80 con 0x55bb649c6360 2024-09-19T05:10:37.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41ab850 con 0x55bb649c6c80 2024-09-19T05:10:37.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7dae2da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41a96a0 con 0x55bb649c2a30 2024-09-19T05:10:37.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7da9c9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7da9c9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49970/0 (socket says 172.21.15.64:49970) 2024-09-19T05:10:37.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3475687141 learned_addr learned my addr 172.21.15.64:0/3475687141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c2a30 0x55bb649e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3475687141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c2a30 msgr2=0x55bb649e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c2a30 0x55bb649e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.808+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3475687141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 msgr2=0x55bb649c1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649c1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7da9c9d640 1 -- 172.21.15.64:0/3475687141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb649d1620 con 0x55bb649c6360 2024-09-19T05:10:37.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649c1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:37.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d0f40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000c00 tx=0x7f7da0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c925ec62906bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:37.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7d8f7fe640 1 -- 172.21.15.64:0/3475687141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0016040 con 0x55bb649c6360 2024-09-19T05:10:37.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7d8f7fe640 1 -- 172.21.15.64:0/3475687141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7da0009830 con 0x55bb649c6360 2024-09-19T05:10:37.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.809+0000 7f7d8f7fe640 1 -- 172.21.15.64:0/3475687141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0003c80 con 0x55bb649c6360 2024-09-19T05:10:37.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 -- 172.21.15.64:0/3475687141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 msgr2=0x55bb649d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d0f40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7da0000c00 tx=0x7f7da0005850 comp rx=0 tx=0).stop 2024-09-19T05:10:37.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 -- 172.21.15.64:0/3475687141 shutdown_connections 2024-09-19T05:10:37.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c6c80 0x55bb649c1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6360 0x55bb649d0f40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3475687141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c2a30 0x55bb649e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 -- 172.21.15.64:0/3475687141 >> 172.21.15.64:0/3475687141 conn(0x55bb649d2cf0 msgr2=0x55bb649c1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:37.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 -- 172.21.15.64:0/3475687141 shutdown_connections 2024-09-19T05:10:37.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 -- 172.21.15.64:0/3475687141 wait complete. 2024-09-19T05:10:37.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 -- start start 2024-09-19T05:10:37.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.810+0000 7f7dae2da740 1 Processor -- start 2024-09-19T05:10:37.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c2a30 0x55bb649dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c6360 0x55bb649ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649daa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41a9d10 con 0x55bb649c6c80 2024-09-19T05:10:37.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41ab180 con 0x55bb649c2a30 2024-09-19T05:10:37.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41aa3e0 con 0x55bb649c6360 2024-09-19T05:10:37.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 10 client.0 _ll_register_callbacks cb 0x7f7d9c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:37.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649daa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649daa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49974/0 (socket says 172.21.15.64:49974) 2024-09-19T05:10:37.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 -- 172.21.15.64:0/3774466479 learned_addr learned my addr 172.21.15.64:0/3774466479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:10:37.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c2a30 0x55bb649dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c6360 0x55bb649ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 -- 172.21.15.64:0/3774466479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c6360 msgr2=0x55bb649ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c6360 0x55bb649ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 -- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c2a30 msgr2=0x55bb649dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:37.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c2a30 0x55bb649dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 -- 172.21.15.64:0/3774466479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb649daf80 con 0x55bb649c6c80 2024-09-19T05:10:37.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7daa49e640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649daa40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009bf0 tx=0x7f7d9c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95322cfd4d891e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:37.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7d8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:10:37.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3774466479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0305c0 con 0x55bb649c6c80 2024-09-19T05:10:37.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3774466479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f7d9c030760 con 0x55bb649c6c80 2024-09-19T05:10:37.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.811+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da0009830 con 0x55bb649c6c80 2024-09-19T05:10:37.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3774466479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c004b40 con 0x55bb649c6c80 2024-09-19T05:10:37.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 10 client.16569 fetch_fsmap learned FSMap version 25 2024-09-19T05:10:37.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb64a50850 con 0x55bb649c6c80 2024-09-19T05:10:37.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3774466479 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7da0009830 con 0x55bb649c6c80 2024-09-19T05:10:37.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3774466479 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d9c067480 con 0x55bb649c6c80 2024-09-19T05:10:37.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 10 client.16569 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:10:37.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:37.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 20 client.16569 populate_metadata read hostname 'smithi064' 2024-09-19T05:10:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649d5a90 0x55bb64a52ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 4 client.16569 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:10:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb64933d60 con 0x55bb649d5a90 2024-09-19T05:10:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.812+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649d5a90 0x55bb64a52ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7da9c9d640 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649d5a90 0x55bb64a52ed0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc97b7a166d6f0f server_cookie=7e7636b20c2af110 in_seq=0 out_seq=0 2024-09-19T05:10:37.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7d8d7fa640 10 client.16569 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:37.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7d8d7fa640 1 -- 172.21.15.64:0/3774466479 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bb64933d60 con 0x55bb649d5a90 2024-09-19T05:10:37.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7d8d7fa640 10 client.16569 handle_command_reply: tid=0 2024-09-19T05:10:37.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 client.16569 shutdown 2024-09-19T05:10:37.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 msgr2=0x55bb649daa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649daa40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009bf0 tx=0x7f7d9c009900 comp rx=0 tx=0).stop 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 shutdown_connections 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55bb649d5a90 0x55bb64a52ed0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb649c6c80 0x55bb649daa40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb649c6360 0x55bb649ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7d8d7fa640 0 client.16569 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 --2- 172.21.15.64:0/3774466479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb649c2a30 0x55bb649dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 >> 172.21.15.64:0/3774466479 conn(0x55bb649d2cf0 msgr2=0x55bb649c1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 shutdown_connections 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 1 -- 172.21.15.64:0/3774466479 wait complete. 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.813+0000 7f7dae2da740 20 client.16569 trim_cache size 0 max 16384 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stdout: "4f135230-2cf6-4cca-9e33-c32b4df6b186": { 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:10:37.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "4f135230-2cf6-4cca-9e33-c32b4df6b186", 2024-09-19T05:10:37.845 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:10:37.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:10:37.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:10:37.845 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:10:37.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7d900a3430 msgr2=0x7f7d900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7d900a3430 0x7f7d900a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d940ccf40 tx=0x7f7d94086040 comp rx=0 tx=0).stop 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 msgr2=0x7f7da4198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 0x7f7da4198850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400d950 tx=0x7f7d9400de10 comp rx=0 tx=0).stop 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 shutdown_connections 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f7d900a3430 0x7f7d900a58f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4106ed0 0x7f7da419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da4100eb0 0x7f7da4198850 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 --2- 172.21.15.64:0/3477015769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da4100500 0x7f7da4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 >> 172.21.15.64:0/3477015769 conn(0x7f7da40fc5c0 msgr2=0x7f7da4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 shutdown_connections 2024-09-19T05:10:37.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:10:37.814+0000 7f7dabf01640 1 -- 172.21.15.64:0/3477015769 wait complete. 2024-09-19T05:10:37.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:37 smithi103 ceph-mon[34446]: pgmap v144: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:37.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:37 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:10:37.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:37 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:37.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:37 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:10:37.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:37 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.650 INFO:tasks.cephfs.filesystem:scrub status for tag:4f135230-2cf6-4cca-9e33-c32b4df6b186 - {'path': '/', 'tag': '4f135230-2cf6-4cca-9e33-c32b4df6b186', 'options': 'recursive,force'} 2024-09-19T05:10:39.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:39 smithi174 ceph-mon[34488]: pgmap v145: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:39.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:39 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:10:39.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:39 smithi064 ceph-mon[30960]: pgmap v145: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:39.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:39 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:10:39.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:39 smithi103 ceph-mon[34446]: pgmap v145: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:39.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:39 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:10:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:41 smithi174 ceph-mon[34488]: pgmap v146: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:41.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:41 smithi064 ceph-mon[30960]: pgmap v146: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:41.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:41 smithi103 ceph-mon[34446]: pgmap v146: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:43.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:43 smithi064 ceph-mon[30960]: pgmap v147: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:43.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:43 smithi103 ceph-mon[34446]: pgmap v147: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:44.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:43 smithi174 ceph-mon[34488]: pgmap v147: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:45.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:45 smithi064 ceph-mon[30960]: pgmap v148: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:45.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:45 smithi103 ceph-mon[34446]: pgmap v148: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:46.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:45 smithi174 ceph-mon[34488]: pgmap v148: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:47.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:47 smithi064 ceph-mon[30960]: pgmap v149: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:47.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:47 smithi103 ceph-mon[34446]: pgmap v149: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:48.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:47 smithi174 ceph-mon[34488]: pgmap v149: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:49 smithi064 ceph-mon[30960]: pgmap v150: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:49.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:49 smithi103 ceph-mon[34446]: pgmap v150: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:50.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:49 smithi174 ceph-mon[34488]: pgmap v150: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:10:51.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:51 smithi064 ceph-mon[30960]: pgmap v151: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:51.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:51 smithi103 ceph-mon[34446]: pgmap v151: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:52.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:51 smithi174 ceph-mon[34488]: pgmap v151: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:10:53.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:53 smithi064 ceph-mon[30960]: pgmap v152: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:53.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:53 smithi103 ceph-mon[34446]: pgmap v152: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:54.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:53 smithi174 ceph-mon[34488]: pgmap v152: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:55.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:55 smithi064 ceph-mon[30960]: pgmap v153: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:55.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:55 smithi103 ceph-mon[34446]: pgmap v153: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:56.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:55 smithi174 ceph-mon[34488]: pgmap v153: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:57 smithi064 ceph-mon[30960]: pgmap v154: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:57.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:57 smithi103 ceph-mon[34446]: pgmap v154: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:58.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:57 smithi174 ceph-mon[34488]: pgmap v154: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:10:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:10:59 smithi064 ceph-mon[30960]: pgmap v155: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:10:59.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:10:59 smithi103 ceph-mon[34446]: pgmap v155: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:00.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:10:59 smithi174 ceph-mon[34488]: pgmap v155: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:01.830 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:01 smithi174 ceph-mon[34488]: pgmap v156: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:11:01.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:01 smithi064 ceph-mon[30960]: pgmap v156: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:11:01.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:01 smithi103 ceph-mon[34446]: pgmap v156: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:11:03.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:03 smithi064 ceph-mon[30960]: pgmap v157: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:03.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:03 smithi103 ceph-mon[34446]: pgmap v157: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:04.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:03 smithi174 ceph-mon[34488]: pgmap v157: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:05.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:05 smithi064 ceph-mon[30960]: pgmap v158: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:05.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:05 smithi103 ceph-mon[34446]: pgmap v158: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:06.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:05 smithi174 ceph-mon[34488]: pgmap v158: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:11:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:11:06.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:11:06.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:11:06.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:11:06.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:11:06.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:11:06.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:11:06.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:11:07.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:11:07.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:11:07.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:11:07.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:11:07.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:07 smithi064 ceph-mon[30960]: pgmap v159: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:07.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:07 smithi103 ceph-mon[34446]: pgmap v159: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:08.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:07 smithi174 ceph-mon[34488]: pgmap v159: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:08.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:08.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:08 smithi064 ceph-mon[30960]: pgmap v160: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:08.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:08 smithi103 ceph-mon[34446]: pgmap v160: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:09.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:08 smithi174 ceph-mon[34488]: pgmap v160: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 -- 172.21.15.64:0/3484560335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 msgr2=0x7f365c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 --2- 172.21.15.64:0/3484560335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1067f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f364c009900 tx=0x7f364c02a410 comp rx=0 tx=0).stop 2024-09-19T05:11:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 -- 172.21.15.64:0/3484560335 shutdown_connections 2024-09-19T05:11:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 --2- 172.21.15.64:0/3484560335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f365c106f20 0x7f365c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 --2- 172.21.15.64:0/3484560335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1067f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 --2- 172.21.15.64:0/3484560335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 0x7f365c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 -- 172.21.15.64:0/3484560335 >> 172.21.15.64:0/3484560335 conn(0x7f365c0fd8f0 msgr2=0x7f365c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 -- 172.21.15.64:0/3484560335 shutdown_connections 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 -- 172.21.15.64:0/3484560335 wait complete. 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.409+0000 7f3663cb3640 1 Processor -- start 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 -- start start 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 0x7f365c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f365c106f20 0x7f365c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f365c111e30 con 0x7f365c1027a0 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f365c111cb0 con 0x7f365c101df0 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3663cb3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f365c111fb0 con 0x7f365c106f20 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45342/0 (socket says 172.21.15.64:45342) 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 -- 172.21.15.64:0/1220724309 learned_addr learned my addr 172.21.15.64:0/1220724309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3662250640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f365c106f20 0x7f365c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 0x7f365c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 -- 172.21.15.64:0/1220724309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f365c106f20 msgr2=0x7f365c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f365c106f20 0x7f365c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 -- 172.21.15.64:0/1220724309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 msgr2=0x7f365c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 0x7f365c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f366124e640 1 -- 172.21.15.64:0/1220724309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c19d320 con 0x7f365c1027a0 2024-09-19T05:11:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.410+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 0x7f365c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.411+0000 7f366124e640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1988b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f364c005940 tx=0x7f364c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.411+0000 7f3652ffd640 1 -- 172.21.15.64:0/1220724309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c03f030 con 0x7f365c1027a0 2024-09-19T05:11:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.411+0000 7f3652ffd640 1 -- 172.21.15.64:0/1220724309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364c003aa0 con 0x7f365c1027a0 2024-09-19T05:11:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.411+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c19d5b0 con 0x7f365c1027a0 2024-09-19T05:11:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.411+0000 7f3652ffd640 1 -- 172.21.15.64:0/1220724309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c035070 con 0x7f365c1027a0 2024-09-19T05:11:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.411+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c1a4ef0 con 0x7f365c1027a0 2024-09-19T05:11:09.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.414+0000 7f3652ffd640 1 -- 172.21.15.64:0/1220724309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f364c042030 con 0x7f365c1027a0 2024-09-19T05:11:09.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.414+0000 7f366608c740 1 Processor -- start 2024-09-19T05:11:09.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 -- start start 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d5949f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d5949f670 0x559d5949faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559d593fb2e0 con 0x559d594a4650 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d593a2ce0 con 0x559d5949f670 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366608c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d593f72a0 con 0x559d594a5000 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3662250640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3661a4f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d5949f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3662250640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594a4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45350/0 (socket says 172.21.15.64:45350) 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3662250640 1 -- 172.21.15.64:0/1512005412 learned_addr learned my addr 172.21.15.64:0/1512005412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3652ffd640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36480a3430 0x7f36480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f366124e640 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d5949f670 0x559d5949faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3662250640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36480a3430 0x7f36480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3661a4f640 1 -- 172.21.15.64:0/1512005412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d5949f670 msgr2=0x559d5949faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d5949f670 0x559d5949faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3661a4f640 1 -- 172.21.15.64:0/1512005412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 msgr2=0x559d594a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3661a4f640 1 -- 172.21.15.64:0/1512005412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d59387440 con 0x559d594a5000 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.415+0000 7f3652ffd640 1 -- 172.21.15.64:0/1220724309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f364c0f0430 con 0x7f365c1027a0 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.416+0000 7f3662250640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36480a3430 0x7f36480a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3654005f40 tx=0x7f3654004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.416+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d5949f050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f365800bc80 tx=0x7f365800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b950e425fa1ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.416+0000 7f3647fff640 1 -- 172.21.15.64:0/1512005412 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658015070 con 0x559d594a5000 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.416+0000 7f3647fff640 1 -- 172.21.15.64:0/1512005412 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f365801a910 con 0x559d594a5000 2024-09-19T05:11:09.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.416+0000 7f3647fff640 1 -- 172.21.15.64:0/1512005412 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580118a0 con 0x559d594a5000 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 -- 172.21.15.64:0/1512005412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 msgr2=0x559d5949f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d5949f050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f365800bc80 tx=0x7f365800f730 comp rx=0 tx=0).stop 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 -- 172.21.15.64:0/1512005412 shutdown_connections 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d5949f670 0x559d5949faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d5949f050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 --2- 172.21.15.64:0/1512005412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 -- 172.21.15.64:0/1512005412 >> 172.21.15.64:0/1512005412 conn(0x559d594914e0 msgr2=0x559d594918c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 -- 172.21.15.64:0/1512005412 shutdown_connections 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.417+0000 7f366608c740 1 -- 172.21.15.64:0/1512005412 wait complete. 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 -- start start 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 Processor -- start 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594aae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a4650 0x559d594a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559d594aba20 con 0x559d5949f670 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d593b6050 con 0x559d594a5000 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d594abba0 con 0x559d594a4650 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f3661a4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f366608c740 10 client.0 _ll_register_callbacks cb 0x559d593f6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f3661a4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45358/0 (socket says 172.21.15.64:45358) 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.418+0000 7f3661a4f640 1 -- 172.21.15.64:0/1491311563 learned_addr learned my addr 172.21.15.64:0/1491311563 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f366124e640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a4650 0x559d594a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3662250640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3661a4f640 1 -- 172.21.15.64:0/1491311563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a4650 msgr2=0x559d594a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a4650 0x559d594a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3661a4f640 1 -- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 msgr2=0x559d594a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3661a4f640 1 -- 172.21.15.64:0/1491311563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d5916c3c0 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594aae70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f36580115c0 tx=0x7f365800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3303ae21a160408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f36457fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f36457fa640 1 -- 172.21.15.64:0/1491311563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658015070 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f36457fa640 1 -- 172.21.15.64:0/1491311563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f365801aac0 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365801a910 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f36457fa640 1 -- 172.21.15.64:0/1491311563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f365801acc0 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f366608c740 10 client.16581 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.419+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d594b7410 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f36457fa640 1 -- 172.21.15.64:0/1491311563 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f365801a910 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f36457fa640 1 -- 172.21.15.64:0/1491311563 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36580046c0 con 0x559d5949f670 2024-09-19T05:11:09.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366608c740 10 client.16581 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366608c740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366608c740 20 client.16581 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366608c740 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d594b8810 0x559d594bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366608c740 4 client.16581 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d59408850 con 0x559d594b8810 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.420+0000 7f366124e640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d594b8810 0x559d594bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.421+0000 7f366124e640 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d594b8810 0x559d594bac00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be1c70168196ba33 server_cookie=f9e71c3580415a70 in_seq=0 out_seq=0 2024-09-19T05:11:09.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.421+0000 7f36457fa640 10 client.16581 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:09.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.421+0000 7f36457fa640 1 -- 172.21.15.64:0/1491311563 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d59408850 con 0x559d594b8810 2024-09-19T05:11:09.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.421+0000 7f36457fa640 10 client.16581 handle_command_reply: tid=0 2024-09-19T05:11:09.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.421+0000 7f366608c740 1 client.16581 shutdown 2024-09-19T05:11:09.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 msgr2=0x559d594aae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594aae70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f36580115c0 tx=0x7f365800bfa0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 shutdown_connections 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a4650 0x559d594a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d594b8810 0x559d594bac00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 --2- 172.21.15.64:0/1491311563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594aae70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f36457fa640 0 client.16581 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 >> 172.21.15.64:0/1491311563 conn(0x559d594914e0 msgr2=0x559d59490bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 shutdown_connections 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 1 -- 172.21.15.64:0/1491311563 wait complete. 2024-09-19T05:11:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.422+0000 7f366608c740 20 client.16581 trim_cache size 0 max 16384 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.439+0000 7f366608c740 1 Processor -- start 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.439+0000 7f366608c740 1 -- start start 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.439+0000 7f366608c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.439+0000 7f366608c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a4650 0x559d594903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.439+0000 7f366608c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d594908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f366608c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1adf40 con 0x559d5949f670 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f366608c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1ab810 con 0x559d594a4650 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f366608c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1a9660 con 0x559d594a5000 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45372/0 (socket says 172.21.15.64:45372) 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 -- 172.21.15.64:0/3789355895 learned_addr learned my addr 172.21.15.64:0/3789355895 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3662250640 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d594908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f366124e640 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a4650 0x559d594903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 -- 172.21.15.64:0/3789355895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 msgr2=0x559d594908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d594908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 -- 172.21.15.64:0/3789355895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a4650 msgr2=0x559d594903a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a4650 0x559d594903a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 -- 172.21.15.64:0/3789355895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d594abb90 con 0x559d5949f670 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f3661a4f640 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594b4cc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f365800c010 tx=0x7f365800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b113b4b44533427 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f36477fe640 1 -- 172.21.15.64:0/3789355895 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365801ae60 con 0x559d5949f670 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f36477fe640 1 -- 172.21.15.64:0/3789355895 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3658004680 con 0x559d5949f670 2024-09-19T05:11:09.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.440+0000 7f36477fe640 1 -- 172.21.15.64:0/3789355895 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658015040 con 0x559d5949f670 2024-09-19T05:11:09.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 -- 172.21.15.64:0/3789355895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 msgr2=0x559d594b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594b4cc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f365800c010 tx=0x7f365800b980 comp rx=0 tx=0).stop 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 -- 172.21.15.64:0/3789355895 shutdown_connections 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d594a5000 0x559d594908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a4650 0x559d594903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 --2- 172.21.15.64:0/3789355895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d5949f670 0x559d594b4cc0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 -- 172.21.15.64:0/3789355895 >> 172.21.15.64:0/3789355895 conn(0x559d594914e0 msgr2=0x559d594a7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 -- 172.21.15.64:0/3789355895 shutdown_connections 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.441+0000 7f366608c740 1 -- 172.21.15.64:0/3789355895 wait complete. 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 -- start start 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 Processor -- start 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d5949f670 0x559d594bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1a9cd0 con 0x559d594a4650 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1ab140 con 0x559d594a5000 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1aa3a0 con 0x559d5949f670 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366608c740 10 client.0 _ll_register_callbacks cb 0x7f3654009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366124e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366124e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594c0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45386/0 (socket says 172.21.15.64:45386) 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f366124e640 1 -- 172.21.15.64:0/1854219737 learned_addr learned my addr 172.21.15.64:0/1854219737 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.442+0000 7f3662250640 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366124e640 1 -- 172.21.15.64:0/1854219737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d5949f670 msgr2=0x559d594bfdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366124e640 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d5949f670 0x559d594bfdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366124e640 1 -- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 msgr2=0x559d594c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366124e640 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594c0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366124e640 1 -- 172.21.15.64:0/1854219737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d595221d0 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366124e640 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594c0330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f364c0bc6a0 tx=0x7f364c0bc130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ded8c7f55908dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f36457fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f36457fa640 1 -- 172.21.15.64:0/1854219737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c049040 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f36457fa640 1 -- 172.21.15.64:0/1854219737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f364c003c50 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3658004680 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f36457fa640 1 -- 172.21.15.64:0/1854219737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f364c0bc710 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366608c740 10 client.16587 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d59523530 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.443+0000 7f36457fa640 1 -- 172.21.15.64:0/1854219737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364c0c5380 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f36457fa640 1 -- 172.21.15.64:0/1854219737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f364c0a1e80 con 0x559d594a4650 2024-09-19T05:11:09.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f366608c740 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f366608c740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f366608c740 20 client.16587 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f366608c740 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d59523ad0 0x559d59525ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f366608c740 4 client.16587 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3658004680 con 0x559d59523ad0 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d59523ad0 0x559d59525ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f3661a4f640 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d59523ad0 0x559d59525ef0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19510caf4aebd4da server_cookie=26df81101ba729cc in_seq=0 out_seq=0 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.444+0000 7f36457fa640 10 client.16587 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f36457fa640 1 -- 172.21.15.64:0/1854219737 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f3658004680 con 0x559d59523ad0 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f36457fa640 10 client.16587 handle_command_reply: tid=0 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 client.16587 shutdown 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 msgr2=0x559d594c0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594c0330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f364c0bc6a0 tx=0x7f364c0bc130 comp rx=0 tx=0).stop 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 shutdown_connections 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d594a5000 0x559d594c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559d59523ad0 0x559d59525ef0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d594a4650 0x559d594c0330 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 --2- 172.21.15.64:0/1854219737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d5949f670 0x559d594bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f36457fa640 0 client.16587 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 >> 172.21.15.64:0/1854219737 conn(0x559d594914e0 msgr2=0x559d594903a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 shutdown_connections 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 1 -- 172.21.15.64:0/1854219737 wait complete. 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.445+0000 7f366608c740 20 client.16587 trim_cache size 0 max 16384 2024-09-19T05:11:09.451 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:09.452 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:11:09.452 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:11:09.452 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36480a3430 msgr2=0x7f36480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36480a3430 0x7f36480a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3654005f40 tx=0x7f3654004ab0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 msgr2=0x7f365c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1988b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f364c005940 tx=0x7f364c004270 comp rx=0 tx=0).stop 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 shutdown_connections 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36480a3430 0x7f36480a58f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f365c106f20 0x7f365c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f365c1027a0 0x7f365c1988b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 --2- 172.21.15.64:0/1220724309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f365c101df0 0x7f365c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 >> 172.21.15.64:0/1220724309 conn(0x7f365c0fd8f0 msgr2=0x7f365c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 shutdown_connections 2024-09-19T05:11:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:09.446+0000 7f3663cb3640 1 -- 172.21.15.64:0/1220724309 wait complete. 2024-09-19T05:11:10.299 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:11:10.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:11:10.558 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:10.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:10 smithi064 ceph-mon[30960]: pgmap v161: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 -- 172.21.15.64:0/558387168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 msgr2=0x7f1e501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 --2- 172.21.15.64:0/558387168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e501021c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009900 tx=0x7f1e4002a410 comp rx=0 tx=0).stop 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 -- 172.21.15.64:0/558387168 shutdown_connections 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 --2- 172.21.15.64:0/558387168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e50106f10 0x7f1e5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 --2- 172.21.15.64:0/558387168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e50102790 0x7f1e501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 --2- 172.21.15.64:0/558387168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e501021c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 -- 172.21.15.64:0/558387168 >> 172.21.15.64:0/558387168 conn(0x7f1e500fd8c0 msgr2=0x7f1e500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:11.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 -- 172.21.15.64:0/558387168 shutdown_connections 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.048+0000 7f1e56289640 1 -- 172.21.15.64:0/558387168 wait complete. 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 Processor -- start 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 -- start start 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50111e20 con 0x7f1e50101de0 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50111ca0 con 0x7f1e50102790 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e56289640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50111fa0 con 0x7f1e50106f10 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45408/0 (socket says 172.21.15.64:45408) 2024-09-19T05:11:11.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.049+0000 7f1e4ffff640 1 -- 172.21.15.64:0/951385716 learned_addr learned my addr 172.21.15.64:0/951385716 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e47fff640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e54826640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4ffff640 1 -- 172.21.15.64:0/951385716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e50106f10 msgr2=0x7f1e50074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4ffff640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4ffff640 1 -- 172.21.15.64:0/951385716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e50102790 msgr2=0x7f1e50074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4ffff640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4ffff640 1 -- 172.21.15.64:0/951385716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501a6840 con 0x7f1e50101de0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4ffff640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005940 tx=0x7f1e40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4003f030 con 0x7f1e50101de0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e40003aa0 con 0x7f1e50101de0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501a6ad0 con 0x7f1e50101de0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.050+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40035070 con 0x7f1e50101de0 2024-09-19T05:11:11.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.051+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501a6fb0 con 0x7f1e50101de0 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.052+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e501021c0 con 0x7f1e50101de0 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.053+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1e40042030 con 0x7f1e50101de0 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.053+0000 7f1e4dffb640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.054+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1e400f0820 con 0x7f1e50101de0 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.054+0000 7f1e47fff640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.054+0000 7f1e47fff640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e38005e10 tx=0x7f1e38005da0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:11.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.057+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e400bd200 con 0x7f1e50101de0 2024-09-19T05:11:11.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:10 smithi103 ceph-mon[34446]: pgmap v161: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.269+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e5010a0f0 con 0x7f1e50101de0 2024-09-19T05:11:11.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.271+0000 7f1e4dffb640 1 -- 172.21.15.64:0/951385716 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f1e400bcb00 con 0x7f1e50101de0 2024-09-19T05:11:11.272 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:11:11.272 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:11:11.273 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:11:11.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e3c0a33b0 msgr2=0x7f1e3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:11.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e38005e10 tx=0x7f1e38005da0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 msgr2=0x7f1e500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005940 tx=0x7f1e40004270 comp rx=0 tx=0).stop 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 shutdown_connections 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 --2- 172.21.15.64:0/951385716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 >> 172.21.15.64:0/951385716 conn(0x7f1e500fd8c0 msgr2=0x7f1e500fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 shutdown_connections 2024-09-19T05:11:11.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:11.273+0000 7f1e56289640 1 -- 172.21.15.64:0/951385716 wait complete. 2024-09-19T05:11:11.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:10 smithi174 ceph-mon[34488]: pgmap v161: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:12.046 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:11 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/951385716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:12.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:11:12.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:11 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/951385716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:12.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:11 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/951385716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:12.338 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 -- 172.21.15.64:0/453858984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101420 msgr2=0x7fdac810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 --2- 172.21.15.64:0/453858984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101420 0x7fdac810d950 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009cd0 tx=0x7fdac002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 -- 172.21.15.64:0/453858984 shutdown_connections 2024-09-19T05:11:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 --2- 172.21.15.64:0/453858984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8101420 0x7fdac810d950 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 --2- 172.21.15.64:0/453858984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8100a80 0x7fdac8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 --2- 172.21.15.64:0/453858984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8106a80 0x7fdac8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 -- 172.21.15.64:0/453858984 >> 172.21.15.64:0/453858984 conn(0x7fdac80fc7a0 msgr2=0x7fdac80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:12.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 -- 172.21.15.64:0/453858984 shutdown_connections 2024-09-19T05:11:12.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.823+0000 7fdacd139640 1 -- 172.21.15.64:0/453858984 wait complete. 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.824+0000 7fdacd139640 1 Processor -- start 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.824+0000 7fdacd139640 1 -- start start 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.824+0000 7fdacd139640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8100a80 0x7fdac8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.824+0000 7fdacd139640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101420 0x7fdac8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdacd139640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 0x7fdac819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdacd139640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8110d30 con 0x7fdac8106a80 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdacd139640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8110bb0 con 0x7fdac8101420 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdacd139640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8110eb0 con 0x7fdac8100a80 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 0x7fdac819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 0x7fdac819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45430/0 (socket says 172.21.15.64:45430) 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 -- 172.21.15.64:0/29122554 learned_addr learned my addr 172.21.15.64:0/29122554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac659c640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101420 0x7fdac8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8100a80 0x7fdac8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 -- 172.21.15.64:0/29122554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8100a80 msgr2=0x7fdac8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8100a80 0x7fdac8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 -- 172.21.15.64:0/29122554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101420 msgr2=0x7fdac8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101420 0x7fdac8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 -- 172.21.15.64:0/29122554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac819d2d0 con 0x7fdac8106a80 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8100a80 0x7fdac8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.825+0000 7fdac759e640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 0x7fdac819cbf0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005cc0 tx=0x7fdac0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.826+0000 7fdaaffff640 1 -- 172.21.15.64:0/29122554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0004240 con 0x7fdac8106a80 2024-09-19T05:11:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.826+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac819d560 con 0x7fdac8106a80 2024-09-19T05:11:12.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.826+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a4ea0 con 0x7fdac8106a80 2024-09-19T05:11:12.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.826+0000 7fdaaffff640 1 -- 172.21.15.64:0/29122554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdac00305a0 con 0x7fdac8106a80 2024-09-19T05:11:12.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.826+0000 7fdaaffff640 1 -- 172.21.15.64:0/29122554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0046480 con 0x7fdac8106a80 2024-09-19T05:11:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.828+0000 7fdaaffff640 1 -- 172.21.15.64:0/29122554 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fdac0030aa0 con 0x7fdac8106a80 2024-09-19T05:11:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.828+0000 7fdacf512740 1 Processor -- start 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.828+0000 7fdacf512740 1 -- start start 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdacf512740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdacf512740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 0x55bb63f1ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdacf512740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdacf512740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bb63e822f0 con 0x55bb63f1c5d0 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdacf512740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb63e29cf0 con 0x55bb63f27370 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdacf512740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb63e7e2b0 con 0x55bb63f1db40 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 0x55bb63f1ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdaaffff640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdab00a33b0 0x7fdab00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 0x55bb63f1ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45444/0 (socket says 172.21.15.64:45444) 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1885571120 learned_addr learned my addr 172.21.15.64:0/1885571120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac759e640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdab00a33b0 0x7fdab00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdaaffff640 1 -- 172.21.15.64:0/29122554 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdac00f13c0 con 0x7fdac8106a80 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac659c640 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1885571120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 msgr2=0x55bb63f1c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f1c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1885571120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 msgr2=0x55bb63f29760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1885571120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb63e0e3a0 con 0x55bb63f1c5d0 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac659c640 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 0x55bb63f1ca10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdab8008be0 tx=0x7fdab800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b9a437feb36a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.829+0000 7fdac759e640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdab00a33b0 0x7fdab00a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdac00bd4b0 tx=0x7fdac00a8040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdaad7fa640 1 -- 172.21.15.64:0/1885571120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8014070 con 0x55bb63f1c5d0 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdaad7fa640 1 -- 172.21.15.64:0/1885571120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab80047a0 con 0x55bb63f1c5d0 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdaad7fa640 1 -- 172.21.15.64:0/1885571120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8010690 con 0x55bb63f1c5d0 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 -- 172.21.15.64:0/1885571120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 msgr2=0x55bb63f1ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 0x55bb63f1ca10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdab8008be0 tx=0x7fdab800e720 comp rx=0 tx=0).stop 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 -- 172.21.15.64:0/1885571120 shutdown_connections 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1c5d0 0x55bb63f1ca10 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 --2- 172.21.15.64:0/1885571120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 -- 172.21.15.64:0/1885571120 >> 172.21.15.64:0/1885571120 conn(0x55bb63ee5310 msgr2=0x55bb63f18c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 -- 172.21.15.64:0/1885571120 shutdown_connections 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.830+0000 7fdacf512740 1 -- 172.21.15.64:0/1885571120 wait complete. 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 -- start start 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 Processor -- start 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f26dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f24080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f24810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bb63f2e480 con 0x55bb63f27370 2024-09-19T05:11:12.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb63e3d060 con 0x55bb63f1c5d0 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb63f2e600 con 0x55bb63f1db40 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdac659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f24810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdacf512740 10 client.0 _ll_register_callbacks cb 0x55bb63b908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdac659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f24810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45454/0 (socket says 172.21.15.64:45454) 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdac659c640 1 -- 172.21.15.64:0/248287189 learned_addr learned my addr 172.21.15.64:0/248287189 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdac659c640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f26dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.831+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f24080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdac659c640 1 -- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 msgr2=0x55bb63f24080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdac659c640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f24080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdac659c640 1 -- 172.21.15.64:0/248287189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 msgr2=0x55bb63f26dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdac659c640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f26dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdac659c640 1 -- 172.21.15.64:0/248287189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb63bf33c0 con 0x55bb63f27370 2024-09-19T05:11:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdac659c640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f24810 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdabc010e60 tx=0x7fdabc00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd3cff6b8a6e837 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdaaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdaaaff5640 1 -- 172.21.15.64:0/248287189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00ee50 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdaaaff5640 1 -- 172.21.15.64:0/248287189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdabc004020 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab80047a0 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdaaaff5640 1 -- 172.21.15.64:0/248287189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdabc004220 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdacf512740 10 client.16605 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb63f39fc0 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.832+0000 7fdaaaff5640 1 -- 172.21.15.64:0/248287189 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab80047a0 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdaaaff5640 1 -- 172.21.15.64:0/248287189 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdabc01c030 con 0x55bb63f27370 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdacf512740 10 client.16605 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdacf512740 10 client.16605 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdacf512740 20 client.16605 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdacf512740 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63f3b3d0 0x55bb63f3d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdacf512740 4 client.16605 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb63e8f860 con 0x55bb63f3b3d0 2024-09-19T05:11:12.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63f3b3d0 0x55bb63f3d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdaaaff5640 10 client.16605 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.833+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63f3b3d0 0x55bb63f3d7e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ea1d243c9614d62 server_cookie=24260ae46a927a48 in_seq=0 out_seq=0 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdaaaff5640 1 -- 172.21.15.64:0/248287189 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb63e8f860 con 0x55bb63f3b3d0 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdaaaff5640 10 client.16605 handle_command_reply: tid=0 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 client.16605 shutdown 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 msgr2=0x55bb63f24810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f24810 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdabc010e60 tx=0x7fdabc00a6a0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 shutdown_connections 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63f3b3d0 0x55bb63f3d7e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f24810 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f24080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 --2- 172.21.15.64:0/248287189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f26dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 >> 172.21.15.64:0/248287189 conn(0x55bb63ee5310 msgr2=0x55bb63f1e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.834+0000 7fdaaaff5640 0 client.16605 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.835+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 shutdown_connections 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.835+0000 7fdacf512740 1 -- 172.21.15.64:0/248287189 wait complete. 2024-09-19T05:11:12.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.835+0000 7fdacf512740 20 client.16605 trim_cache size 0 max 16384 2024-09-19T05:11:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 Processor -- start 2024-09-19T05:11:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 -- start start 2024-09-19T05:11:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f18e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f23940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81ade20 con 0x55bb63f27370 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81ab6f0 con 0x55bb63f1c5d0 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdacf512740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81a9560 con 0x55bb63f1db40 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.848+0000 7fdac659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f1f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45462/0 (socket says 172.21.15.64:45462) 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 -- 172.21.15.64:0/1900932966 learned_addr learned my addr 172.21.15.64:0/1900932966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f23940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f18e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 -- 172.21.15.64:0/1900932966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 msgr2=0x55bb63f23940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f23940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 -- 172.21.15.64:0/1900932966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 msgr2=0x55bb63f18e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f18e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 -- 172.21.15.64:0/1900932966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb63f19cf0 con 0x55bb63f27370 2024-09-19T05:11:12.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdac659c640 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f1f120 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0109e0 tx=0x7fdabc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bdd297a1a5d100 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdaacff9640 1 -- 172.21.15.64:0/1900932966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00a2d0 con 0x55bb63f27370 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdaacff9640 1 -- 172.21.15.64:0/1900932966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc02d070 con 0x55bb63f27370 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.849+0000 7fdaacff9640 1 -- 172.21.15.64:0/1900932966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc004a70 con 0x55bb63f27370 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 -- 172.21.15.64:0/1900932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 msgr2=0x55bb63f1f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f1f120 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0109e0 tx=0x7fdabc0043c0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 -- 172.21.15.64:0/1900932966 shutdown_connections 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f27370 0x55bb63f1f120 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1db40 0x55bb63f23940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 --2- 172.21.15.64:0/1900932966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f1c5d0 0x55bb63f18e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 -- 172.21.15.64:0/1900932966 >> 172.21.15.64:0/1900932966 conn(0x55bb63ee5310 msgr2=0x55bb63f18700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 -- 172.21.15.64:0/1900932966 shutdown_connections 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 -- 172.21.15.64:0/1900932966 wait complete. 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 -- start start 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.850+0000 7fdacf512740 1 Processor -- start 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1c5d0 0x55bb63f33890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 0x55bb63f33dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81a9bb0 con 0x55bb63f1db40 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81ab020 con 0x55bb63f27370 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81aa280 con 0x55bb63f1c5d0 2024-09-19T05:11:12.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdacf512740 10 client.0 _ll_register_callbacks cb 0x7fdabc00aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 0x55bb63f33dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 0x55bb63f33dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45476/0 (socket says 172.21.15.64:45476) 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1480769264 learned_addr learned my addr 172.21.15.64:0/1480769264 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac659c640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1c5d0 0x55bb63f33890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac659c640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1c5d0 msgr2=0x55bb63f33890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1c5d0 0x55bb63f33890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1480769264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 msgr2=0x55bb63f206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 -- 172.21.15.64:0/1480769264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb63f20c30 con 0x55bb63f1db40 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac659c640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:12.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdac6d9d640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 0x55bb63f33dd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005da0 tx=0x7fdab8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775ca2be6810493f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdaaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.851+0000 7fdaaaff5640 1 -- 172.21.15.64:0/1480769264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8029030 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdaaaff5640 1 -- 172.21.15.64:0/1480769264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fdab8004f90 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc02d070 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdaaaff5640 1 -- 172.21.15.64:0/1480769264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdab8014040 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 10 client.16611 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb63fa5760 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdaaaff5640 1 -- 172.21.15.64:0/1480769264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdabc02d070 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdaaaff5640 1 -- 172.21.15.64:0/1480769264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdab8037860 con 0x55bb63f1db40 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 10 client.16611 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 10 client.16611 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:12.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 20 client.16611 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63fa5c00 0x55bb63fa7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 4 client.16611 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb63e89d70 con 0x55bb63fa5c00 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.852+0000 7fdac659c640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63fa5c00 0x55bb63fa7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdac659c640 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63fa5c00 0x55bb63fa7ff0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3ecc8676d0419e server_cookie=78947f795829c98d in_seq=0 out_seq=0 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdaaaff5640 10 client.16611 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdaaaff5640 1 -- 172.21.15.64:0/1480769264 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb63e89d70 con 0x55bb63fa5c00 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdaaaff5640 10 client.16611 handle_command_reply: tid=0 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 client.16611 shutdown 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 msgr2=0x55bb63f33dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 0x55bb63f33dd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005da0 tx=0x7fdab8005dd0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 shutdown_connections 2024-09-19T05:11:12.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55bb63fa5c00 0x55bb63fa7ff0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb63f27370 0x55bb63f206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bb63f1db40 0x55bb63f33dd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 --2- 172.21.15.64:0/1480769264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb63f1c5d0 0x55bb63f33890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 >> 172.21.15.64:0/1480769264 conn(0x55bb63ee5310 msgr2=0x55bb63f174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdaaaff5640 0 client.16611 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 shutdown_connections 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 1 -- 172.21.15.64:0/1480769264 wait complete. 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.853+0000 7fdacf512740 20 client.16611 trim_cache size 0 max 16384 2024-09-19T05:11:12.861 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdab00a33b0 msgr2=0x7fdab00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdab00a33b0 0x7fdab00a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdac00bd4b0 tx=0x7fdac00a8040 comp rx=0 tx=0).stop 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 msgr2=0x7fdac819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 0x7fdac819cbf0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005cc0 tx=0x7fdac0003ba0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 shutdown_connections 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdab00a33b0 0x7fdab00a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac8106a80 0x7fdac819cbf0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac8101420 0x7fdac8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 --2- 172.21.15.64:0/29122554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdac8100a80 0x7fdac8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 >> 172.21.15.64:0/29122554 conn(0x7fdac80fc7a0 msgr2=0x7fdac806a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 shutdown_connections 2024-09-19T05:11:12.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:12.854+0000 7fdacd139640 1 -- 172.21.15.64:0/29122554 wait complete. 2024-09-19T05:11:13.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:12 smithi064 ceph-mon[30960]: pgmap v162: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:13.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:12 smithi103 ceph-mon[34446]: pgmap v162: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:13.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:12 smithi174 ceph-mon[34488]: pgmap v162: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:13.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:11:13.859 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:14.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 -- 172.21.15.64:0/610539177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 msgr2=0x7fceac10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 --2- 172.21.15.64:0/610539177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac10eaa0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009ca0 tx=0x7fcea402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 -- 172.21.15.64:0/610539177 shutdown_connections 2024-09-19T05:11:14.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 --2- 172.21.15.64:0/610539177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac10eaa0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 --2- 172.21.15.64:0/610539177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceac1027a0 0x7fceac1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 --2- 172.21.15.64:0/610539177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fceac101df0 0x7fceac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 -- 172.21.15.64:0/610539177 >> 172.21.15.64:0/610539177 conn(0x7fceac0fd8b0 msgr2=0x7fceac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:14.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 -- 172.21.15.64:0/610539177 shutdown_connections 2024-09-19T05:11:14.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 -- 172.21.15.64:0/610539177 wait complete. 2024-09-19T05:11:14.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 Processor -- start 2024-09-19T05:11:14.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.316+0000 7fceb2502640 1 -- start start 2024-09-19T05:11:14.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb2502640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceac101df0 0x7fceac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb2502640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fceac1027a0 0x7fceac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb2502640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb2502640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceac111e30 con 0x7fceac106f20 2024-09-19T05:11:14.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb2502640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fceac111cb0 con 0x7fceac1027a0 2024-09-19T05:11:14.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb2502640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fceac111fb0 con 0x7fceac101df0 2024-09-19T05:11:14.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33100/0 (socket says 172.21.15.64:33100) 2024-09-19T05:11:14.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 -- 172.21.15.64:0/3869228229 learned_addr learned my addr 172.21.15.64:0/3869228229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceabfff640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceac101df0 0x7fceac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fcea3fff640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fceac1027a0 0x7fceac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 -- 172.21.15.64:0/3869228229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceac101df0 msgr2=0x7fceac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceac101df0 0x7fceac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 -- 172.21.15.64:0/3869228229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fceac1027a0 msgr2=0x7fceac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fceac1027a0 0x7fceac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 -- 172.21.15.64:0/3869228229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac1a67f0 con 0x7fceac106f20 2024-09-19T05:11:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.317+0000 7fceb0a9f640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac074f70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009dc0 tx=0x7fcea4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.318+0000 7fcea9ffb640 1 -- 172.21.15.64:0/3869228229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4003cf0 con 0x7fceac106f20 2024-09-19T05:11:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.318+0000 7fcea9ffb640 1 -- 172.21.15.64:0/3869228229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea40044f0 con 0x7fceac106f20 2024-09-19T05:11:14.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.318+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac1a6a80 con 0x7fceac106f20 2024-09-19T05:11:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.318+0000 7fcea9ffb640 1 -- 172.21.15.64:0/3869228229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4030bf0 con 0x7fceac106f20 2024-09-19T05:11:14.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.318+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac1a7010 con 0x7fceac106f20 2024-09-19T05:11:14.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.320+0000 7fcea9ffb640 1 -- 172.21.15.64:0/3869228229 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fcea4004040 con 0x7fceac106f20 2024-09-19T05:11:14.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.320+0000 7fceb48db740 1 Processor -- start 2024-09-19T05:11:14.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.320+0000 7fceb48db740 1 -- start start 2024-09-19T05:11:14.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.320+0000 7fceb48db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.320+0000 7fceb48db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d55d0 0x55faf65d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fceb48db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fceb48db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faf653b2f0 con 0x55faf65e0370 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fceb48db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55faf64e2cf0 con 0x55faf65d6b40 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fceb48db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55faf65372b0 con 0x55faf65d55d0 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33102/0 (socket says 172.21.15.64:33102) 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 -- 172.21.15.64:0/1581957812 learned_addr learned my addr 172.21.15.64:0/1581957812 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fceabfff640 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d55d0 0x55faf65d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea9ffb640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fce980a3430 0x7fce980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fceb0a9f640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fce980a3430 0x7fce980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea9ffb640 1 -- 172.21.15.64:0/3869228229 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fcea40f1110 con 0x7fceac106f20 2024-09-19T05:11:14.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 -- 172.21.15.64:0/1581957812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d55d0 msgr2=0x55faf65d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d55d0 0x55faf65d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 -- 172.21.15.64:0/1581957812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 msgr2=0x55faf65d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 -- 172.21.15.64:0/1581957812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf64c73a0 con 0x55faf65e0370 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.321+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65e2760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce94012720 tx=0x7fce9400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2fbd6cd3bd5023 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb0a9f640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fce980a3430 0x7fce980a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcea40eee60 tx=0x7fcea408f280 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fcea27fc640 1 -- 172.21.15.64:0/1581957812 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94012ca0 con 0x55faf65e0370 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fcea27fc640 1 -- 172.21.15.64:0/1581957812 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce94012e40 con 0x55faf65e0370 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fcea27fc640 1 -- 172.21.15.64:0/1581957812 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94004a00 con 0x55faf65e0370 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 -- 172.21.15.64:0/1581957812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 msgr2=0x55faf65e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65e2760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce94012720 tx=0x7fce9400b6e0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 -- 172.21.15.64:0/1581957812 shutdown_connections 2024-09-19T05:11:14.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65e2760 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d55d0 0x55faf65d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 --2- 172.21.15.64:0/1581957812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.322+0000 7fceb48db740 1 -- 172.21.15.64:0/1581957812 >> 172.21.15.64:0/1581957812 conn(0x55faf659e310 msgr2=0x55faf65d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 -- 172.21.15.64:0/1581957812 shutdown_connections 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 -- 172.21.15.64:0/1581957812 wait complete. 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 -- start start 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 Processor -- start 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 0x55faf65dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d6b40 0x55faf65dcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faf65e3070 con 0x55faf65d55d0 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55faf64f6060 con 0x55faf65e0370 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55faf65e31f0 con 0x55faf65d6b40 2024-09-19T05:11:14.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fceb48db740 10 client.0 _ll_register_callbacks cb 0x55faf62498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 0x55faf65dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 0x55faf65dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33106/0 (socket says 172.21.15.64:33106) 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.323+0000 7fcea3fff640 1 -- 172.21.15.64:0/4082645529 learned_addr learned my addr 172.21.15.64:0/4082645529 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fceabfff640 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d6b40 0x55faf65dcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fcea3fff640 1 -- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d6b40 msgr2=0x55faf65dcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fcea3fff640 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d6b40 0x55faf65dcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fcea3fff640 1 -- 172.21.15.64:0/4082645529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 msgr2=0x55faf65dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fcea3fff640 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fcea3fff640 1 -- 172.21.15.64:0/4082645529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf62ac3c0 con 0x55faf65d55d0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fcea3fff640 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 0x55faf65dfd30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fce94004600 tx=0x7fce9400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6b0a09891578a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fce8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fce8b7fe640 1 -- 172.21.15.64:0/4082645529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94010970 con 0x55faf65d55d0 2024-09-19T05:11:14.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fce8b7fe640 1 -- 172.21.15.64:0/4082645529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fce94010b10 con 0x55faf65d55d0 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55faf6548860 con 0x55faf65d55d0 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fce8b7fe640 1 -- 172.21.15.64:0/4082645529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce94010d10 con 0x55faf65d55d0 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fceb48db740 10 client.16623 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.324+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faf65eeb00 con 0x55faf65d55d0 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fce8b7fe640 1 -- 172.21.15.64:0/4082645529 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55faf6548860 con 0x55faf65d55d0 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fce8b7fe640 1 -- 172.21.15.64:0/4082645529 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fce9401f680 con 0x55faf65d55d0 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceb48db740 10 client.16623 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceb48db740 10 client.16623 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceb48db740 20 client.16623 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceb48db740 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf65efe50 0x55faf65f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceb48db740 4 client.16623 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faf6542d70 con 0x55faf65efe50 2024-09-19T05:11:14.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.325+0000 7fceabfff640 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf65efe50 0x55faf65f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.326+0000 7fceabfff640 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf65efe50 0x55faf65f2240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89ada429a7350943 server_cookie=b9d36775650fd444 in_seq=0 out_seq=0 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.326+0000 7fce8b7fe640 10 client.16623 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.326+0000 7fce8b7fe640 1 -- 172.21.15.64:0/4082645529 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faf6542d70 con 0x55faf65efe50 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.326+0000 7fce8b7fe640 10 client.16623 handle_command_reply: tid=0 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.326+0000 7fceb48db740 1 client.16623 shutdown 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 msgr2=0x55faf65dfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 0x55faf65dfd30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fce94004600 tx=0x7fce9400b400 comp rx=0 tx=0).stop 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 shutdown_connections 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d6b40 0x55faf65dcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf65efe50 0x55faf65f2240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 --2- 172.21.15.64:0/4082645529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d55d0 0x55faf65dfd30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 >> 172.21.15.64:0/4082645529 conn(0x55faf659e310 msgr2=0x55faf65d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:14.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fce8b7fe640 0 client.16623 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 shutdown_connections 2024-09-19T05:11:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 1 -- 172.21.15.64:0/4082645529 wait complete. 2024-09-19T05:11:14.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.327+0000 7fceb48db740 20 client.16623 trim_cache size 0 max 16384 2024-09-19T05:11:14.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 Processor -- start 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 -- start start 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 0x55faf65ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1b22b0 con 0x55faf65d6b40 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1afb80 con 0x55faf65e0370 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceb48db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1ad9d0 con 0x55faf65d1ea0 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 0x55faf65ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 0x55faf65ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33118/0 (socket says 172.21.15.64:33118) 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fcea3fff640 1 -- 172.21.15.64:0/78379561 learned_addr learned my addr 172.21.15.64:0/78379561 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fceabfff640 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fcea3fff640 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.340+0000 7fcea3fff640 1 -- 172.21.15.64:0/78379561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 msgr2=0x55faf65f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea3fff640 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea3fff640 1 -- 172.21.15.64:0/78379561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 msgr2=0x55faf65dbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea3fff640 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea3fff640 1 -- 172.21.15.64:0/78379561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf65db550 con 0x55faf65d6b40 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea3fff640 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 0x55faf65ec340 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fce9401fc40 tx=0x7fce940056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc32e769b34d12e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea1ffb640 1 -- 172.21.15.64:0/78379561 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94012070 con 0x55faf65d6b40 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea1ffb640 1 -- 172.21.15.64:0/78379561 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce94016050 con 0x55faf65d6b40 2024-09-19T05:11:14.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fcea1ffb640 1 -- 172.21.15.64:0/78379561 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94004630 con 0x55faf65d6b40 2024-09-19T05:11:14.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fceb48db740 1 -- 172.21.15.64:0/78379561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 msgr2=0x55faf65ec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.341+0000 7fceb48db740 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 0x55faf65ec340 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fce9401fc40 tx=0x7fce940056a0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- 172.21.15.64:0/78379561 shutdown_connections 2024-09-19T05:11:14.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65e0370 0x55faf65dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 --2- 172.21.15.64:0/78379561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65d6b40 0x55faf65ec340 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- 172.21.15.64:0/78379561 >> 172.21.15.64:0/78379561 conn(0x55faf659e310 msgr2=0x55faf65db8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- 172.21.15.64:0/78379561 shutdown_connections 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- 172.21.15.64:0/78379561 wait complete. 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- start start 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 Processor -- start 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1ae040 con 0x55faf65e0370 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1af4b0 con 0x55faf65d6b40 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.342+0000 7fceb48db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fceac1ae710 con 0x55faf65d1ea0 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fceb48db740 10 client.0 _ll_register_callbacks cb 0x7fce9400bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65d9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33128/0 (socket says 172.21.15.64:33128) 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 -- 172.21.15.64:0/1475976118 learned_addr learned my addr 172.21.15.64:0/1475976118 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:14.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fceabfff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 -- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 msgr2=0x55faf65e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 -- 172.21.15.64:0/1475976118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 msgr2=0x55faf65e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 -- 172.21.15.64:0/1475976118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf65da3e0 con 0x55faf65e0370 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fcea3fff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65d9be0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fce9401fbc0 tx=0x7fce9401f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fba1027c223418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:14.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fce8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fce8b7fe640 1 -- 172.21.15.64:0/1475976118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94010740 con 0x55faf65e0370 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fce8b7fe640 1 -- 172.21.15.64:0/1475976118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fce940108e0 con 0x55faf65e0370 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.343+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce94016050 con 0x55faf65e0370 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fce8b7fe640 1 -- 172.21.15.64:0/1475976118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce94010b00 con 0x55faf65e0370 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 10 client.16629 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faf665a650 con 0x55faf65e0370 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fce8b7fe640 1 -- 172.21.15.64:0/1475976118 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce94016050 con 0x55faf65e0370 2024-09-19T05:11:14.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fce8b7fe640 1 -- 172.21.15.64:0/1475976118 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fce940484c0 con 0x55faf65e0370 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 10 client.16629 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 10 client.16629 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 20 client.16629 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf665ac50 0x55faf665d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 4 client.16629 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faf64bf170 con 0x55faf665ac50 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.344+0000 7fceabfff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf665ac50 0x55faf665d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceabfff640 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf665ac50 0x55faf665d070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7941daaabff07b0e server_cookie=21fc0cc5cd6567bf in_seq=0 out_seq=0 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fce8b7fe640 10 client.16629 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fce8b7fe640 1 -- 172.21.15.64:0/1475976118 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faf64bf170 con 0x55faf665ac50 2024-09-19T05:11:14.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fce8b7fe640 10 client.16629 handle_command_reply: tid=0 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 client.16629 shutdown 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 msgr2=0x55faf65d9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65d9be0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fce9401fbc0 tx=0x7fce9401f5a0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 shutdown_connections 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55faf665ac50 0x55faf665d070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf65e0370 0x55faf65d9be0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55faf65d6b40 0x55faf65e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 --2- 172.21.15.64:0/1475976118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faf65d1ea0 0x55faf65e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 >> 172.21.15.64:0/1475976118 conn(0x55faf659e310 msgr2=0x55faf65dc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fce8b7fe640 0 client.16629 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 shutdown_connections 2024-09-19T05:11:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 1 -- 172.21.15.64:0/1475976118 wait complete. 2024-09-19T05:11:14.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.345+0000 7fceb48db740 20 client.16629 trim_cache size 0 max 16384 2024-09-19T05:11:14.357 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fce980a3430 msgr2=0x7fce980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fce980a3430 0x7fce980a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcea40eee60 tx=0x7fcea408f280 comp rx=0 tx=0).stop 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 msgr2=0x7fceac074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac074f70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009dc0 tx=0x7fcea4032ff0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 shutdown_connections 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fce980a3430 0x7fce980a58f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fceac106f20 0x7fceac074f70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fceac1027a0 0x7fceac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 --2- 172.21.15.64:0/3869228229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceac101df0 0x7fceac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:14.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 >> 172.21.15.64:0/3869228229 conn(0x7fceac0fd8b0 msgr2=0x7fceac0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:14.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 shutdown_connections 2024-09-19T05:11:14.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:14.346+0000 7fceb2502640 1 -- 172.21.15.64:0/3869228229 wait complete. 2024-09-19T05:11:15.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:14 smithi064 ceph-mon[30960]: pgmap v163: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:15.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:11:15.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:14 smithi103 ceph-mon[34446]: pgmap v163: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:15.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:14 smithi174 ceph-mon[34488]: pgmap v163: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:11:15.364 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.807+0000 7f22147d6640 1 -- 172.21.15.64:0/723801768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c106f10 msgr2=0x7f220c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.807+0000 7f22147d6640 1 --2- 172.21.15.64:0/723801768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c106f10 0x7f220c10ea90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2204009ca0 tx=0x7f220402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 -- 172.21.15.64:0/723801768 shutdown_connections 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 --2- 172.21.15.64:0/723801768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c106f10 0x7f220c10ea90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 --2- 172.21.15.64:0/723801768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 0x7f220c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 --2- 172.21.15.64:0/723801768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f220c101de0 0x7f220c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 -- 172.21.15.64:0/723801768 >> 172.21.15.64:0/723801768 conn(0x7f220c0fd8c0 msgr2=0x7f220c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 -- 172.21.15.64:0/723801768 shutdown_connections 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 -- 172.21.15.64:0/723801768 wait complete. 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 Processor -- start 2024-09-19T05:11:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.808+0000 7f22147d6640 1 -- start start 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 0x7f220c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 0x7f220c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f220c106f10 0x7f220c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f220c111e40 con 0x7f220c101de0 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f220c111cc0 con 0x7f220c106f10 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f220c111fc0 con 0x7f220c102790 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 0x7f220c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 0x7f220c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33154/0 (socket says 172.21.15.64:33154) 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 -- 172.21.15.64:0/3347758372 learned_addr learned my addr 172.21.15.64:0/3347758372 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212d73640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f220c106f10 0x7f220c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2211d71640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 0x7f220c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 -- 172.21.15.64:0/3347758372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 msgr2=0x7f220c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 0x7f220c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 -- 172.21.15.64:0/3347758372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f220c106f10 msgr2=0x7f220c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f220c106f10 0x7f220c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 -- 172.21.15.64:0/3347758372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c19f570 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2211d71640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 0x7f220c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f2212572640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 0x7f220c19a5c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f220800c890 tx=0x7f220800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22037fe640 1 -- 172.21.15.64:0/3347758372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208004220 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c19f860 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22037fe640 1 -- 172.21.15.64:0/3347758372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22080043c0 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.809+0000 7f22037fe640 1 -- 172.21.15.64:0/3347758372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208004a00 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.810+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a7140 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.812+0000 7f22037fe640 1 -- 172.21.15.64:0/3347758372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2208004ba0 con 0x7f220c101de0 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.812+0000 7f2216baf740 1 Processor -- start 2024-09-19T05:11:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.812+0000 7f2216baf740 1 -- start start 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2216baf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f008090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2216baf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f008a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2216baf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f015760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2216baf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c0ef6e2f0 con 0x563c0f009b40 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2216baf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c0ef15cf0 con 0x563c0f013370 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2216baf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c0ef6a2b0 con 0x563c0f0085d0 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f008090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f008090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33168/0 (socket says 172.21.15.64:33168) 2024-09-19T05:11:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 -- 172.21.15.64:0/1483313313 learned_addr learned my addr 172.21.15.64:0/1483313313 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212572640 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f008a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f22037fe640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21f80a33b0 0x7f21f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2211d71640 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f015760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 -- 172.21.15.64:0/1483313313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 msgr2=0x563c0f008a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f008a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 -- 172.21.15.64:0/1483313313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 msgr2=0x563c0f015760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f015760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 -- 172.21.15.64:0/1483313313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0eefa3a0 con 0x563c0f009b40 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2211d71640 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f015760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f22037fe640 1 -- 172.21.15.64:0/3347758372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f22080cfaf0 con 0x7f220c101de0 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.813+0000 7f2212d73640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21f80a33b0 0x7f21f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.814+0000 7f2212d73640 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f008090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2204005cc0 tx=0x7f2204031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff2ea85ca3e3569 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.814+0000 7f2200ff9640 1 -- 172.21.15.64:0/1483313313 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204031690 con 0x563c0f009b40 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.814+0000 7f2200ff9640 1 -- 172.21.15.64:0/1483313313 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2204036040 con 0x563c0f009b40 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.814+0000 7f2200ff9640 1 -- 172.21.15.64:0/1483313313 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204030800 con 0x563c0f009b40 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.814+0000 7f2212d73640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21f80a33b0 0x7f21f80a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f220404c000 tx=0x7f2204049000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 -- 172.21.15.64:0/1483313313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 msgr2=0x563c0f008090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f008090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2204005cc0 tx=0x7f2204031b10 comp rx=0 tx=0).stop 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 -- 172.21.15.64:0/1483313313 shutdown_connections 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f015760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f008a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 --2- 172.21.15.64:0/1483313313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f008090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 -- 172.21.15.64:0/1483313313 >> 172.21.15.64:0/1483313313 conn(0x563c0efd1310 msgr2=0x563c0f004c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 -- 172.21.15.64:0/1483313313 shutdown_connections 2024-09-19T05:11:15.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 -- 172.21.15.64:0/1483313313 wait complete. 2024-09-19T05:11:15.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.815+0000 7f2216baf740 1 -- start start 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 Processor -- start 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f012d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f010010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f010600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c0f015ef0 con 0x563c0f009b40 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c0ef29060 con 0x563c0f013370 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c0f016070 con 0x563c0f0085d0 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2216baf740 10 client.0 _ll_register_callbacks cb 0x563c0ec7c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f010010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2212572640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f012d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f010010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33176/0 (socket says 172.21.15.64:33176) 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 -- 172.21.15.64:0/827641412 learned_addr learned my addr 172.21.15.64:0/827641412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2212d73640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f010600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 -- 172.21.15.64:0/827641412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 msgr2=0x563c0f012d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f012d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 -- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 msgr2=0x563c0f010600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f010600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.816+0000 7f2211d71640 1 -- 172.21.15.64:0/827641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0ecdf3c0 con 0x563c0f009b40 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f2212572640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f012d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f2211d71640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f010010 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00ded0 tx=0x7f21fc00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1488c57d6c98a57e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f21f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f21f5ffb640 1 -- 172.21.15.64:0/827641412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc018070 con 0x563c0f009b40 2024-09-19T05:11:15.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f21f5ffb640 1 -- 172.21.15.64:0/827641412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f21fc014680 con 0x563c0f009b40 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2204036040 con 0x563c0f009b40 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f21f5ffb640 1 -- 172.21.15.64:0/827641412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f21fc024050 con 0x563c0f009b40 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f2216baf740 10 client.16641 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c0f0219f0 con 0x563c0f009b40 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.817+0000 7f21f5ffb640 1 -- 172.21.15.64:0/827641412 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2204036040 con 0x563c0f009b40 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f21f5ffb640 1 -- 172.21.15.64:0/827641412 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f21fc013ac0 con 0x563c0f009b40 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2216baf740 10 client.16641 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2216baf740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2216baf740 20 client.16641 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2216baf740 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f022e50 0x563c0f0252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2216baf740 4 client.16641 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c0ef7b860 con 0x563c0f022e50 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2212572640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f022e50 0x563c0f0252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f21f5ffb640 10 client.16641 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.818+0000 7f2212572640 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f022e50 0x563c0f0252b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b51f09eb40e409 server_cookie=fb9877cfbf672ea in_seq=0 out_seq=0 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f21f5ffb640 1 -- 172.21.15.64:0/827641412 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c0ef7b860 con 0x563c0f022e50 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f21f5ffb640 10 client.16641 handle_command_reply: tid=0 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 client.16641 shutdown 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 msgr2=0x563c0f010010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f010010 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00ded0 tx=0x7f21fc00f5b0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 shutdown_connections 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f010600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f022e50 0x563c0f0252b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f21f5ffb640 0 client.16641 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:15.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f010010 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 --2- 172.21.15.64:0/827641412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f012d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.819+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 >> 172.21.15.64:0/827641412 conn(0x563c0efd1310 msgr2=0x563c0f00a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:15.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.820+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 shutdown_connections 2024-09-19T05:11:15.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.820+0000 7f2216baf740 1 -- 172.21.15.64:0/827641412 wait complete. 2024-09-19T05:11:15.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.820+0000 7f2216baf740 20 client.16641 trim_cache size 0 max 16384 2024-09-19T05:11:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.832+0000 7f2216baf740 1 Processor -- start 2024-09-19T05:11:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 -- start start 2024-09-19T05:11:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f025240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f009b40 0x563c0f011f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 0x563c0f00b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1b2330 con 0x563c0f013370 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1afc00 con 0x563c0f009b40 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2216baf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1ada50 con 0x563c0f0085d0 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 0x563c0f00b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 0x563c0f00b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33182/0 (socket says 172.21.15.64:33182) 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 -- 172.21.15.64:0/258362113 learned_addr learned my addr 172.21.15.64:0/258362113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212572640 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f025240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2211d71640 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f009b40 0x563c0f011f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 -- 172.21.15.64:0/258362113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 msgr2=0x563c0f025240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f025240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 -- 172.21.15.64:0/258362113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f009b40 msgr2=0x563c0f011f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f009b40 0x563c0f011f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 -- 172.21.15.64:0/258362113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0f01fcb0 con 0x563c0f013370 2024-09-19T05:11:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2211d71640 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f009b40 0x563c0f011f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.833+0000 7f2212d73640 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 0x563c0f00b120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f220402ade0 tx=0x7f2204004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42eafa4195fd26ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f21f7fff640 1 -- 172.21.15.64:0/258362113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204031690 con 0x563c0f013370 2024-09-19T05:11:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f21f7fff640 1 -- 172.21.15.64:0/258362113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2204036030 con 0x563c0f013370 2024-09-19T05:11:15.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f21f7fff640 1 -- 172.21.15.64:0/258362113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204004260 con 0x563c0f013370 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 -- 172.21.15.64:0/258362113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 msgr2=0x563c0f00b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 0x563c0f00b120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f220402ade0 tx=0x7f2204004780 comp rx=0 tx=0).stop 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 -- 172.21.15.64:0/258362113 shutdown_connections 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f013370 0x563c0f00b120 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f009b40 0x563c0f011f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 --2- 172.21.15.64:0/258362113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f025240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 -- 172.21.15.64:0/258362113 >> 172.21.15.64:0/258362113 conn(0x563c0efd1310 msgr2=0x563c0f011870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 -- 172.21.15.64:0/258362113 shutdown_connections 2024-09-19T05:11:15.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.834+0000 7f2216baf740 1 -- 172.21.15.64:0/258362113 wait complete. 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 -- start start 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 Processor -- start 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f01bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f00cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f00d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1ae0c0 con 0x563c0f009b40 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1af530 con 0x563c0f013370 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1ae790 con 0x563c0f0085d0 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2216baf740 10 client.0 _ll_register_callbacks cb 0x7f21fc00d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f00cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f00cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33186/0 (socket says 172.21.15.64:33186) 2024-09-19T05:11:15.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 -- 172.21.15.64:0/1795095938 learned_addr learned my addr 172.21.15.64:0/1795095938 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2212d73640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f00d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2212572640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f01bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 -- 172.21.15.64:0/1795095938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 msgr2=0x563c0f01bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f01bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 -- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 msgr2=0x563c0f00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f00d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2211d71640 1 -- 172.21.15.64:0/1795095938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0f08cb50 con 0x563c0f009b40 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.835+0000 7f2212d73640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f00d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2211d71640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f00cee0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f21fc001eb0 tx=0x7f21fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307a9b2440d53b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f21f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f21f5ffb640 1 -- 172.21.15.64:0/1795095938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc02d060 con 0x563c0f009b40 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f21f5ffb640 1 -- 172.21.15.64:0/1795095938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f21fc01ca80 con 0x563c0f009b40 2024-09-19T05:11:15.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2204036030 con 0x563c0f009b40 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f21f5ffb640 1 -- 172.21.15.64:0/1795095938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f21fc018040 con 0x563c0f009b40 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 10 client.16647 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c0f08dc40 con 0x563c0f009b40 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f21f5ffb640 1 -- 172.21.15.64:0/1795095938 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2204036030 con 0x563c0f009b40 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f21f5ffb640 1 -- 172.21.15.64:0/1795095938 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f21fc014480 con 0x563c0f009b40 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 10 client.16647 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 20 client.16647 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f08e3c0 0x563c0f0907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.836+0000 7f2216baf740 4 client.16647 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c0ef75d70 con 0x563c0f08e3c0 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f2212572640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f08e3c0 0x563c0f0907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:15.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f21f5ffb640 10 client.16647 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f2212572640 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f08e3c0 0x563c0f0907e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f849f08d9b7e52c server_cookie=2977afc5bb8b8d6c in_seq=0 out_seq=0 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f21f5ffb640 1 -- 172.21.15.64:0/1795095938 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c0ef75d70 con 0x563c0f08e3c0 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f21f5ffb640 10 client.16647 handle_command_reply: tid=0 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.837+0000 7f2216baf740 1 client.16647 shutdown 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 msgr2=0x563c0f00cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f00cee0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f21fc001eb0 tx=0x7f21fc01c400 comp rx=0 tx=0).stop 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 shutdown_connections 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c0f013370 0x563c0f00d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563c0f08e3c0 0x563c0f0907e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f009b40 0x563c0f00cee0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 --2- 172.21.15.64:0/1795095938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c0f0085d0 0x563c0f01bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f21f5ffb640 0 client.16647 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:15.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 >> 172.21.15.64:0/1795095938 conn(0x563c0efd1310 msgr2=0x563c0f003d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:15.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 shutdown_connections 2024-09-19T05:11:15.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 1 -- 172.21.15.64:0/1795095938 wait complete. 2024-09-19T05:11:15.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f2216baf740 20 client.16647 trim_cache size 0 max 16384 2024-09-19T05:11:15.846 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21f80a33b0 msgr2=0x7f21f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21f80a33b0 0x7f21f80a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f220404c000 tx=0x7f2204049000 comp rx=0 tx=0).stop 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 msgr2=0x7f220c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 0x7f220c19a5c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f220800c890 tx=0x7f220800cd50 comp rx=0 tx=0).stop 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 shutdown_connections 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21f80a33b0 0x7f21f80a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f220c106f10 0x7f220c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f220c102790 0x7f220c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.838+0000 7f22147d6640 1 --2- 172.21.15.64:0/3347758372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f220c101de0 0x7f220c19a5c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.839+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 >> 172.21.15.64:0/3347758372 conn(0x7f220c0fd8c0 msgr2=0x7f220c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:15.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.839+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 shutdown_connections 2024-09-19T05:11:15.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:15.839+0000 7f22147d6640 1 -- 172.21.15.64:0/3347758372 wait complete. 2024-09-19T05:11:17.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:16 smithi064 ceph-mon[30960]: pgmap v164: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:17.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:16 smithi103 ceph-mon[34446]: pgmap v164: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:17.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:16 smithi174 ceph-mon[34488]: pgmap v164: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:11:17.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:11:17.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:11:17.832 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 -- 172.21.15.64:0/3018342993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c102570 msgr2=0x7f138c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 --2- 172.21.15.64:0/3018342993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c102570 0x7f138c1029b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1378009900 tx=0x7f137802a480 comp rx=0 tx=0).stop 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 -- 172.21.15.64:0/3018342993 shutdown_connections 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 --2- 172.21.15.64:0/3018342993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 0x7f138c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 --2- 172.21.15.64:0/3018342993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c102570 0x7f138c1029b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 --2- 172.21.15.64:0/3018342993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c100e20 0x7f138c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 -- 172.21.15.64:0/3018342993 >> 172.21.15.64:0/3018342993 conn(0x7f138c0fc7d0 msgr2=0x7f138c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 -- 172.21.15.64:0/3018342993 shutdown_connections 2024-09-19T05:11:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.309+0000 7f1392dc5640 1 -- 172.21.15.64:0/3018342993 wait complete. 2024-09-19T05:11:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 Processor -- start 2024-09-19T05:11:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 -- start start 2024-09-19T05:11:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 0x7f138c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c102570 0x7f138c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 0x7f138c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f138c110e30 con 0x7f138c100e20 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f138c110cb0 con 0x7f138c102570 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1392dc5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f138c110fb0 con 0x7f138c102ef0 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 0x7f138c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 0x7f138c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33210/0 (socket says 172.21.15.64:33210) 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 -- 172.21.15.64:0/1499941378 learned_addr learned my addr 172.21.15.64:0/1499941378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f138bfff640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c102570 0x7f138c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1391362640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 0x7f138c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 -- 172.21.15.64:0/1499941378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 msgr2=0x7f138c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 0x7f138c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 -- 172.21.15.64:0/1499941378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c102570 msgr2=0x7f138c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c102570 0x7f138c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1390b61640 1 -- 172.21.15.64:0/1499941378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c19f5e0 con 0x7f138c100e20 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f1391362640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 0x7f138c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.310+0000 7f138bfff640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c102570 0x7f138c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.311+0000 7f1390b61640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 0x7f138c19a630 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f138000d950 tx=0x7f138000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.311+0000 7f1389ffb640 1 -- 172.21.15.64:0/1499941378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380017070 con 0x7f138c100e20 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.311+0000 7f1389ffb640 1 -- 172.21.15.64:0/1499941378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1380009c30 con 0x7f138c100e20 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.311+0000 7f1389ffb640 1 -- 172.21.15.64:0/1499941378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000f620 con 0x7f138c100e20 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.311+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c19f8d0 con 0x7f138c100e20 2024-09-19T05:11:18.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.311+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c1a71b0 con 0x7f138c100e20 2024-09-19T05:11:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.313+0000 7f1389ffb640 1 -- 172.21.15.64:0/1499941378 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f13800109a0 con 0x7f138c100e20 2024-09-19T05:11:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.313+0000 7f139519e740 1 Processor -- start 2024-09-19T05:11:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 -- start start 2024-09-19T05:11:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55557351e280 con 0x5555735b8b60 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555734c5b10 con 0x5555735c3d40 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f139519e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55557351a240 con 0x5555735b9480 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1389ffb640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f137c0a3430 0x7f137c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33224/0 (socket says 172.21.15.64:33224) 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 -- 172.21.15.64:0/1401895522 learned_addr learned my addr 172.21.15.64:0/1401895522 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f137c0a3430 0x7f137c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1390b61640 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1389ffb640 1 -- 172.21.15.64:0/1499941378 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f13800cfa00 con 0x7f138c100e20 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f138bfff640 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 -- 172.21.15.64:0/1401895522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 msgr2=0x5555735c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 -- 172.21.15.64:0/1401895522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 msgr2=0x5555735c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f1391362640 1 -- 172.21.15.64:0/1401895522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555734d21f0 con 0x5555735b8b60 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.314+0000 7f138bfff640 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:18.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.315+0000 7f1391362640 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735b8f40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1384014e40 tx=0x7f1384011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c95bd48032918a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.315+0000 7f1391362640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f137c0a3430 0x7f137c0a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1384009fd0 tx=0x7f1384016000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.315+0000 7f1376ffd640 1 -- 172.21.15.64:0/1401895522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138400cc00 con 0x5555735b8b60 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.315+0000 7f1376ffd640 1 -- 172.21.15.64:0/1401895522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1384009c20 con 0x5555735b8b60 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.315+0000 7f1376ffd640 1 -- 172.21.15.64:0/1401895522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384005280 con 0x5555735b8b60 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 -- 172.21.15.64:0/1401895522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 msgr2=0x5555735b8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735b8f40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1384014e40 tx=0x7f1384011750 comp rx=0 tx=0).stop 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 -- 172.21.15.64:0/1401895522 shutdown_connections 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 --2- 172.21.15.64:0/1401895522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735b8f40 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 -- 172.21.15.64:0/1401895522 >> 172.21.15.64:0/1401895522 conn(0x5555735b50f0 msgr2=0x5555735b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 -- 172.21.15.64:0/1401895522 shutdown_connections 2024-09-19T05:11:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 -- 172.21.15.64:0/1401895522 wait complete. 2024-09-19T05:11:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 -- start start 2024-09-19T05:11:18.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.316+0000 7f139519e740 1 Processor -- start 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5555735c6910 con 0x5555735c3d40 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555734d8f10 con 0x5555735b9480 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555735c6a90 con 0x5555735b8b60 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f139519e740 10 client.0 _ll_register_callbacks cb 0x5555734a1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33240/0 (socket says 172.21.15.64:33240) 2024-09-19T05:11:18.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 -- 172.21.15.64:0/2208753660 learned_addr learned my addr 172.21.15.64:0/2208753660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1390b61640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f138bfff640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 -- 172.21.15.64:0/2208753660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 msgr2=0x5555735c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735c0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 -- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 msgr2=0x5555735c11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 -- 172.21.15.64:0/2208753660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557328f6a0 con 0x5555735c3d40 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1390b61640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f1391362640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735c1700 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1384009bb0 tx=0x7f138400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fd898500aa94d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.317+0000 7f136bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f136bfff640 1 -- 172.21.15.64:0/2208753660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384004bc0 con 0x5555735c3d40 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55557352b7f0 con 0x5555735c3d40 2024-09-19T05:11:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f136bfff640 1 -- 172.21.15.64:0/2208753660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f1384004d60 con 0x5555735c3d40 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f136bfff640 1 -- 172.21.15.64:0/2208753660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13840140c0 con 0x5555735c3d40 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 10 client.16659 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555735d25c0 con 0x5555735c3d40 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f136bfff640 1 -- 172.21.15.64:0/2208753660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55557352b7f0 con 0x5555735c3d40 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f136bfff640 1 -- 172.21.15.64:0/2208753660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f138402f070 con 0x5555735c3d40 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 10 client.16659 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 20 client.16659 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5555735d39d0 0x5555735d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 4 client.16659 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.318+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555573525d00 con 0x5555735d39d0 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.319+0000 7f1390b61640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5555735d39d0 0x5555735d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.319+0000 7f1390b61640 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5555735d39d0 0x5555735d5dc0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5470131bc5d19b server_cookie=6c82233df5a85fe in_seq=0 out_seq=0 2024-09-19T05:11:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.319+0000 7f136bfff640 10 client.16659 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f136bfff640 1 -- 172.21.15.64:0/2208753660 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555573525d00 con 0x5555735d39d0 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f136bfff640 10 client.16659 handle_command_reply: tid=0 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 client.16659 shutdown 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 msgr2=0x5555735c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735c1700 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1384009bb0 tx=0x7f138400ce30 comp rx=0 tx=0).stop 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 shutdown_connections 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5555735d39d0 0x5555735d5dc0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735c1700 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 --2- 172.21.15.64:0/2208753660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 >> 172.21.15.64:0/2208753660 conn(0x5555735b50f0 msgr2=0x5555735b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f136bfff640 0 client.16659 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 shutdown_connections 2024-09-19T05:11:18.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 1 -- 172.21.15.64:0/2208753660 wait complete. 2024-09-19T05:11:18.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.320+0000 7f139519e740 20 client.16659 trim_cache size 0 max 16384 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.337+0000 7f139519e740 1 Processor -- start 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.337+0000 7f139519e740 1 -- start start 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f139519e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735bb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f139519e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f139519e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735db530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f139519e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1b2320 con 0x5555735c3d40 2024-09-19T05:11:18.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f139519e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1afbf0 con 0x5555735b9480 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f139519e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1ada40 con 0x5555735b8b60 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735db530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735db530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33252/0 (socket says 172.21.15.64:33252) 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 -- 172.21.15.64:0/3936009038 learned_addr learned my addr 172.21.15.64:0/3936009038 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1390b61640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735bb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f138bfff640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 -- 172.21.15.64:0/3936009038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 msgr2=0x5555735bb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735bb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 -- 172.21.15.64:0/3936009038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 msgr2=0x5555735c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 -- 172.21.15.64:0/3936009038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555735b61a0 con 0x5555735c3d40 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1390b61640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:18.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f138bfff640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:18.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f1391362640 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735db530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1384009270 tx=0x7f1384005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b13d355f8e07649 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:18.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f13767fc640 1 -- 172.21.15.64:0/3936009038 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384031800 con 0x5555735c3d40 2024-09-19T05:11:18.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.338+0000 7f13767fc640 1 -- 172.21.15.64:0/3936009038 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13840149a0 con 0x5555735c3d40 2024-09-19T05:11:18.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f13767fc640 1 -- 172.21.15.64:0/3936009038 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840052a0 con 0x5555735c3d40 2024-09-19T05:11:18.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 -- 172.21.15.64:0/3936009038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 msgr2=0x5555735db530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735db530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1384009270 tx=0x7f1384005bd0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 -- 172.21.15.64:0/3936009038 shutdown_connections 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735c3d40 0x5555735db530 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735b9480 0x5555735c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 --2- 172.21.15.64:0/3936009038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b8b60 0x5555735bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 -- 172.21.15.64:0/3936009038 >> 172.21.15.64:0/3936009038 conn(0x5555735b50f0 msgr2=0x5555735c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 -- 172.21.15.64:0/3936009038 shutdown_connections 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 -- 172.21.15.64:0/3936009038 wait complete. 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.339+0000 7f139519e740 1 -- start start 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 Processor -- start 2024-09-19T05:11:18.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1ae0b0 con 0x5555735b8b60 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1af520 con 0x5555735c3d40 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f138c1ae780 con 0x5555735b9480 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f139519e740 10 client.0 _ll_register_callbacks cb 0x7f138000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33256/0 (socket says 172.21.15.64:33256) 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 -- 172.21.15.64:0/592083166 learned_addr learned my addr 172.21.15.64:0/592083166 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1391362640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f138bfff640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 -- 172.21.15.64:0/592083166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 msgr2=0x5555735bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735bda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 -- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 msgr2=0x5555735bdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735bdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.340+0000 7f1390b61640 1 -- 172.21.15.64:0/592083166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555735be820 con 0x5555735b8b60 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f1390b61640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735cc4e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f138009bc10 tx=0x7f1380086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a59a5205d6bb65a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f136bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f136bfff640 1 -- 172.21.15.64:0/592083166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380020040 con 0x5555735b8b60 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f136bfff640 1 -- 172.21.15.64:0/592083166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f13800982f0 con 0x5555735b8b60 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13840149a0 con 0x5555735b8b60 2024-09-19T05:11:18.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f136bfff640 1 -- 172.21.15.64:0/592083166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13800a48f0 con 0x5555735b8b60 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 10 client.16665 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55557363e640 con 0x5555735b8b60 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f136bfff640 1 -- 172.21.15.64:0/592083166 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13840149a0 con 0x5555735b8b60 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f136bfff640 1 -- 172.21.15.64:0/592083166 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f13800b4030 con 0x5555735b8b60 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 10 client.16665 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 20 client.16665 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55557363ecc0 0x5555736410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 4 client.16665 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:11:18.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.341+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555734a2110 con 0x55557363ecc0 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.342+0000 7f138bfff640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55557363ecc0 0x5555736410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.342+0000 7f138bfff640 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55557363ecc0 0x5555736410b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15784a94c327cb8 server_cookie=6eebc30134f19b4f in_seq=0 out_seq=0 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.342+0000 7f136bfff640 10 client.16665 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f136bfff640 1 -- 172.21.15.64:0/592083166 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555734a2110 con 0x55557363ecc0 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f136bfff640 10 client.16665 handle_command_reply: tid=0 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 client.16665 shutdown 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 msgr2=0x5555735cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735cc4e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f138009bc10 tx=0x7f1380086040 comp rx=0 tx=0).stop 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 shutdown_connections 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555735c3d40 0x5555735bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555735b9480 0x5555735bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55557363ecc0 0x5555736410b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 --2- 172.21.15.64:0/592083166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555735b8b60 0x5555735cc4e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f136bfff640 0 client.16665 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.343+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 >> 172.21.15.64:0/592083166 conn(0x5555735b50f0 msgr2=0x5555735c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 shutdown_connections 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f139519e740 1 -- 172.21.15.64:0/592083166 wait complete. 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f139519e740 20 client.16665 trim_cache size 0 max 16384 2024-09-19T05:11:18.351 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:18.352 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:11:18.352 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "4728a28e-096d-4df6-95df-c3d36a1bbcda", 2024-09-19T05:11:18.352 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:11:18.352 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f137c0a3430 msgr2=0x7f137c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f137c0a3430 0x7f137c0a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1384009fd0 tx=0x7f1384016000 comp rx=0 tx=0).stop 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 msgr2=0x7f138c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 0x7f138c19a630 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f138000d950 tx=0x7f138000de10 comp rx=0 tx=0).stop 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 shutdown_connections 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f137c0a3430 0x7f137c0a58f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c102ef0 0x7f138c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138c102570 0x7f138c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 --2- 172.21.15.64:0/1499941378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f138c100e20 0x7f138c19a630 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.344+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 >> 172.21.15.64:0/1499941378 conn(0x7f138c0fc7d0 msgr2=0x7f138c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.345+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 shutdown_connections 2024-09-19T05:11:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:18.345+0000 7f1392dc5640 1 -- 172.21.15.64:0/1499941378 wait complete. 2024-09-19T05:11:19.083 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:19.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:18 smithi064 ceph-mon[30960]: pgmap v165: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:11:19.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:18 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:11:19.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:18 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:19.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:18 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:11:19.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:18 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:19.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:18 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:11:19.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:18 smithi103 ceph-mon[34446]: pgmap v165: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:11:19.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:18 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:11:19.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:18 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:19.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:18 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:11:19.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:18 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:19.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:18 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:11:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:18 smithi174 ceph-mon[34488]: pgmap v165: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:11:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:18 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:11:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:18 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:18 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:11:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:18 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:19.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:18 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:11:19.342 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:19.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 -- 172.21.15.64:0/4147637085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 msgr2=0x7f27e0106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4147637085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e0106810 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009900 tx=0x7f27d002a410 comp rx=0 tx=0).stop 2024-09-19T05:11:19.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 -- 172.21.15.64:0/4147637085 shutdown_connections 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4147637085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 0x7f27e010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4147637085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e0106810 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4147637085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 0x7f27e01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 -- 172.21.15.64:0/4147637085 >> 172.21.15.64:0/4147637085 conn(0x7f27e00fd8d0 msgr2=0x7f27e00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 -- 172.21.15.64:0/4147637085 shutdown_connections 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 -- 172.21.15.64:0/4147637085 wait complete. 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.781+0000 7f27e7b41640 1 Processor -- start 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 -- start start 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 0x7f27e019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 0x7f27e019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111e70 con 0x7f27e01027c0 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111cf0 con 0x7f27e0106f40 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e7b41640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111ff0 con 0x7f27e0101e10 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33276/0 (socket says 172.21.15.64:33276) 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 -- 172.21.15.64:0/4159830720 learned_addr learned my addr 172.21.15.64:0/4159830720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e58dd640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 0x7f27e019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e60de640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 0x7f27e019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 -- 172.21.15.64:0/4159830720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 msgr2=0x7f27e019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 0x7f27e019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 -- 172.21.15.64:0/4159830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 msgr2=0x7f27e019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 0x7f27e019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e50dc640 1 -- 172.21.15.64:0/4159830720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e019f5e0 con 0x7f27e01027c0 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e58dd640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 0x7f27e019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.782+0000 7f27e60de640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 0x7f27e019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.783+0000 7f27e50dc640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e019ab70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006010 tx=0x7f27d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.783+0000 7f27d6ffd640 1 -- 172.21.15.64:0/4159830720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d003f030 con 0x7f27e01027c0 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.783+0000 7f27d6ffd640 1 -- 172.21.15.64:0/4159830720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27d00035f0 con 0x7f27e01027c0 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.783+0000 7f27d6ffd640 1 -- 172.21.15.64:0/4159830720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0035070 con 0x7f27e01027c0 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.783+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e019f870 con 0x7f27e01027c0 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.783+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a71b0 con 0x7f27e01027c0 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.785+0000 7f27e9f1a740 1 Processor -- start 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.785+0000 7f27e9f1a740 1 -- start start 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.785+0000 7f27d6ffd640 1 -- 172.21.15.64:0/4159830720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f27d0003b10 con 0x7f27e01027c0 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a4c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4cdc0 0x563b23a57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a59a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563b239b22e0 con 0x563b23a57680 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b23959ce0 con 0x563b23a4c4a0 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b239ae2a0 con 0x563b23a4cdc0 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a59a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a59a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33288/0 (socket says 172.21.15.64:33288) 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 -- 172.21.15.64:0/3373398217 learned_addr learned my addr 172.21.15.64:0/3373398217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e60de640 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a4c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e58dd640 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4cdc0 0x563b23a57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 -- 172.21.15.64:0/3373398217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4cdc0 msgr2=0x563b23a57140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4cdc0 0x563b23a57140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 -- 172.21.15.64:0/3373398217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 msgr2=0x563b23a4c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a4c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 -- 172.21.15.64:0/3373398217 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2393e440 con 0x563b23a57680 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27d6ffd640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27cc0a33b0 0x7f27cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e50dc640 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a59a70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f27d00ee010 tx=0x7f27d00f6c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3cc5cb0ec9606d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.786+0000 7f27e60de640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27cc0a33b0 0x7f27cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27d6ffd640 1 -- 172.21.15.64:0/4159830720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27d00f0950 con 0x7f27e01027c0 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27cbfff640 1 -- 172.21.15.64:0/3373398217 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d00b8ed0 con 0x563b23a57680 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27cbfff640 1 -- 172.21.15.64:0/3373398217 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27d00bd330 con 0x563b23a57680 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27cbfff640 1 -- 172.21.15.64:0/3373398217 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d00bc590 con 0x563b23a57680 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27e9f1a740 1 -- 172.21.15.64:0/3373398217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 msgr2=0x563b23a59a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27e60de640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27cc0a33b0 0x7f27cc0a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f27d800bad0 tx=0x7f27d8008040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a59a70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f27d00ee010 tx=0x7f27d00f6c80 comp rx=0 tx=0).stop 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.787+0000 7f27e9f1a740 1 -- 172.21.15.64:0/3373398217 shutdown_connections 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a59a70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4cdc0 0x563b23a57140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/3373398217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a4c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 -- 172.21.15.64:0/3373398217 >> 172.21.15.64:0/3373398217 conn(0x563b23a48a30 msgr2=0x563b23a48e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 -- 172.21.15.64:0/3373398217 shutdown_connections 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 -- 172.21.15.64:0/3373398217 wait complete. 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 -- start start 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 Processor -- start 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a56fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a54250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.788+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563b23a5a010 con 0x563b23a57680 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b2396d050 con 0x563b23a4cdc0 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b23a5a190 con 0x563b23a4c4a0 2024-09-19T05:11:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e9f1a740 10 client.0 _ll_register_callbacks cb 0x563b239ad160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a54790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33290/0 (socket says 172.21.15.64:33290) 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 -- 172.21.15.64:0/1296206179 learned_addr learned my addr 172.21.15.64:0/1296206179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e50dc640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a54250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a56fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 -- 172.21.15.64:0/1296206179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 msgr2=0x563b23a56fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a56fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 -- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 msgr2=0x563b23a54250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a54250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 -- 172.21.15.64:0/1296206179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b237233c0 con 0x563b23a57680 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a56fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e60de640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a54790 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f27d80167d0 tx=0x7f27d80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4347c7cb2dfded4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27c97fa640 1 -- 172.21.15.64:0/1296206179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d801c050 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d00bd330 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27c97fa640 1 -- 172.21.15.64:0/1296206179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27d80149b0 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27c97fa640 1 -- 172.21.15.64:0/1296206179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27d8014bb0 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.789+0000 7f27e9f1a740 10 client.16677 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b23a659b0 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27c97fa640 1 -- 172.21.15.64:0/1296206179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27d00bd330 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27c97fa640 1 -- 172.21.15.64:0/1296206179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27d8006550 con 0x563b23a57680 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 10 client.16677 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 20 client.16677 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a66dc0 0x563b23a69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 4 client.16677 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b239bf850 con 0x563b23a66dc0 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.790+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a66dc0 0x563b23a69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.791+0000 7f27c97fa640 10 client.16677 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.791+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a66dc0 0x563b23a69240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5f30323d2e8960 server_cookie=b82f250edd9a2d74 in_seq=0 out_seq=0 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.791+0000 7f27c97fa640 1 -- 172.21.15.64:0/1296206179 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b239bf850 con 0x563b23a66dc0 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.791+0000 7f27c97fa640 10 client.16677 handle_command_reply: tid=0 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.791+0000 7f27e9f1a740 1 client.16677 shutdown 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 msgr2=0x563b23a54790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a54790 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f27d80167d0 tx=0x7f27d80047d0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 shutdown_connections 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a66dc0 0x563b23a69240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a57680 0x563b23a54790 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a54250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1296206179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a56fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 >> 172.21.15.64:0/1296206179 conn(0x563b23a48a30 msgr2=0x563b23a4e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27c97fa640 0 client.16677 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 shutdown_connections 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1296206179 wait complete. 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.792+0000 7f27e9f1a740 20 client.16677 trim_cache size 0 max 16384 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 Processor -- start 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 -- start start 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 0x563b23a631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a4f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01b2330 con 0x563b23a48a30 2024-09-19T05:11:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01afc00 con 0x563b23a4cdc0 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01ada50 con 0x563b23a4c4a0 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 0x563b23a631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 0x563b23a631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33294/0 (socket says 172.21.15.64:33294) 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 -- 172.21.15.64:0/7037881 learned_addr learned my addr 172.21.15.64:0/7037881 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e50dc640 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a4f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 -- 172.21.15.64:0/7037881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 msgr2=0x563b23a4f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a4f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 -- 172.21.15.64:0/7037881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 msgr2=0x563b23a6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.809+0000 7f27e58dd640 1 -- 172.21.15.64:0/7037881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b23a49bc0 con 0x563b23a48a30 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e58dd640 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 0x563b23a631e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007f20 tx=0x7f27dc0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a46acd842e11cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27cb7fe640 1 -- 172.21.15.64:0/7037881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc01b3f0 con 0x563b23a48a30 2024-09-19T05:11:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27cb7fe640 1 -- 172.21.15.64:0/7037881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27dc01ba40 con 0x563b23a48a30 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27cb7fe640 1 -- 172.21.15.64:0/7037881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011bf0 con 0x563b23a48a30 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 -- 172.21.15.64:0/7037881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 msgr2=0x563b23a631e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 0x563b23a631e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007f20 tx=0x7f27dc0075f0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 -- 172.21.15.64:0/7037881 shutdown_connections 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4cdc0 0x563b23a6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a4c4a0 0x563b23a4f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/7037881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a48a30 0x563b23a631e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.810+0000 7f27e9f1a740 1 -- 172.21.15.64:0/7037881 >> 172.21.15.64:0/7037881 conn(0x563b23a548f0 msgr2=0x563b23a4f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 -- 172.21.15.64:0/7037881 shutdown_connections 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 -- 172.21.15.64:0/7037881 wait complete. 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 -- start start 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 Processor -- start 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a48a30 0x563b23a5f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a5aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 0x563b23a5b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01ae0c0 con 0x563b23a4cdc0 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01af530 con 0x563b23a4c4a0 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01ae790 con 0x563b23a48a30 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e9f1a740 10 client.0 _ll_register_callbacks cb 0x7f27d800b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e60de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 0x563b23a5b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e60de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 0x563b23a5b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33306/0 (socket says 172.21.15.64:33306) 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e60de640 1 -- 172.21.15.64:0/1781047043 learned_addr learned my addr 172.21.15.64:0/1781047043 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.811+0000 7f27e50dc640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a5aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a48a30 0x563b23a5f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e60de640 1 -- 172.21.15.64:0/1781047043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a48a30 msgr2=0x563b23a5f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e60de640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a48a30 0x563b23a5f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e60de640 1 -- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 msgr2=0x563b23a5aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e60de640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a5aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e60de640 1 -- 172.21.15.64:0/1781047043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b23a5b570 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e60de640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 0x563b23a5b030 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000c00 tx=0x7f27d8014880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de7b35355bf499e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27c97fa640 1 -- 172.21.15.64:0/1781047043 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8037030 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27c97fa640 1 -- 172.21.15.64:0/1781047043 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27d800f040 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27dc01ba40 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27c97fa640 1 -- 172.21.15.64:0/1781047043 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27d801c070 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e9f1a740 10 client.16683 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b23ad1850 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.812+0000 7f27c97fa640 1 -- 172.21.15.64:0/1781047043 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27dc01ba40 con 0x563b23a4cdc0 2024-09-19T05:11:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27c97fa640 1 -- 172.21.15.64:0/1781047043 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27d8033a20 con 0x563b23a4cdc0 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e9f1a740 10 client.16683 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e9f1a740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e9f1a740 20 client.16683 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a57680 0x563b23ad3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e9f1a740 4 client.16683 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b239b9d60 con 0x563b23a57680 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a57680 0x563b23ad3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27c97fa640 10 client.16683 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.813+0000 7f27e58dd640 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a57680 0x563b23ad3f70 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853eb18c98fc8c55 server_cookie=789fd9052a445011 in_seq=0 out_seq=0 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27c97fa640 1 -- 172.21.15.64:0/1781047043 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b239b9d60 con 0x563b23a57680 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27c97fa640 10 client.16683 handle_command_reply: tid=0 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 client.16683 shutdown 2024-09-19T05:11:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 msgr2=0x563b23a5b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 0x563b23a5b030 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000c00 tx=0x7f27d8014880 comp rx=0 tx=0).stop 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 shutdown_connections 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563b23a57680 0x563b23ad3f70 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27c97fa640 0 client.16683 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b23a4cdc0 0x563b23a5b030 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b23a4c4a0 0x563b23a5aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 --2- 172.21.15.64:0/1781047043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23a48a30 0x563b23a5f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 >> 172.21.15.64:0/1781047043 conn(0x563b23a548f0 msgr2=0x563b23a4f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 shutdown_connections 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 1 -- 172.21.15.64:0/1781047043 wait complete. 2024-09-19T05:11:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.814+0000 7f27e9f1a740 20 client.16683 trim_cache size 0 max 16384 2024-09-19T05:11:19.835 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:19.835 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:11:19.835 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:11:19.835 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27cc0a33b0 msgr2=0x7f27cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27cc0a33b0 0x7f27cc0a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f27d800bad0 tx=0x7f27d8008040 comp rx=0 tx=0).stop 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 msgr2=0x7f27e019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e019ab70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006010 tx=0x7f27d0004270 comp rx=0 tx=0).stop 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 shutdown_connections 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27cc0a33b0 0x7f27cc0a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27e0106f40 0x7f27e019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27e01027c0 0x7f27e019ab70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 --2- 172.21.15.64:0/4159830720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0101e10 0x7f27e019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 >> 172.21.15.64:0/4159830720 conn(0x7f27e00fd8d0 msgr2=0x7f27e00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 shutdown_connections 2024-09-19T05:11:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:19.815+0000 7f27e7b41640 1 -- 172.21.15.64:0/4159830720 wait complete. 2024-09-19T05:11:20.507 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:11:20.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:11:20.823 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:21.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:20 smithi064 ceph-mon[30960]: pgmap v166: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:11:21.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:20 smithi103 ceph-mon[34446]: pgmap v166: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:11:21.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:20 smithi174 ceph-mon[34488]: pgmap v166: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/884888269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34100a90 msgr2=0x7f0d34100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/884888269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34100a90 0x7f0d34100ef0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009930 tx=0x7f0d2802a420 comp rx=0 tx=0).stop 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/884888269 shutdown_connections 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/884888269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d34101430 0x7f0d3410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/884888269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34100a90 0x7f0d34100ef0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/884888269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 0x7f0d34106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/884888269 >> 172.21.15.64:0/884888269 conn(0x7f0d340fc7d0 msgr2=0x7f0d340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:21.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/884888269 shutdown_connections 2024-09-19T05:11:21.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.312+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/884888269 wait complete. 2024-09-19T05:11:21.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 Processor -- start 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 -- start start 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d34100a90 0x7f0d34198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 0x7f0d34198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 0x7f0d3419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34111e10 con 0x7f0d34101430 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34111c90 con 0x7f0d34100a90 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3bf4b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34111f90 con 0x7f0d34106a90 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 0x7f0d34198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 0x7f0d34198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33320/0 (socket says 172.21.15.64:33320) 2024-09-19T05:11:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 -- 172.21.15.64:0/2824183617 learned_addr learned my addr 172.21.15.64:0/2824183617 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d39ce7640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d34100a90 0x7f0d34198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d3a4e8640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 0x7f0d3419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 -- 172.21.15.64:0/2824183617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 msgr2=0x7f0d3419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 0x7f0d3419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 -- 172.21.15.64:0/2824183617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d34100a90 msgr2=0x7f0d34198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d34100a90 0x7f0d34198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.313+0000 7f0d394e6640 1 -- 172.21.15.64:0/2824183617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3419d2e0 con 0x7f0d34101430 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d3a4e8640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 0x7f0d3419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d394e6640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 0x7f0d34198870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0d28004770 tx=0x7f0d280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2803f070 con 0x7f0d34101430 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d2803b460 con 0x7f0d34101430 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3419d570 con 0x7f0d34101430 2024-09-19T05:11:21.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28035070 con 0x7f0d34101430 2024-09-19T05:11:21.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.314+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d341a4eb0 con 0x7f0d34101430 2024-09-19T05:11:21.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.315+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d34100ef0 con 0x7f0d34101430 2024-09-19T05:11:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.316+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0d2803b610 con 0x7f0d34101430 2024-09-19T05:11:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.317+0000 7f0d26ffd640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0d200a3430 0x7f0d200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.317+0000 7f0d39ce7640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0d200a3430 0x7f0d200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.317+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0d280f02c0 con 0x7f0d34101430 2024-09-19T05:11:21.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.318+0000 7f0d39ce7640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0d200a3430 0x7f0d200a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000ab60 tx=0x7f0d3000a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:21.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.320+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d280bcca0 con 0x7f0d34101430 2024-09-19T05:11:21.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.534+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d34199690 con 0x7f0d34101430 2024-09-19T05:11:21.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.534+0000 7f0d26ffd640 1 -- 172.21.15.64:0/2824183617 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f0d34199690 con 0x7f0d34101430 2024-09-19T05:11:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:11:21.538 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:11:21.538 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0d200a3430 msgr2=0x7f0d200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:21.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0d200a3430 0x7f0d200a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000ab60 tx=0x7f0d3000a2d0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 msgr2=0x7f0d34198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 0x7f0d34198870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0d28004770 tx=0x7f0d280042f0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 shutdown_connections 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0d200a3430 0x7f0d200a58f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d34106a90 0x7f0d3419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34101430 0x7f0d34198870 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 --2- 172.21.15.64:0/2824183617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d34100a90 0x7f0d34198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 >> 172.21.15.64:0/2824183617 conn(0x7f0d340fc7d0 msgr2=0x7f0d340fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 shutdown_connections 2024-09-19T05:11:21.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:21.537+0000 7f0d3bf4b640 1 -- 172.21.15.64:0/2824183617 wait complete. 2024-09-19T05:11:22.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:21 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2824183617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:22.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:11:22.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:21 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2824183617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:22.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:21 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2824183617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:22.452 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 -- 172.21.15.64:0/1525531121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 msgr2=0x7f50e0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 --2- 172.21.15.64:0/1525531121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0107b90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009930 tx=0x7f50d402a420 comp rx=0 tx=0).stop 2024-09-19T05:11:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 -- 172.21.15.64:0/1525531121 shutdown_connections 2024-09-19T05:11:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 --2- 172.21.15.64:0/1525531121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50e01082a0 0x7f50e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 --2- 172.21.15.64:0/1525531121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0107b90 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 --2- 172.21.15.64:0/1525531121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e0102d30 0x7f50e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 -- 172.21.15.64:0/1525531121 >> 172.21.15.64:0/1525531121 conn(0x7f50e00fc7d0 msgr2=0x7f50e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 -- 172.21.15.64:0/1525531121 shutdown_connections 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.933+0000 7f50e7f69640 1 -- 172.21.15.64:0/1525531121 wait complete. 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 Processor -- start 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 -- start start 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50e0102d30 0x7f50e01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e01082a0 0x7f50e019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0111f50 con 0x7f50e01036e0 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0111dd0 con 0x7f50e01082a0 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e7f69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01120d0 con 0x7f50e0102d30 2024-09-19T05:11:22.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33336/0 (socket says 172.21.15.64:33336) 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e5504640 1 -- 172.21.15.64:0/3057433912 learned_addr learned my addr 172.21.15.64:0/3057433912 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e5d05640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50e0102d30 0x7f50e01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.934+0000 7f50e6506640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e01082a0 0x7f50e019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e5504640 1 -- 172.21.15.64:0/3057433912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50e0102d30 msgr2=0x7f50e01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e5504640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50e0102d30 0x7f50e01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e5504640 1 -- 172.21.15.64:0/3057433912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e01082a0 msgr2=0x7f50e019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e5504640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e01082a0 0x7f50e019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e5504640 1 -- 172.21.15.64:0/3057433912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e019d4a0 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e6506640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e01082a0 0x7f50e019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e5504640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0198a30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f50d4031440 tx=0x7f50d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50deffd640 1 -- 172.21.15.64:0/3057433912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d403f030 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50deffd640 1 -- 172.21.15.64:0/3057433912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d4003aa0 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e019d730 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50deffd640 1 -- 172.21.15.64:0/3057433912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4035070 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.935+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e01a5070 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.937+0000 7f50deffd640 1 -- 172.21.15.64:0/3057433912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f50d4042030 con 0x7f50e01036e0 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 Processor -- start 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 -- start start 2024-09-19T05:11:22.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50deffd640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50c80a33b0 0x7f50c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x55558421e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x55558421ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x55558422b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5555841842f0 con 0x55558421e5d0 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55558412bcf0 con 0x55558421fb40 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50ea342740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555841802b0 con 0x555584229370 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50e6506640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50c80a33b0 0x7f50c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x55558421ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x55558421ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33344/0 (socket says 172.21.15.64:33344) 2024-09-19T05:11:22.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50e5504640 1 -- 172.21.15.64:0/3797839650 learned_addr learned my addr 172.21.15.64:0/3797839650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50e5d05640 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x55558421e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.938+0000 7f50deffd640 1 -- 172.21.15.64:0/3057433912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f50d40f04e0 con 0x7f50e01036e0 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e6506640 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x55558422b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e5504640 1 -- 172.21.15.64:0/3797839650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 msgr2=0x55558422b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e5504640 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x55558422b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e5504640 1 -- 172.21.15.64:0/3797839650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 msgr2=0x55558421e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e5504640 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x55558421e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e5504640 1 -- 172.21.15.64:0/3797839650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555841103a0 con 0x55558421e5d0 2024-09-19T05:11:22.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e6506640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50c80a33b0 0x7f50c80a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005fd0 tx=0x7f50d0009de0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50e5504640 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x55558421ea10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f50d4039fd0 tx=0x7f50d4049dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dcfa2fedea416d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50c7fff640 1 -- 172.21.15.64:0/3797839650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4049070 con 0x55558421e5d0 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50c7fff640 1 -- 172.21.15.64:0/3797839650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d40bc760 con 0x55558421e5d0 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.939+0000 7f50c7fff640 1 -- 172.21.15.64:0/3797839650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40baa60 con 0x55558421e5d0 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 -- 172.21.15.64:0/3797839650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 msgr2=0x55558421ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x55558421ea10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f50d4039fd0 tx=0x7f50d4049dd0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 -- 172.21.15.64:0/3797839650 shutdown_connections 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x55558422b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x55558421ea10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 --2- 172.21.15.64:0/3797839650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x55558421e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 -- 172.21.15.64:0/3797839650 >> 172.21.15.64:0/3797839650 conn(0x5555841e7310 msgr2=0x55558421ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 -- 172.21.15.64:0/3797839650 shutdown_connections 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 -- 172.21.15.64:0/3797839650 wait complete. 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.940+0000 7f50ea342740 1 -- start start 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 Processor -- start 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x555584228d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x555584225fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x5555842265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55558422be30 con 0x55558421e5d0 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55558413f060 con 0x55558421fb40 2024-09-19T05:11:22.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55558422bfb0 con 0x555584229370 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x555584228d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x555584228d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33350/0 (socket says 172.21.15.64:33350) 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 -- 172.21.15.64:0/2922512513 learned_addr learned my addr 172.21.15.64:0/2922512513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50ea342740 10 client.0 _ll_register_callbacks cb 0x555583e928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5504640 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x555584225fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 -- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 msgr2=0x5555842265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x5555842265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 -- 172.21.15.64:0/2922512513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 msgr2=0x555584225fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x555584225fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.941+0000 7f50e5d05640 1 -- 172.21.15.64:0/2922512513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555583ef53c0 con 0x55558421e5d0 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50e5d05640 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x555584228d30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f50d8010a50 tx=0x7f50d8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aab2c130d210f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:22.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50c57fa640 1 -- 172.21.15.64:0/2922512513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d801e060 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50c57fa640 1 -- 172.21.15.64:0/2922512513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f50d800fc30 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d40bc760 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50c57fa640 1 -- 172.21.15.64:0/2922512513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50d800fe30 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 10 client.16701 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555842377c0 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50c57fa640 1 -- 172.21.15.64:0/2922512513 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d40bc760 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50c57fa640 1 -- 172.21.15.64:0/2922512513 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50d80214d0 con 0x55558421e5d0 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 10 client.16701 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 10 client.16701 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 20 client.16701 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x555584238c80 0x55558423b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 4 client.16701 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.942+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555584191860 con 0x555584238c80 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.943+0000 7f50e5504640 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x555584238c80 0x55558423b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.943+0000 7f50e5504640 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x555584238c80 0x55558423b110 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8d619135cbbfb1b server_cookie=3847c6da7f8207b0 in_seq=0 out_seq=0 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.943+0000 7f50c57fa640 10 client.16701 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50c57fa640 1 -- 172.21.15.64:0/2922512513 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555584191860 con 0x555584238c80 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50c57fa640 10 client.16701 handle_command_reply: tid=0 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 client.16701 shutdown 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 msgr2=0x555584228d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x555584228d30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f50d8010a50 tx=0x7f50d8010f10 comp rx=0 tx=0).stop 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 shutdown_connections 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x555584238c80 0x55558423b110 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x5555842265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421fb40 0x555584225fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50c57fa640 0 client.16701 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:22.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 --2- 172.21.15.64:0/2922512513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421e5d0 0x555584228d30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 >> 172.21.15.64:0/2922512513 conn(0x5555841e7310 msgr2=0x555584220990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:22.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 shutdown_connections 2024-09-19T05:11:22.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.944+0000 7f50ea342740 1 -- 172.21.15.64:0/2922512513 wait complete. 2024-09-19T05:11:22.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.945+0000 7f50ea342740 20 client.16701 trim_cache size 0 max 16384 2024-09-19T05:11:22.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 Processor -- start 2024-09-19T05:11:22.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 -- start start 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421e5d0 0x5555842258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x555584221120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x555584221660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01adf00 con 0x55558421fb40 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01ab7d0 con 0x55558421e5d0 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50ea342740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01a9640 con 0x555584229370 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x555584221120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x555584221120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33364/0 (socket says 172.21.15.64:33364) 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 -- 172.21.15.64:0/1487610648 learned_addr learned my addr 172.21.15.64:0/1487610648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5d05640 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421e5d0 0x5555842258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e6506640 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x555584221660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 -- 172.21.15.64:0/1487610648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 msgr2=0x555584221660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x555584221660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 -- 172.21.15.64:0/1487610648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421e5d0 msgr2=0x5555842258a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421e5d0 0x5555842258a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.958+0000 7f50e5504640 1 -- 172.21.15.64:0/1487610648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555584224f90 con 0x55558421fb40 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50e5504640 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x555584221120 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50d4032fd0 tx=0x7f50d40b1bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bc41dfa021b428 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50c77fe640 1 -- 172.21.15.64:0/1487610648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4049040 con 0x55558421fb40 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50c77fe640 1 -- 172.21.15.64:0/1487610648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d4049e40 con 0x55558421fb40 2024-09-19T05:11:22.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50c77fe640 1 -- 172.21.15.64:0/1487610648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40b8dd0 con 0x55558421fb40 2024-09-19T05:11:22.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 -- 172.21.15.64:0/1487610648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 msgr2=0x555584221120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x555584221120 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50d4032fd0 tx=0x7f50d40b1bf0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 -- 172.21.15.64:0/1487610648 shutdown_connections 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555584229370 0x555584221660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x555584221120 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 --2- 172.21.15.64:0/1487610648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55558421e5d0 0x5555842258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.959+0000 7f50ea342740 1 -- 172.21.15.64:0/1487610648 >> 172.21.15.64:0/1487610648 conn(0x5555841e7310 msgr2=0x555584219d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 -- 172.21.15.64:0/1487610648 shutdown_connections 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 -- 172.21.15.64:0/1487610648 wait complete. 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 -- start start 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 Processor -- start 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558421e5d0 0x55558422d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x55558422d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555584229370 0x5555842a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01a9c90 con 0x55558421fb40 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01ab100 con 0x555584229370 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.960+0000 7f50ea342740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01aa360 con 0x55558421e5d0 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50ea342740 10 client.0 _ll_register_callbacks cb 0x7f50d800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x55558422d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x55558422d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33376/0 (socket says 172.21.15.64:33376) 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 -- 172.21.15.64:0/1012735080 learned_addr learned my addr 172.21.15.64:0/1012735080 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5d05640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558421e5d0 0x55558422d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e6506640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555584229370 0x5555842a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 -- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558421e5d0 msgr2=0x55558422d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558421e5d0 0x55558422d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 -- 172.21.15.64:0/1012735080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555584229370 msgr2=0x5555842a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555584229370 0x5555842a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 -- 172.21.15.64:0/1012735080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555842a2ef0 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50e5504640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x55558422d830 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f50d40bc460 tx=0x7f50d40c5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7134073c58ec683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50c57fa640 1 -- 172.21.15.64:0/1012735080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4049050 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50c57fa640 1 -- 172.21.15.64:0/1012735080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f50d403bd00 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d4049e40 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50c57fa640 1 -- 172.21.15.64:0/1012735080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50d40da050 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50ea342740 10 client.16707 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.961+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555842a40b0 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50c57fa640 1 -- 172.21.15.64:0/1012735080 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d4049e40 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50c57fa640 1 -- 172.21.15.64:0/1012735080 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50d40ede50 con 0x55558421fb40 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50ea342740 10 client.16707 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:22.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50ea342740 10 client.16707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50ea342740 20 client.16707 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50ea342740 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5555842a4440 0x5555842a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50ea342740 4 client.16707 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55558418bd70 con 0x5555842a4440 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.962+0000 7f50e5d05640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5555842a4440 0x5555842a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50c57fa640 10 client.16707 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50e5d05640 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5555842a4440 0x5555842a6860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf5d2ca97ecc76fa server_cookie=1f9961c49f5e7403 in_seq=0 out_seq=0 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50c57fa640 1 -- 172.21.15.64:0/1012735080 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55558418bd70 con 0x5555842a4440 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50c57fa640 10 client.16707 handle_command_reply: tid=0 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 client.16707 shutdown 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 msgr2=0x55558422d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x55558422d830 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f50d40bc460 tx=0x7f50d40c5550 comp rx=0 tx=0).stop 2024-09-19T05:11:22.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 shutdown_connections 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5555842a4440 0x5555842a6860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555584229370 0x5555842a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50c57fa640 0 client.16707 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55558421fb40 0x55558422d830 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 --2- 172.21.15.64:0/1012735080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55558421e5d0 0x55558422d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 >> 172.21.15.64:0/1012735080 conn(0x5555841e7310 msgr2=0x555584227840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 shutdown_connections 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 1 -- 172.21.15.64:0/1012735080 wait complete. 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.963+0000 7f50ea342740 20 client.16707 trim_cache size 0 max 16384 2024-09-19T05:11:22.971 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:22.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50c80a33b0 msgr2=0x7f50c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50c80a33b0 0x7f50c80a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005fd0 tx=0x7f50d0009de0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 msgr2=0x7f50e0198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:22.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0198a30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f50d4031440 tx=0x7f50d4004270 comp rx=0 tx=0).stop 2024-09-19T05:11:22.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 shutdown_connections 2024-09-19T05:11:22.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f50c80a33b0 0x7f50c80a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e01082a0 0x7f50e019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50e01036e0 0x7f50e0198a30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 --2- 172.21.15.64:0/3057433912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50e0102d30 0x7f50e01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:22.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 >> 172.21.15.64:0/3057433912 conn(0x7f50e00fc7d0 msgr2=0x7f50e0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:22.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 shutdown_connections 2024-09-19T05:11:22.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:22.964+0000 7f50e7f69640 1 -- 172.21.15.64:0/3057433912 wait complete. 2024-09-19T05:11:23.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:22 smithi064 ceph-mon[30960]: pgmap v167: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:23.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:22 smithi103 ceph-mon[34446]: pgmap v167: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:23.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:22 smithi174 ceph-mon[34488]: pgmap v167: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:23.773 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:11:24.034 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 -- 172.21.15.64:0/797372022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 msgr2=0x7fe070103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 --2- 172.21.15.64:0/797372022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe070103120 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe064009900 tx=0x7fe06402a480 comp rx=0 tx=0).stop 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 -- 172.21.15.64:0/797372022 shutdown_connections 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 --2- 172.21.15.64:0/797372022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe07010c170 0x7fe07010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 --2- 172.21.15.64:0/797372022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe070103660 0x7fe07010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 --2- 172.21.15.64:0/797372022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe070103120 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.480+0000 7fe076f95640 1 -- 172.21.15.64:0/797372022 >> 172.21.15.64:0/797372022 conn(0x7fe0700fc7a0 msgr2=0x7fe0700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 -- 172.21.15.64:0/797372022 shutdown_connections 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 -- 172.21.15.64:0/797372022 wait complete. 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 Processor -- start 2024-09-19T05:11:24.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 -- start start 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe07019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe070103660 0x7fe07019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe07010c170 0x7fe07019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe070111e20 con 0x7fe070102d40 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe070111ca0 con 0x7fe07010c170 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.481+0000 7fe076f95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe070111fa0 con 0x7fe070103660 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe07019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe07019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52568/0 (socket says 172.21.15.64:52568) 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 -- 172.21.15.64:0/611415722 learned_addr learned my addr 172.21.15.64:0/611415722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe06ffff640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe070103660 0x7fe07019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe075532640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe07010c170 0x7fe07019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 -- 172.21.15.64:0/611415722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe070103660 msgr2=0x7fe07019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe070103660 0x7fe07019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 -- 172.21.15.64:0/611415722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe07010c170 msgr2=0x7fe07019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe07010c170 0x7fe07019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 -- 172.21.15.64:0/611415722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07019f4a0 con 0x7fe070102d40 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe06ffff640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe070103660 0x7fe07019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe074d31640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe07019a4f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe064005940 tx=0x7fe064004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe06dffb640 1 -- 172.21.15.64:0/611415722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06403f030 con 0x7fe070102d40 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe06dffb640 1 -- 172.21.15.64:0/611415722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe064003aa0 con 0x7fe070102d40 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe06dffb640 1 -- 172.21.15.64:0/611415722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064035070 con 0x7fe070102d40 2024-09-19T05:11:24.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.482+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07019f730 con 0x7fe070102d40 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.483+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0701a7070 con 0x7fe070102d40 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe06dffb640 1 -- 172.21.15.64:0/611415722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe064042030 con 0x7fe070102d40 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 Processor -- start 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 -- start start 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 0x55775300f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x55775300fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x55775301c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557752f752f0 con 0x55775300f5d0 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557752f1ccf0 con 0x557753010b40 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe07936e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557752f712b0 con 0x55775301a370 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe06dffb640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe0600a3430 0x7fe0600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.485+0000 7fe074d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x55775300fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 0x55775300f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 0x55775300f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:48334/0 (socket says 172.21.15.64:48334) 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 -- 172.21.15.64:0/2207403749 learned_addr learned my addr 172.21.15.64:0/2207403749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe074d31640 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x55775300fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52570/0 (socket says 172.21.15.64:52570) 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe0600a3430 0x7fe0600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe06ffff640 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x55775301c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe06dffb640 1 -- 172.21.15.64:0/611415722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe0640f0520 con 0x7fe070102d40 2024-09-19T05:11:24.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 -- 172.21.15.64:0/2207403749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 msgr2=0x55775301c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x55775301c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 -- 172.21.15.64:0/2207403749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 msgr2=0x55775300fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x55775300fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 -- 172.21.15.64:0/2207403749 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557752f013a0 con 0x557753010b40 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe074d31640 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x55775300fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe0600a3430 0x7fe0600a58f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe068010c40 tx=0x7fe068017000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.486+0000 7fe075532640 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 0x55775300f090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe068012c50 tx=0x7fe068010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90b2ca0f514b8a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe05affd640 1 -- 172.21.15.64:0/2207403749 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068012e00 con 0x557753010b40 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe05affd640 1 -- 172.21.15.64:0/2207403749 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe068009c00 con 0x557753010b40 2024-09-19T05:11:24.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe05affd640 1 -- 172.21.15.64:0/2207403749 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068005240 con 0x557753010b40 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 -- 172.21.15.64:0/2207403749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 msgr2=0x55775300f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 0x55775300f090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe068012c50 tx=0x7fe068010740 comp rx=0 tx=0).stop 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 -- 172.21.15.64:0/2207403749 shutdown_connections 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x55775301c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x55775300fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 --2- 172.21.15.64:0/2207403749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557753010b40 0x55775300f090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.487+0000 7fe07936e740 1 -- 172.21.15.64:0/2207403749 >> 172.21.15.64:0/2207403749 conn(0x557752fd8310 msgr2=0x55775300bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 -- 172.21.15.64:0/2207403749 shutdown_connections 2024-09-19T05:11:24.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 -- 172.21.15.64:0/2207403749 wait complete. 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 -- start start 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 Processor -- start 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x557753019d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557753010b40 0x557753017030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775301a370 0x557753017640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55775301d010 con 0x55775300f5d0 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557752f30060 con 0x55775301a370 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55775301d190 con 0x557753010b40 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe07936e740 10 client.0 _ll_register_callbacks cb 0x557752c838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.488+0000 7fe074d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x557753019d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x557753019d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52576/0 (socket says 172.21.15.64:52576) 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 -- 172.21.15.64:0/857264198 learned_addr learned my addr 172.21.15.64:0/857264198 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:24.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe075532640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775301a370 0x557753017640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe06ffff640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557753010b40 0x557753017030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 -- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557753010b40 msgr2=0x557753017030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557753010b40 0x557753017030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 -- 172.21.15.64:0/857264198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775301a370 msgr2=0x557753017640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775301a370 0x557753017640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 -- 172.21.15.64:0/857264198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557752ce63c0 con 0x55775300f5d0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe06ffff640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557753010b40 0x557753017030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe074d31640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x557753019d80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe0640bc500 tx=0x7fe0640bc220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2253988cb33aa49f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe04ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe04ffff640 1 -- 172.21.15.64:0/857264198 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064049070 con 0x55775300f5d0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe04ffff640 1 -- 172.21.15.64:0/857264198 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe0640edef0 con 0x55775300f5d0 2024-09-19T05:11:24.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557752f82860 con 0x55775300f5d0 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe04ffff640 1 -- 172.21.15.64:0/857264198 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0640bcf00 con 0x55775300f5d0 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe07936e740 10 client.16719 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557753028ae0 con 0x55775300f5d0 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.489+0000 7fe04ffff640 1 -- 172.21.15.64:0/857264198 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557752f82860 con 0x55775300f5d0 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe04ffff640 1 -- 172.21.15.64:0/857264198 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0640b8e70 con 0x55775300f5d0 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe07936e740 10 client.16719 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe07936e740 10 client.16719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe07936e740 20 client.16719 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe07936e740 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753029f40 0x55775302c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe07936e740 4 client.16719 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557752f7cd70 con 0x557753029f40 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe06ffff640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753029f40 0x55775302c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe06ffff640 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753029f40 0x55775302c370 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23cf475c6b6f5c9f server_cookie=11d13315339cd6ff in_seq=0 out_seq=0 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.490+0000 7fe04ffff640 10 client.16719 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe04ffff640 1 -- 172.21.15.64:0/857264198 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557752f7cd70 con 0x557753029f40 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe04ffff640 10 client.16719 handle_command_reply: tid=0 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 client.16719 shutdown 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 msgr2=0x557753019d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x557753019d80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe0640bc500 tx=0x7fe0640bc220 comp rx=0 tx=0).stop 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 shutdown_connections 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775301a370 0x557753017640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557753010b40 0x557753017030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753029f40 0x55775302c370 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe04ffff640 0 client.16719 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 --2- 172.21.15.64:0/857264198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55775300f5d0 0x557753019d80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.491+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 >> 172.21.15.64:0/857264198 conn(0x557752fd8310 msgr2=0x557753011960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.492+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 shutdown_connections 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.492+0000 7fe07936e740 1 -- 172.21.15.64:0/857264198 wait complete. 2024-09-19T05:11:24.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.492+0000 7fe07936e740 20 client.16719 trim_cache size 0 max 16384 2024-09-19T05:11:24.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 Processor -- start 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 -- start start 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x5577530262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753031c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x5577530321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701b2230 con 0x557753010b40 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701afb00 con 0x55775300f5d0 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe07936e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701ad950 con 0x55775301a370 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753031c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753031c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52580/0 (socket says 172.21.15.64:52580) 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 -- 172.21.15.64:0/1531558648 learned_addr learned my addr 172.21.15.64:0/1531558648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:24.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe074d31640 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x5577530262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe075532640 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x5577530321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 -- 172.21.15.64:0/1531558648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 msgr2=0x5577530321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x5577530321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 -- 172.21.15.64:0/1531558648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 msgr2=0x5577530262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.505+0000 7fe06ffff640 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x5577530262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe06ffff640 1 -- 172.21.15.64:0/1531558648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775302c9f0 con 0x557753010b40 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe06ffff640 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753031c80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5577530168c0 tx=0x7fe05c008480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cba352f970b4eff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe05a7fc640 1 -- 172.21.15.64:0/1531558648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0130d0 con 0x557753010b40 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe05a7fc640 1 -- 172.21.15.64:0/1531558648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05c01e020 con 0x557753010b40 2024-09-19T05:11:24.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe05a7fc640 1 -- 172.21.15.64:0/1531558648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c003b80 con 0x557753010b40 2024-09-19T05:11:24.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe07936e740 1 -- 172.21.15.64:0/1531558648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 msgr2=0x557753031c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.506+0000 7fe07936e740 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753031c80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5577530168c0 tx=0x7fe05c008480 comp rx=0 tx=0).stop 2024-09-19T05:11:24.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 -- 172.21.15.64:0/1531558648 shutdown_connections 2024-09-19T05:11:24.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x5577530321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753031c80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 --2- 172.21.15.64:0/1531558648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x5577530262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 -- 172.21.15.64:0/1531558648 >> 172.21.15.64:0/1531558648 conn(0x557752fd8310 msgr2=0x5577530122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 -- 172.21.15.64:0/1531558648 shutdown_connections 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 -- 172.21.15.64:0/1531558648 wait complete. 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 -- start start 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 Processor -- start 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.507+0000 7fe07936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x557753014110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe07936e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753014650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe07936e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x557753014b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe07936e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701adfc0 con 0x557753010b40 2024-09-19T05:11:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe07936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701af430 con 0x55775300f5d0 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe07936e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701ae690 con 0x55775301a370 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe07936e740 10 client.0 _ll_register_callbacks cb 0x7fe064005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753014650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753014650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52594/0 (socket says 172.21.15.64:52594) 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe06ffff640 1 -- 172.21.15.64:0/3040436387 learned_addr learned my addr 172.21.15.64:0/3040436387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x557753014110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe075532640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x557753014b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 -- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 msgr2=0x557753014b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x557753014b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 -- 172.21.15.64:0/3040436387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 msgr2=0x557753014650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753014650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 -- 172.21.15.64:0/3040436387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775300a310 con 0x55775300f5d0 2024-09-19T05:11:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe06ffff640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753014650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe074d31640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x557753014110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0640b8c60 tx=0x7fe0640ee120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4384c965777273b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe04ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.508+0000 7fe04ffff640 1 -- 172.21.15.64:0/3040436387 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640d4030 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe04ffff640 1 -- 172.21.15.64:0/3040436387 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe0640c50b0 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c01e020 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe04ffff640 1 -- 172.21.15.64:0/3040436387 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe064049070 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 10 client.26267 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557753094e30 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe04ffff640 1 -- 172.21.15.64:0/3040436387 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe05c01e020 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe04ffff640 1 -- 172.21.15.64:0/3040436387 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe0640bdb30 con 0x55775300f5d0 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 10 client.26267 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:24.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 10 client.26267 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 20 client.26267 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753095450 0x557753097870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 4 client.26267 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557752ef9170 con 0x557753095450 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.509+0000 7fe06ffff640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753095450 0x557753097870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe06ffff640 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753095450 0x557753097870 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4642ae926e9cbc84 server_cookie=1e5bde2e494065bc in_seq=0 out_seq=0 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe04ffff640 10 client.26267 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe04ffff640 1 -- 172.21.15.64:0/3040436387 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557752ef9170 con 0x557753095450 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe04ffff640 10 client.26267 handle_command_reply: tid=0 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 client.26267 shutdown 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 msgr2=0x557753014110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x557753014110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0640b8c60 tx=0x7fe0640ee120 comp rx=0 tx=0).stop 2024-09-19T05:11:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 shutdown_connections 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55775301a370 0x557753014b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x557753095450 0x557753097870 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557753010b40 0x557753014650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 --2- 172.21.15.64:0/3040436387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55775300f5d0 0x557753014110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe04ffff640 0 client.26267 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 >> 172.21.15.64:0/3040436387 conn(0x557752fd8310 msgr2=0x557753011110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.510+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 shutdown_connections 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe07936e740 1 -- 172.21.15.64:0/3040436387 wait complete. 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe07936e740 20 client.26267 trim_cache size 0 max 16384 2024-09-19T05:11:24.519 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe0600a3430 msgr2=0x7fe0600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe0600a3430 0x7fe0600a58f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe068010c40 tx=0x7fe068017000 comp rx=0 tx=0).stop 2024-09-19T05:11:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 msgr2=0x7fe07019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe07019a4f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe064005940 tx=0x7fe064004270 comp rx=0 tx=0).stop 2024-09-19T05:11:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 shutdown_connections 2024-09-19T05:11:24.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe0600a3430 0x7fe0600a58f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe07010c170 0x7fe07019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe070103660 0x7fe07019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 --2- 172.21.15.64:0/611415722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe070102d40 0x7fe07019a4f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 >> 172.21.15.64:0/611415722 conn(0x7fe0700fc7a0 msgr2=0x7fe07010d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 shutdown_connections 2024-09-19T05:11:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:24.511+0000 7fe076f95640 1 -- 172.21.15.64:0/611415722 wait complete. 2024-09-19T05:11:25.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:24 smithi064 ceph-mon[30960]: pgmap v168: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:25.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:11:25.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:24 smithi103 ceph-mon[34446]: pgmap v168: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:25.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:24 smithi174 ceph-mon[34488]: pgmap v168: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:25.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 -- 172.21.15.64:0/3119433867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710106a90 msgr2=0x7f1710106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 --2- 172.21.15.64:0/3119433867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710106a90 0x7f1710106e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1700009cd0 tx=0x7f170002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 -- 172.21.15.64:0/3119433867 shutdown_connections 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 --2- 172.21.15.64:0/3119433867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1710101430 0x7f171010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 --2- 172.21.15.64:0/3119433867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710100a90 0x7f1710100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 --2- 172.21.15.64:0/3119433867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710106a90 0x7f1710106e70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 -- 172.21.15.64:0/3119433867 >> 172.21.15.64:0/3119433867 conn(0x7f17100fc7d0 msgr2=0x7f17100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 -- 172.21.15.64:0/3119433867 shutdown_connections 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.896+0000 7f1716aa9640 1 -- 172.21.15.64:0/3119433867 wait complete. 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 Processor -- start 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 -- start start 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 0x7f171019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710101430 0x7f171019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1710106a90 0x7f171019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1710110db0 con 0x7f1710100a90 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1710110c30 con 0x7f1710101430 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1716aa9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1710110f30 con 0x7f1710106a90 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 0x7f171019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 0x7f171019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52606/0 (socket says 172.21.15.64:52606) 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 -- 172.21.15.64:0/1732060663 learned_addr learned my addr 172.21.15.64:0/1732060663 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1715046640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1710106a90 0x7f171019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f170ffff640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710101430 0x7f171019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 -- 172.21.15.64:0/1732060663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1710106a90 msgr2=0x7f171019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1710106a90 0x7f171019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 -- 172.21.15.64:0/1732060663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710101430 msgr2=0x7f171019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710101430 0x7f171019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f1714845640 1 -- 172.21.15.64:0/1732060663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171019f540 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.897+0000 7f170ffff640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710101430 0x7f171019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.898+0000 7f1714845640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 0x7f171019a590 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f170002aed0 tx=0x7f1700003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.898+0000 7f170dffb640 1 -- 172.21.15.64:0/1732060663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700004450 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.898+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171019f7d0 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.898+0000 7f170dffb640 1 -- 172.21.15.64:0/1732060663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17000045f0 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.898+0000 7f170dffb640 1 -- 172.21.15.64:0/1732060663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700045690 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.898+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101a7110 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.900+0000 7f1718e82740 1 Processor -- start 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.900+0000 7f170dffb640 1 -- 172.21.15.64:0/1732060663 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f17000305a0 con 0x7f1710100a90 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 -- start start 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 0x564154a0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a19370 0x564154a1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641549742f0 con 0x564154a0e5d0 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56415491bcf0 con 0x564154a19370 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1718e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5641549702b0 con 0x564154a0fb40 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 0x564154a0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 0x564154a0ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52610/0 (socket says 172.21.15.64:52610) 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1715046640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 -- 172.21.15.64:0/485910708 learned_addr learned my addr 172.21.15.64:0/485910708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f170ffff640 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a19370 0x564154a1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 -- 172.21.15.64:0/485910708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 msgr2=0x564154a0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 -- 172.21.15.64:0/485910708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a19370 msgr2=0x564154a1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a19370 0x564154a1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1714845640 1 -- 172.21.15.64:0/485910708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641549003a0 con 0x564154a0e5d0 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f1715046640 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.901+0000 7f170dffb640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f16fc0a33b0 0x7f16fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.902+0000 7f1715046640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f16fc0a33b0 0x7f16fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.902+0000 7f170dffb640 1 -- 172.21.15.64:0/1732060663 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f170002e030 con 0x7f1710100a90 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.902+0000 7f1714845640 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 0x564154a0ea10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f17000eea80 tx=0x7f17000f9640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c11b2b6cb92c359 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:25.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.902+0000 7f1706ffd640 1 -- 172.21.15.64:0/485910708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17000bd4a0 con 0x564154a0e5d0 2024-09-19T05:11:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.902+0000 7f1706ffd640 1 -- 172.21.15.64:0/485910708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1700036070 con 0x564154a0e5d0 2024-09-19T05:11:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.902+0000 7f1706ffd640 1 -- 172.21.15.64:0/485910708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700037270 con 0x564154a0e5d0 2024-09-19T05:11:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 -- 172.21.15.64:0/485910708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 msgr2=0x564154a0ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1715046640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f16fc0a33b0 0x7f16fc0a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1708006fd0 tx=0x7f1708008040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 0x564154a0ea10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f17000eea80 tx=0x7f17000f9640 comp rx=0 tx=0).stop 2024-09-19T05:11:25.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 -- 172.21.15.64:0/485910708 shutdown_connections 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a19370 0x564154a1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0e5d0 0x564154a0ea10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 --2- 172.21.15.64:0/485910708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 -- 172.21.15.64:0/485910708 >> 172.21.15.64:0/485910708 conn(0x5641549d7310 msgr2=0x564154a0ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 -- 172.21.15.64:0/485910708 shutdown_connections 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 -- 172.21.15.64:0/485910708 wait complete. 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 -- start start 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.903+0000 7f1718e82740 1 Processor -- start 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a18de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 0x564154a167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564154a1be30 con 0x564154a19370 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56415492f060 con 0x564154a0e5d0 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564154a1bfb0 con 0x564154a0fb40 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1718e82740 10 client.0 _ll_register_callbacks cb 0x5641546828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 0x564154a167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 0x564154a167d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52626/0 (socket says 172.21.15.64:52626) 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 -- 172.21.15.64:0/640659017 learned_addr learned my addr 172.21.15.64:0/640659017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f170ffff640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1714845640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a18de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 -- 172.21.15.64:0/640659017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 msgr2=0x564154a16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a16090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 -- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 msgr2=0x564154a18de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a18de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 -- 172.21.15.64:0/640659017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641546e53c0 con 0x564154a19370 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1714845640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a18de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f1715046640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 0x564154a167d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1708011010 tx=0x7f17080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65605cb19d7457d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.904+0000 7f16effff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f16effff640 1 -- 172.21.15.64:0/640659017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708014800 con 0x564154a19370 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f16effff640 1 -- 172.21.15.64:0/640659017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f17080149a0 con 0x564154a19370 2024-09-19T05:11:25.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1700036070 con 0x564154a19370 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f16effff640 1 -- 172.21.15.64:0/640659017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f170802e050 con 0x564154a19370 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 10 client.16737 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564154a27950 con 0x564154a19370 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f16effff640 1 -- 172.21.15.64:0/640659017 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1700036070 con 0x564154a19370 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f16effff640 1 -- 172.21.15.64:0/640659017 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1708029030 con 0x564154a19370 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 10 client.16737 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 20 client.16737 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a28e10 0x564154a2b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 4 client.16737 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.905+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564154981860 con 0x564154a28e10 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.906+0000 7f1714845640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a28e10 0x564154a2b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.906+0000 7f1714845640 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a28e10 0x564154a2b280 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fc2417435b2ce4 server_cookie=252d51f8f00cc669 in_seq=0 out_seq=0 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.906+0000 7f16effff640 10 client.16737 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f16effff640 1 -- 172.21.15.64:0/640659017 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564154981860 con 0x564154a28e10 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f16effff640 10 client.16737 handle_command_reply: tid=0 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 client.16737 shutdown 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 msgr2=0x564154a167d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 0x564154a167d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1708011010 tx=0x7f17080046c0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 shutdown_connections 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a28e10 0x564154a2b280 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a19370 0x564154a167d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a0fb40 0x564154a16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 --2- 172.21.15.64:0/640659017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a18de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 >> 172.21.15.64:0/640659017 conn(0x5641549d7310 msgr2=0x564154a10960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f16effff640 0 client.16737 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 shutdown_connections 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 1 -- 172.21.15.64:0/640659017 wait complete. 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.907+0000 7f1718e82740 20 client.16737 trim_cache size 0 max 16384 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.920+0000 7f1718e82740 1 Processor -- start 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 -- start start 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a0ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a30b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b21e0 con 0x564154a0fb40 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17101afab0 con 0x564154a0e5d0 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1718e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17101ad900 con 0x564154a19370 2024-09-19T05:11:25.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a25090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52642/0 (socket says 172.21.15.64:52642) 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 -- 172.21.15.64:0/2484617913 learned_addr learned my addr 172.21.15.64:0/2484617913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1714845640 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a0ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f1715046640 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a30b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 -- 172.21.15.64:0/2484617913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 msgr2=0x564154a30b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a30b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 -- 172.21.15.64:0/2484617913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 msgr2=0x564154a0ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a0ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 -- 172.21.15.64:0/2484617913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564154a05a30 con 0x564154a0fb40 2024-09-19T05:11:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.921+0000 7f170ffff640 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a25090 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f17100fec10 tx=0x7f16f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16f18e863e0dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f17067fc640 1 -- 172.21.15.64:0/2484617913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f80110d0 con 0x564154a0fb40 2024-09-19T05:11:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f17067fc640 1 -- 172.21.15.64:0/2484617913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16f8011270 con 0x564154a0fb40 2024-09-19T05:11:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f17067fc640 1 -- 172.21.15.64:0/2484617913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f801b470 con 0x564154a0fb40 2024-09-19T05:11:25.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 -- 172.21.15.64:0/2484617913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 msgr2=0x564154a25090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a25090 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f17100fec10 tx=0x7f16f8011810 comp rx=0 tx=0).stop 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 -- 172.21.15.64:0/2484617913 shutdown_connections 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a30b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a25090 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 --2- 172.21.15.64:0/2484617913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a0ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 -- 172.21.15.64:0/2484617913 >> 172.21.15.64:0/2484617913 conn(0x5641549d7310 msgr2=0x564154a0a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 -- 172.21.15.64:0/2484617913 shutdown_connections 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.922+0000 7f1718e82740 1 -- 172.21.15.64:0/2484617913 wait complete. 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 -- start start 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 Processor -- start 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a21200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a21740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17101adf70 con 0x564154a0fb40 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17101af3e0 con 0x564154a0e5d0 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17101ae640 con 0x564154a19370 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1718e82740 10 client.0 _ll_register_callbacks cb 0x7f1708006ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f170ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a21740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1714845640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a21200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f170ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a21740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52654/0 (socket says 172.21.15.64:52654) 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f170ffff640 1 -- 172.21.15.64:0/226433961 learned_addr learned my addr 172.21.15.64:0/226433961 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.923+0000 7f1715046640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f170ffff640 1 -- 172.21.15.64:0/226433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 msgr2=0x564154a126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f170ffff640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f170ffff640 1 -- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 msgr2=0x564154a21200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f170ffff640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a21200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f170ffff640 1 -- 172.21.15.64:0/226433961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564154a12be0 con 0x564154a0fb40 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f1714845640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a21200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f170ffff640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a21740 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005250 tx=0x7f16f8005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d836016022b6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f16effff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f16effff640 1 -- 172.21.15.64:0/226433961 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f80110d0 con 0x564154a0fb40 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f8011270 con 0x564154a0fb40 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f16effff640 1 -- 172.21.15.64:0/226433961 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f16f801b470 con 0x564154a0fb40 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f16effff640 1 -- 172.21.15.64:0/226433961 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16f80046c0 con 0x564154a0fb40 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f1718e82740 10 client.16743 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:25.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564154a93340 con 0x564154a0fb40 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.924+0000 7f16effff640 1 -- 172.21.15.64:0/226433961 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16f8011270 con 0x564154a0fb40 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f16effff640 1 -- 172.21.15.64:0/226433961 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16f8022070 con 0x564154a0fb40 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1718e82740 10 client.16743 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1718e82740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1718e82740 20 client.16743 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1718e82740 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a939f0 0x564154a95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1718e82740 4 client.16743 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56415497bd70 con 0x564154a939f0 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1714845640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a939f0 0x564154a95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:25.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f1714845640 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a939f0 0x564154a95e10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8381b7a5e47b18c6 server_cookie=ec4bcb5d96fd660b in_seq=0 out_seq=0 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f16effff640 10 client.16743 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f16effff640 1 -- 172.21.15.64:0/226433961 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56415497bd70 con 0x564154a939f0 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.925+0000 7f16effff640 10 client.16743 handle_command_reply: tid=0 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 client.16743 shutdown 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 msgr2=0x564154a21740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a21740 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005250 tx=0x7f16f8005280 comp rx=0 tx=0).stop 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 shutdown_connections 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564154a19370 0x564154a126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564154a939f0 0x564154a95e10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564154a0fb40 0x564154a21740 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f16effff640 0 client.16743 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:25.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 --2- 172.21.15.64:0/226433961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564154a0e5d0 0x564154a21200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 >> 172.21.15.64:0/226433961 conn(0x5641549d7310 msgr2=0x564154a14980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:25.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 shutdown_connections 2024-09-19T05:11:25.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 1 -- 172.21.15.64:0/226433961 wait complete. 2024-09-19T05:11:25.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.926+0000 7f1718e82740 20 client.16743 trim_cache size 0 max 16384 2024-09-19T05:11:25.944 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f16fc0a33b0 msgr2=0x7f16fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f16fc0a33b0 0x7f16fc0a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1708006fd0 tx=0x7f1708008040 comp rx=0 tx=0).stop 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 msgr2=0x7f171019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 0x7f171019a590 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f170002aed0 tx=0x7f1700003c20 comp rx=0 tx=0).stop 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 shutdown_connections 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f16fc0a33b0 0x7f16fc0a5870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1710106a90 0x7f171019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1710101430 0x7f171019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 --2- 172.21.15.64:0/1732060663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1710100a90 0x7f171019a590 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:25.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 >> 172.21.15.64:0/1732060663 conn(0x7f17100fc7d0 msgr2=0x7f171006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:25.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 shutdown_connections 2024-09-19T05:11:25.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:25.927+0000 7f1716aa9640 1 -- 172.21.15.64:0/1732060663 wait complete. 2024-09-19T05:11:27.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:26 smithi064 ceph-mon[30960]: pgmap v169: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:27.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:26 smithi103 ceph-mon[34446]: pgmap v169: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:27.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:26 smithi174 ceph-mon[34488]: pgmap v169: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:27.576 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:11:27.576 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:11:27.848 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 -- 172.21.15.64:0/2039244944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c106f10 msgr2=0x7f273c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 --2- 172.21.15.64:0/2039244944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c106f10 0x7f273c10ea90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2734009ca0 tx=0x7f273402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 -- 172.21.15.64:0/2039244944 shutdown_connections 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 --2- 172.21.15.64:0/2039244944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c106f10 0x7f273c10ea90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 --2- 172.21.15.64:0/2039244944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f273c102790 0x7f273c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 --2- 172.21.15.64:0/2039244944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c101de0 0x7f273c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 -- 172.21.15.64:0/2039244944 >> 172.21.15.64:0/2039244944 conn(0x7f273c0fd8c0 msgr2=0x7f273c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:28.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 -- 172.21.15.64:0/2039244944 shutdown_connections 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.314+0000 7f2740b44640 1 -- 172.21.15.64:0/2039244944 wait complete. 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 Processor -- start 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 -- start start 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f273c101de0 0x7f273c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 0x7f273c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c106f10 0x7f273c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f273c111e20 con 0x7f273c102790 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f273c111ca0 con 0x7f273c106f10 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2740b44640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f273c111fa0 con 0x7f273c101de0 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 0x7f273c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 0x7f273c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52662/0 (socket says 172.21.15.64:52662) 2024-09-19T05:11:28.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.315+0000 7f2733fff640 1 -- 172.21.15.64:0/788637569 learned_addr learned my addr 172.21.15.64:0/788637569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f273a59c640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f273c101de0 0x7f273c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f273ad9d640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c106f10 0x7f273c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2733fff640 1 -- 172.21.15.64:0/788637569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f273c101de0 msgr2=0x7f273c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2733fff640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f273c101de0 0x7f273c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2733fff640 1 -- 172.21.15.64:0/788637569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c106f10 msgr2=0x7f273c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2733fff640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c106f10 0x7f273c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2733fff640 1 -- 172.21.15.64:0/788637569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273c19d2f0 con 0x7f273c102790 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f273ad9d640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c106f10 0x7f273c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2733fff640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 0x7f273c198880 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f272400d770 tx=0x7f272400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:28.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f27337fe640 1 -- 172.21.15.64:0/788637569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724017070 con 0x7f273c102790 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273c19d5e0 con 0x7f273c102790 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f27337fe640 1 -- 172.21.15.64:0/788637569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27240054d0 con 0x7f273c102790 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f27337fe640 1 -- 172.21.15.64:0/788637569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724004cf0 con 0x7f273c102790 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.316+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273c1a4ec0 con 0x7f273c102790 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f27337fe640 1 -- 172.21.15.64:0/788637569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2724004e90 con 0x7f273c102790 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 Processor -- start 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 -- start start 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5575ad435280 con 0x5575ad4d0480 2024-09-19T05:11:28.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5575ad3dcb10 con 0x5575ad4cfb60 2024-09-19T05:11:28.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.319+0000 7f2742f1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575ad431240 con 0x5575ad4dad40 2024-09-19T05:11:28.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4da800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52676/0 (socket says 172.21.15.64:52676) 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 -- 172.21.15.64:0/2795720464 learned_addr learned my addr 172.21.15.64:0/2795720464 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f27337fe640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27280a33b0 0x7f27280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273ad9d640 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273ad9d640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27280a33b0 0x7f27280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f2733fff640 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f27337fe640 1 -- 172.21.15.64:0/788637569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27240cf6f0 con 0x7f273c102790 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 -- 172.21.15.64:0/2795720464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 msgr2=0x5575ad4dd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 -- 172.21.15.64:0/2795720464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 msgr2=0x5575ad4cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 -- 172.21.15.64:0/2795720464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ad3e91f0 con 0x5575ad4d0480 2024-09-19T05:11:28.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273a59c640 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4da800 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f272c00bc90 tx=0x7f272c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cbd11b4eece207 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f273ad9d640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27280a33b0 0x7f27280a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2734005fa0 tx=0x7f27340095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f2730ff9640 1 -- 172.21.15.64:0/2795720464 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c004030 con 0x5575ad4d0480 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f2730ff9640 1 -- 172.21.15.64:0/2795720464 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272c00d920 con 0x5575ad4d0480 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.320+0000 7f2730ff9640 1 -- 172.21.15.64:0/2795720464 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c003b00 con 0x5575ad4d0480 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 -- 172.21.15.64:0/2795720464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 msgr2=0x5575ad4da800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4da800 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f272c00bc90 tx=0x7f272c012b30 comp rx=0 tx=0).stop 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 -- 172.21.15.64:0/2795720464 shutdown_connections 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4da800 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2795720464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 -- 172.21.15.64:0/2795720464 >> 172.21.15.64:0/2795720464 conn(0x5575ad4cc0f0 msgr2=0x5575ad4cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 -- 172.21.15.64:0/2795720464 shutdown_connections 2024-09-19T05:11:28.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.321+0000 7f2742f1d740 1 -- 172.21.15.64:0/2795720464 wait complete. 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 -- start start 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 Processor -- start 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 0x5575ad4d8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5575ad4dd910 con 0x5575ad4dad40 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5575ad3eff10 con 0x5575ad4cfb60 2024-09-19T05:11:28.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575ad4dda90 con 0x5575ad4d0480 2024-09-19T05:11:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2742f1d740 10 client.0 _ll_register_callbacks cb 0x5575ad3b8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f273ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 0x5575ad4d8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f273ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 0x5575ad4d8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52684/0 (socket says 172.21.15.64:52684) 2024-09-19T05:11:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f273ad9d640 1 -- 172.21.15.64:0/3653195734 learned_addr learned my addr 172.21.15.64:0/3653195734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f2733fff640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f273a59c640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.322+0000 7f273ad9d640 1 -- 172.21.15.64:0/3653195734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 msgr2=0x5575ad4d80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f273ad9d640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4d80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f273ad9d640 1 -- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 msgr2=0x5575ad4d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f273ad9d640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4d7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f273ad9d640 1 -- 172.21.15.64:0/3653195734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ad1a66a0 con 0x5575ad4dad40 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f273ad9d640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 0x5575ad4d8600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2734005ca0 tx=0x7f273402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aead50d92e7b2a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2721ffb640 1 -- 172.21.15.64:0/3653195734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734051070 con 0x5575ad4dad40 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272c00d920 con 0x5575ad4dad40 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2721ffb640 1 -- 172.21.15.64:0/3653195734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f27340047a0 con 0x5575ad4dad40 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2721ffb640 1 -- 172.21.15.64:0/3653195734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27340305c0 con 0x5575ad4dad40 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2742f1d740 10 client.16755 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575ad4e94d0 con 0x5575ad4dad40 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.323+0000 7f2721ffb640 1 -- 172.21.15.64:0/3653195734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272c00d920 con 0x5575ad4dad40 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2721ffb640 1 -- 172.21.15.64:0/3653195734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f273404a040 con 0x5575ad4dad40 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2742f1d740 10 client.16755 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2742f1d740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2742f1d740 20 client.16755 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2742f1d740 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad4ea8e0 0x5575ad4eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2742f1d740 4 client.16755 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ad4427f0 con 0x5575ad4ea8e0 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f273a59c640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad4ea8e0 0x5575ad4eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f2721ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.324+0000 7f273a59c640 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad4ea8e0 0x5575ad4eccd0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2909e47becc9b73 server_cookie=c0d155b35b60b503 in_seq=0 out_seq=0 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2721ffb640 1 -- 172.21.15.64:0/3653195734 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575ad4427f0 con 0x5575ad4ea8e0 2024-09-19T05:11:28.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2721ffb640 10 client.16755 handle_command_reply: tid=0 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 client.16755 shutdown 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 msgr2=0x5575ad4d8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 0x5575ad4d8600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2734005ca0 tx=0x7f273402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 shutdown_connections 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad4ea8e0 0x5575ad4eccd0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4dad40 0x5575ad4d8600 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 --2- 172.21.15.64:0/3653195734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 >> 172.21.15.64:0/3653195734 conn(0x5575ad4cc0f0 msgr2=0x5575ad4d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2721ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 shutdown_connections 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 1 -- 172.21.15.64:0/3653195734 wait complete. 2024-09-19T05:11:28.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.325+0000 7f2742f1d740 20 client.16755 trim_cache size 0 max 16384 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 Processor -- start 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 -- start start 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 0x5575ad4d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4dad40 0x5575ad4f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1adf50 con 0x5575ad4cfb60 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1ab820 con 0x5575ad4dad40 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f2742f1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1a9670 con 0x5575ad4d0480 2024-09-19T05:11:28.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f273a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 0x5575ad4d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f273a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 0x5575ad4d99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52692/0 (socket says 172.21.15.64:52692) 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.343+0000 7f273a59c640 1 -- 172.21.15.64:0/2880535031 learned_addr learned my addr 172.21.15.64:0/2880535031 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273ad9d640 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4dad40 0x5575ad4f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273a59c640 1 -- 172.21.15.64:0/2880535031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 msgr2=0x5575ad4f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273a59c640 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273a59c640 1 -- 172.21.15.64:0/2880535031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4dad40 msgr2=0x5575ad4f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273a59c640 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4dad40 0x5575ad4f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273a59c640 1 -- 172.21.15.64:0/2880535031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ad4cbf20 con 0x5575ad4cfb60 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273ad9d640 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4dad40 0x5575ad4f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f273a59c640 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 0x5575ad4d99e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c00b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93354afad300a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f2723fff640 1 -- 172.21.15.64:0/2880535031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c00dca0 con 0x5575ad4cfb60 2024-09-19T05:11:28.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f2723fff640 1 -- 172.21.15.64:0/2880535031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272c00de40 con 0x5575ad4cfb60 2024-09-19T05:11:28.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.344+0000 7f2723fff640 1 -- 172.21.15.64:0/2880535031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c003b80 con 0x5575ad4cfb60 2024-09-19T05:11:28.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 -- 172.21.15.64:0/2880535031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 msgr2=0x5575ad4d99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 0x5575ad4d99e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c00b770 comp rx=0 tx=0).stop 2024-09-19T05:11:28.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 -- 172.21.15.64:0/2880535031 shutdown_connections 2024-09-19T05:11:28.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4dad40 0x5575ad4f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4d0480 0x5575ad4f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 --2- 172.21.15.64:0/2880535031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4cfb60 0x5575ad4d99e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 -- 172.21.15.64:0/2880535031 >> 172.21.15.64:0/2880535031 conn(0x5575ad4cc0f0 msgr2=0x5575ad4d92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 -- 172.21.15.64:0/2880535031 shutdown_connections 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 -- 172.21.15.64:0/2880535031 wait complete. 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 -- start start 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.345+0000 7f2742f1d740 1 Processor -- start 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1a9ce0 con 0x5575ad4d0480 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1ab150 con 0x5575ad4cfb60 2024-09-19T05:11:28.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f273c1aa3b0 con 0x5575ad4dad40 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2742f1d740 10 client.0 _ll_register_callbacks cb 0x7f2734005bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4de850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52696/0 (socket says 172.21.15.64:52696) 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 -- 172.21.15.64:0/4025259703 learned_addr learned my addr 172.21.15.64:0/4025259703 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f273a59c640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f273ad9d640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 -- 172.21.15.64:0/4025259703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 msgr2=0x5575ad4dee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 -- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 msgr2=0x5575ad4e3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4e3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 -- 172.21.15.64:0/4025259703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ad4c7830 con 0x5575ad4d0480 2024-09-19T05:11:28.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2733fff640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4de850 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2724021000 tx=0x7f2724022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665f54f5e009e0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.346+0000 7f2721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2721ffb640 1 -- 172.21.15.64:0/4025259703 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724020040 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2721ffb640 1 -- 172.21.15.64:0/4025259703 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2724098040 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272c00de40 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2721ffb640 1 -- 172.21.15.64:0/4025259703 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f272409b330 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 10 client.16761 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575ad5554a0 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2721ffb640 1 -- 172.21.15.64:0/4025259703 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272c00de40 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2721ffb640 1 -- 172.21.15.64:0/4025259703 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2724018b40 con 0x5575ad4d0480 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 10 client.16761 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 20 client.16761 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:28.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad555c70 0x5575ad558090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 4 client.16761 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575ad43cd00 con 0x5575ad555c70 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.347+0000 7f273a59c640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad555c70 0x5575ad558090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.348+0000 7f273a59c640 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad555c70 0x5575ad558090 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18a06ead372b22c server_cookie=7d862988f00ae59a in_seq=0 out_seq=0 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.348+0000 7f2721ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2721ffb640 1 -- 172.21.15.64:0/4025259703 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575ad43cd00 con 0x5575ad555c70 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2721ffb640 10 client.16761 handle_command_reply: tid=0 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 client.16761 shutdown 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 msgr2=0x5575ad4de850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4de850 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2724021000 tx=0x7f2724022000 comp rx=0 tx=0).stop 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 shutdown_connections 2024-09-19T05:11:28.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575ad4dad40 0x5575ad4dee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5575ad555c70 0x5575ad558090 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5575ad4d0480 0x5575ad4de850 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 --2- 172.21.15.64:0/4025259703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575ad4cfb60 0x5575ad4e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2721ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 >> 172.21.15.64:0/4025259703 conn(0x5575ad4cc0f0 msgr2=0x5575ad4d8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 shutdown_connections 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 1 -- 172.21.15.64:0/4025259703 wait complete. 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.349+0000 7f2742f1d740 20 client.16761 trim_cache size 0 max 16384 2024-09-19T05:11:28.357 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:11:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "3d728406-a9b1-482b-827c-fec6b255be3a", 2024-09-19T05:11:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:11:28.358 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27280a33b0 msgr2=0x7f27280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27280a33b0 0x7f27280a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2734005fa0 tx=0x7f27340095c0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 msgr2=0x7f273c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 0x7f273c198880 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f272400d770 tx=0x7f272400dc30 comp rx=0 tx=0).stop 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 shutdown_connections 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27280a33b0 0x7f27280a5870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273c106f10 0x7f273c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f273c102790 0x7f273c198880 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 --2- 172.21.15.64:0/788637569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f273c101de0 0x7f273c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 >> 172.21.15.64:0/788637569 conn(0x7f273c0fd8c0 msgr2=0x7f273c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:28.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 shutdown_connections 2024-09-19T05:11:28.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:28.350+0000 7f2740b44640 1 -- 172.21.15.64:0/788637569 wait complete. 2024-09-19T05:11:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:28 smithi064 ceph-mon[30960]: pgmap v170: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:28 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:11:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:28 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:28 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:11:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:28 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:28 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:11:29.178 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:29.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:28 smithi103 ceph-mon[34446]: pgmap v170: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:29.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:28 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:11:29.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:28 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:29.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:28 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:11:29.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:28 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:29.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:28 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:11:29.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:28 smithi174 ceph-mon[34488]: pgmap v170: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:29.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:28 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:11:29.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:28 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:29.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:28 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:11:29.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:28 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:29.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:28 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:11:29.441 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 -- 172.21.15.64:0/890470018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4106a80 msgr2=0x7f05a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 --2- 172.21.15.64:0/890470018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4106a80 0x7f05a4106e60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0594009930 tx=0x7f059402a420 comp rx=0 tx=0).stop 2024-09-19T05:11:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 -- 172.21.15.64:0/890470018 shutdown_connections 2024-09-19T05:11:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 --2- 172.21.15.64:0/890470018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4101420 0x7f05a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 --2- 172.21.15.64:0/890470018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05a4100a80 0x7f05a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 --2- 172.21.15.64:0/890470018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4106a80 0x7f05a4106e60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 -- 172.21.15.64:0/890470018 >> 172.21.15.64:0/890470018 conn(0x7f05a40fc7a0 msgr2=0x7f05a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 -- 172.21.15.64:0/890470018 shutdown_connections 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.905+0000 7f05a885e640 1 -- 172.21.15.64:0/890470018 wait complete. 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 Processor -- start 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 -- start start 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05a4100a80 0x7f05a4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 0x7f05a4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4106a80 0x7f05a41a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4110d50 con 0x7f05a4101420 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4110bd0 con 0x7f05a4106a80 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a885e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4110ed0 con 0x7f05a4100a80 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 0x7f05a4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 0x7f05a4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52718/0 (socket says 172.21.15.64:52718) 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 -- 172.21.15.64:0/3316248401 learned_addr learned my addr 172.21.15.64:0/3316248401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a259c640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05a4100a80 0x7f05a4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4106a80 0x7f05a41a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 -- 172.21.15.64:0/3316248401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05a4100a80 msgr2=0x7f05a4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05a4100a80 0x7f05a4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 -- 172.21.15.64:0/3316248401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4106a80 msgr2=0x7f05a41a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4106a80 0x7f05a41a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f059bfff640 1 -- 172.21.15.64:0/3316248401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a41a47e0 con 0x7f05a4101420 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.906+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4106a80 0x7f05a41a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.907+0000 7f059bfff640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 0x7f05a4199760 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b4a0 tx=0x7f058c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.907+0000 7f059b7fe640 1 -- 172.21.15.64:0/3316248401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c014070 con 0x7f05a4101420 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.907+0000 7f059b7fe640 1 -- 172.21.15.64:0/3316248401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f058c010ac0 con 0x7f05a4101420 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.907+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a41a49b0 con 0x7f05a4101420 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.907+0000 7f059b7fe640 1 -- 172.21.15.64:0/3316248401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c017c30 con 0x7f05a4101420 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.907+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a41a4ea0 con 0x7f05a4101420 2024-09-19T05:11:29.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.909+0000 7f059b7fe640 1 -- 172.21.15.64:0/3316248401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f058c017070 con 0x7f05a4101420 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 Processor -- start 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 -- start start 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe4fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 0x5586fe509140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f059b7fe640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05900a33b0 0x7f05900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe50ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5586fe4642e0 con 0x5586fe4fedc0 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5586fe40bce0 con 0x5586fe509680 2024-09-19T05:11:29.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05aac37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586fe4602a0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05a259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 0x5586fe509140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05a259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 0x5586fe509140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52730/0 (socket says 172.21.15.64:52730) 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05a259c640 1 -- 172.21.15.64:0/1065219150 learned_addr learned my addr 172.21.15.64:0/1065219150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe50ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f059bfff640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05900a33b0 0x7f05900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.910+0000 7f059b7fe640 1 -- 172.21.15.64:0/3316248401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f058c0cefc0 con 0x7f05a4101420 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe4fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a259c640 1 -- 172.21.15.64:0/1065219150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 msgr2=0x5586fe4fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a259c640 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe4fe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a259c640 1 -- 172.21.15.64:0/1065219150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 msgr2=0x5586fe50ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a259c640 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe50ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a259c640 1 -- 172.21.15.64:0/1065219150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fe3f0440 con 0x5586fe4fedc0 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f059bfff640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05900a33b0 0x7f05900a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f058c0cc8d0 tx=0x7f058c01c000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f05a259c640 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 0x5586fe509140 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0594009930 tx=0x7f0594032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152ecdd812dd5fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f0598ff9640 1 -- 172.21.15.64:0/1065219150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594035040 con 0x5586fe4fedc0 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f0598ff9640 1 -- 172.21.15.64:0/1065219150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05940316a0 con 0x5586fe4fedc0 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.911+0000 7f0598ff9640 1 -- 172.21.15.64:0/1065219150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05940047a0 con 0x5586fe4fedc0 2024-09-19T05:11:29.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 -- 172.21.15.64:0/1065219150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 msgr2=0x5586fe509140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 0x5586fe509140 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0594009930 tx=0x7f0594032d60 comp rx=0 tx=0).stop 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 -- 172.21.15.64:0/1065219150 shutdown_connections 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe50ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fedc0 0x5586fe509140 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 --2- 172.21.15.64:0/1065219150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe4fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 -- 172.21.15.64:0/1065219150 >> 172.21.15.64:0/1065219150 conn(0x5586fe4faa30 msgr2=0x5586fe4fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 -- 172.21.15.64:0/1065219150 shutdown_connections 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 -- 172.21.15.64:0/1065219150 wait complete. 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 -- start start 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.912+0000 7f05aac37740 1 Processor -- start 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe508ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe506180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe5066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5586fe50c250 con 0x5586fe4fe4a0 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5586fe41f050 con 0x5586fe509680 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586fe50c3d0 con 0x5586fe4fedc0 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05aac37740 10 client.0 _ll_register_callbacks cb 0x5586fe45f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe508ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe508ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52740/0 (socket says 172.21.15.64:52740) 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2718806257 learned_addr learned my addr 172.21.15.64:0/2718806257 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a259c640 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe506180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2718806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 msgr2=0x5586fe506180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe506180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 msgr2=0x5586fe5066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe5066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2718806257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fe1d53c0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe508ed0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f059c008010 tx=0x7f059c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa73b1cf998b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f0589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.913+0000 7f0589ffb640 1 -- 172.21.15.64:0/2718806257 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c018030 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f0589ffb640 1 -- 172.21.15.64:0/2718806257 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f059c0130d0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05940316a0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f0589ffb640 1 -- 172.21.15.64:0/2718806257 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f059c0132d0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 10 client.16773 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586fe517bb0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f0589ffb640 1 -- 172.21.15.64:0/2718806257 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05940316a0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f0589ffb640 1 -- 172.21.15.64:0/2718806257 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f059c026770 con 0x5586fe4fe4a0 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 10 client.16773 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:29.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 20 client.16773 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe518fc0 0x5586fe51b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 4 client.16773 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586fe471850 con 0x5586fe518fc0 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.914+0000 7f05a259c640 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe518fc0 0x5586fe51b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.915+0000 7f05a259c640 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe518fc0 0x5586fe51b410 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed918d8aa7848f5b server_cookie=6c342fb5fb81a3d8 in_seq=0 out_seq=0 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.915+0000 7f0589ffb640 10 client.16773 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f0589ffb640 1 -- 172.21.15.64:0/2718806257 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586fe471850 con 0x5586fe518fc0 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f0589ffb640 10 client.16773 handle_command_reply: tid=0 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 client.16773 shutdown 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 msgr2=0x5586fe508ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe508ed0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f059c008010 tx=0x7f059c010800 comp rx=0 tx=0).stop 2024-09-19T05:11:29.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 shutdown_connections 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe5066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe506180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe518fc0 0x5586fe51b410 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 --2- 172.21.15.64:0/2718806257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe508ed0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 >> 172.21.15.64:0/2718806257 conn(0x5586fe4faa30 msgr2=0x5586fe500710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f0589ffb640 0 client.16773 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 shutdown_connections 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 1 -- 172.21.15.64:0/2718806257 wait complete. 2024-09-19T05:11:29.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.916+0000 7f05aac37740 20 client.16773 trim_cache size 0 max 16384 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.933+0000 7f05aac37740 1 Processor -- start 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.933+0000 7f05aac37740 1 -- start start 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.933+0000 7f05aac37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe520ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.933+0000 7f05aac37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe4fedc0 0x5586fe521220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05aac37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 0x5586fe5033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05aac37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b0130 con 0x5586fe509680 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05aac37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ada00 con 0x5586fe4fedc0 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05aac37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ab850 con 0x5586fe4fe4a0 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 0x5586fe5033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 0x5586fe5033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52746/0 (socket says 172.21.15.64:52746) 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2685007591 learned_addr learned my addr 172.21.15.64:0/2685007591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:29.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe520ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a259c640 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe4fedc0 0x5586fe521220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2685007591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 msgr2=0x5586fe520ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe520ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2685007591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe4fedc0 msgr2=0x5586fe521220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe4fedc0 0x5586fe521220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 -- 172.21.15.64:0/2685007591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fe4febf0 con 0x5586fe509680 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a259c640 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe4fedc0 0x5586fe521220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 0x5586fe5033e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f059c013530 tx=0x7f059c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a2c028b3629ada server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.934+0000 7f058bfff640 1 -- 172.21.15.64:0/2685007591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c018040 con 0x5586fe509680 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f058bfff640 1 -- 172.21.15.64:0/2685007591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f059c026930 con 0x5586fe509680 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f058bfff640 1 -- 172.21.15.64:0/2685007591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c004280 con 0x5586fe509680 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 -- 172.21.15.64:0/2685007591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 msgr2=0x5586fe5033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 0x5586fe5033e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f059c013530 tx=0x7f059c004a40 comp rx=0 tx=0).stop 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 -- 172.21.15.64:0/2685007591 shutdown_connections 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe509680 0x5586fe5033e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe4fedc0 0x5586fe521220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 --2- 172.21.15.64:0/2685007591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fe4a0 0x5586fe520ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 -- 172.21.15.64:0/2685007591 >> 172.21.15.64:0/2685007591 conn(0x5586fe4faa30 msgr2=0x5586fe4f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 -- 172.21.15.64:0/2685007591 shutdown_connections 2024-09-19T05:11:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.935+0000 7f05aac37740 1 -- 172.21.15.64:0/2685007591 wait complete. 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 -- start start 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 Processor -- start 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe50fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe5101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe57cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41abec0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ad330 con 0x5586fe509680 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ac590 con 0x5586fe4fedc0 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05aac37740 10 client.0 _ll_register_callbacks cb 0x7f059c007530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe50fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05a2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe50fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52752/0 (socket says 172.21.15.64:52752) 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05a2d9d640 1 -- 172.21.15.64:0/192139317 learned_addr learned my addr 172.21.15.64:0/192139317 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe57cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05a259c640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe5101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.936+0000 7f05a2d9d640 1 -- 172.21.15.64:0/192139317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 msgr2=0x5586fe5101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe5101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05a2d9d640 1 -- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 msgr2=0x5586fe57cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe57cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05a2d9d640 1 -- 172.21.15.64:0/192139317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fe57d280 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05a259c640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe5101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05a2d9d640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe50fc60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f059c013f00 tx=0x7f059c007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fe7a6df2391cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f0589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f0589ffb640 1 -- 172.21.15.64:0/192139317 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c018040 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f0589ffb640 1 -- 172.21.15.64:0/192139317 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f059c003be0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586fe506a50 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f0589ffb640 1 -- 172.21.15.64:0/192139317 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f059c004ab0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 10 client.16779 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586fe584610 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f0589ffb640 1 -- 172.21.15.64:0/192139317 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5586fe506a50 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f0589ffb640 1 -- 172.21.15.64:0/192139317 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f059c0045b0 con 0x5586fe4fe4a0 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 10 client.16779 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 20 client.16779 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe584b70 0x5586fe586f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 4 client.16779 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.937+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586fe46bd60 con 0x5586fe584b70 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.938+0000 7f05a259c640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe584b70 0x5586fe586f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.938+0000 7f05a259c640 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe584b70 0x5586fe586f60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50efd0219cd4a7cd server_cookie=e6f251e241f6105b in_seq=0 out_seq=0 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.938+0000 7f0589ffb640 10 client.16779 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.938+0000 7f0589ffb640 1 -- 172.21.15.64:0/192139317 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5586fe46bd60 con 0x5586fe584b70 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.938+0000 7f0589ffb640 10 client.16779 handle_command_reply: tid=0 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 client.16779 shutdown 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 msgr2=0x5586fe50fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe50fc60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f059c013f00 tx=0x7f059c007640 comp rx=0 tx=0).stop 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 shutdown_connections 2024-09-19T05:11:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586fe509680 0x5586fe57cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586fe4fedc0 0x5586fe5101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5586fe584b70 0x5586fe586f60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 --2- 172.21.15.64:0/192139317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5586fe4fe4a0 0x5586fe50fc60 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 >> 172.21.15.64:0/192139317 conn(0x5586fe4faa30 msgr2=0x5586fe5079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f0589ffb640 0 client.16779 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 shutdown_connections 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 1 -- 172.21.15.64:0/192139317 wait complete. 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05aac37740 20 client.16779 trim_cache size 0 max 16384 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:29.947 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:11:29.948 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:11:29.948 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05900a33b0 msgr2=0x7f05900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05900a33b0 0x7f05900a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f058c0cc8d0 tx=0x7f058c01c000 comp rx=0 tx=0).stop 2024-09-19T05:11:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 msgr2=0x7f05a4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 0x7f05a4199760 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b4a0 tx=0x7f058c00b960 comp rx=0 tx=0).stop 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 shutdown_connections 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f05900a33b0 0x7f05900a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05a4106a80 0x7f05a41a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05a4101420 0x7f05a4199760 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 --2- 172.21.15.64:0/3316248401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05a4100a80 0x7f05a4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.939+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 >> 172.21.15.64:0/3316248401 conn(0x7f05a40fc7a0 msgr2=0x7f05a40fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.940+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 shutdown_connections 2024-09-19T05:11:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:29.940+0000 7f05a885e640 1 -- 172.21.15.64:0/3316248401 wait complete. 2024-09-19T05:11:30.590 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:11:30.590 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:11:30.847 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:31.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:30 smithi064 ceph-mon[30960]: pgmap v171: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:31.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:30 smithi103 ceph-mon[34446]: pgmap v171: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:31.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:30 smithi174 ceph-mon[34488]: pgmap v171: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:31.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.308+0000 7faf3fb58640 1 -- 172.21.15.64:0/1440173972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 msgr2=0x7faf3810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:31.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.308+0000 7faf3fb58640 1 --2- 172.21.15.64:0/1440173972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3810df30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7faf28009900 tx=0x7faf2802a410 comp rx=0 tx=0).stop 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 -- 172.21.15.64:0/1440173972 shutdown_connections 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 --2- 172.21.15.64:0/1440173972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf3810e500 0x7faf38110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 --2- 172.21.15.64:0/1440173972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3810df30 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 --2- 172.21.15.64:0/1440173972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf38100b20 0x7faf38100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 -- 172.21.15.64:0/1440173972 >> 172.21.15.64:0/1440173972 conn(0x7faf380fc7d0 msgr2=0x7faf380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 -- 172.21.15.64:0/1440173972 shutdown_connections 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 -- 172.21.15.64:0/1440173972 wait complete. 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 Processor -- start 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 -- start start 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf38100b20 0x7faf3819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.309+0000 7faf3fb58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3fb58640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf3810e500 0x7faf3819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3fb58640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf38114100 con 0x7faf38101440 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3fb58640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf38113f80 con 0x7faf38100b20 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3fb58640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faf38114280 con 0x7faf3810e500 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52778/0 (socket says 172.21.15.64:52778) 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 -- 172.21.15.64:0/2806957793 learned_addr learned my addr 172.21.15.64:0/2806957793 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3e0f5640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf3810e500 0x7faf3819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d8f4640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf38100b20 0x7faf3819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 -- 172.21.15.64:0/2806957793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf3810e500 msgr2=0x7faf3819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf3810e500 0x7faf3819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 -- 172.21.15.64:0/2806957793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf38100b20 msgr2=0x7faf3819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf38100b20 0x7faf3819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 -- 172.21.15.64:0/2806957793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3819f680 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3d0f3640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3819ac10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faf28004270 tx=0x7faf280042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2803f030 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3819f910 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.310+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf280044c0 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.311+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28035070 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.311+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf381a7250 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.312+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf38100f00 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.313+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7faf28042030 con 0x7faf38101440 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.313+0000 7faf2effd640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7faf240a33b0 0x7faf240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:31.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.314+0000 7faf3d8f4640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7faf240a33b0 0x7faf240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:31.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.314+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7faf280f0790 con 0x7faf38101440 2024-09-19T05:11:31.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.314+0000 7faf3d8f4640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7faf240a33b0 0x7faf240a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf34005c40 tx=0x7faf3400a780 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:31.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.317+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf280bd110 con 0x7faf38101440 2024-09-19T05:11:31.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.530+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf38109600 con 0x7faf38101440 2024-09-19T05:11:31.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.530+0000 7faf2effd640 1 -- 172.21.15.64:0/2806957793 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7faf280bca10 con 0x7faf38101440 2024-09-19T05:11:31.533 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:11:31.533 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:11:31.534 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7faf240a33b0 msgr2=0x7faf240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7faf240a33b0 0x7faf240a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf34005c40 tx=0x7faf3400a780 comp rx=0 tx=0).stop 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 msgr2=0x7faf3819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3819ac10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faf28004270 tx=0x7faf280042a0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 shutdown_connections 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7faf240a33b0 0x7faf240a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf3810e500 0x7faf3819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf38101440 0x7faf3819ac10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 --2- 172.21.15.64:0/2806957793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf38100b20 0x7faf3819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 >> 172.21.15.64:0/2806957793 conn(0x7faf380fc7d0 msgr2=0x7faf380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 shutdown_connections 2024-09-19T05:11:31.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:31.534+0000 7faf3fb58640 1 -- 172.21.15.64:0/2806957793 wait complete. 2024-09-19T05:11:32.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:31 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2806957793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:32.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:31 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2806957793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:32.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:11:32.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:31 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2806957793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:32.513 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 -- 172.21.15.64:0/2264943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c102ef0 msgr2=0x7fc02c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 --2- 172.21.15.64:0/2264943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c102ef0 0x7fc02c10da40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc024009ca0 tx=0x7fc02402a950 comp rx=0 tx=0).stop 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 -- 172.21.15.64:0/2264943741 shutdown_connections 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 --2- 172.21.15.64:0/2264943741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c102ef0 0x7fc02c10da40 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 --2- 172.21.15.64:0/2264943741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102570 0x7fc02c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 --2- 172.21.15.64:0/2264943741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc02c100e20 0x7fc02c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 -- 172.21.15.64:0/2264943741 >> 172.21.15.64:0/2264943741 conn(0x7fc02c0fc7d0 msgr2=0x7fc02c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 -- 172.21.15.64:0/2264943741 shutdown_connections 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.948+0000 7fc031c8a640 1 -- 172.21.15.64:0/2264943741 wait complete. 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 Processor -- start 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 -- start start 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 0x7fc02c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc02c102570 0x7fc02c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102ef0 0x7fc02c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c110e80 con 0x7fc02c100e20 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c110d00 con 0x7fc02c102ef0 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc031c8a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c111000 con 0x7fc02c102570 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 0x7fc02c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 0x7fc02c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52800/0 (socket says 172.21.15.64:52800) 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1988814894 learned_addr learned my addr 172.21.15.64:0/1988814894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02affd640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc02c102570 0x7fc02c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02bfff640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102ef0 0x7fc02c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1988814894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc02c102570 msgr2=0x7fc02c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc02c102570 0x7fc02c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1988814894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102ef0 msgr2=0x7fc02c19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102ef0 0x7fc02c19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1988814894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c19f510 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.949+0000 7fc02bfff640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102ef0 0x7fc02c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.950+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 0x7fc02c19a560 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00d770 tx=0x7fc01c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.950+0000 7fc028ff9640 1 -- 172.21.15.64:0/1988814894 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c017070 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.950+0000 7fc028ff9640 1 -- 172.21.15.64:0/1988814894 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc01c0054d0 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.950+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c19f800 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.950+0000 7fc028ff9640 1 -- 172.21.15.64:0/1988814894 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004cf0 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.950+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c1a70e0 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.952+0000 7fc028ff9640 1 -- 172.21.15.64:0/1988814894 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc01c004e90 con 0x7fc02c100e20 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.952+0000 7fc034063740 1 Processor -- start 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.952+0000 7fc034063740 1 -- start start 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc034063740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc034063740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc034063740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc034063740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557a4d01e2f0 con 0x557a4d0b85d0 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc034063740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a4cfc5cf0 con 0x557a4d0b9b40 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc034063740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a4d01a2b0 con 0x557a4d0c3370 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52802/0 (socket says 172.21.15.64:52802) 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1307423117 learned_addr learned my addr 172.21.15.64:0/1307423117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc028ff9640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc0140a33b0 0x7fc0140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02affd640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02bfff640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02bfff640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc0140a33b0 0x7fc0140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc028ff9640 1 -- 172.21.15.64:0/1988814894 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fc01c0cf8a0 con 0x7fc02c100e20 2024-09-19T05:11:32.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1307423117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 msgr2=0x557a4d0c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1307423117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 msgr2=0x557a4d0b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02b7fe640 1 -- 172.21.15.64:0/1307423117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4cfaa3a0 con 0x557a4d0b85d0 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02bfff640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.953+0000 7fc02affd640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0b8a10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0ccf00 tx=0x7fc01c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e95ce6ffff263fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc011ffb640 1 -- 172.21.15.64:0/1307423117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c020050 con 0x557a4d0b85d0 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc02bfff640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc0140a33b0 0x7fc0140a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc024006010 tx=0x7fc0240095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc011ffb640 1 -- 172.21.15.64:0/1307423117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc01c009c30 con 0x557a4d0b85d0 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc011ffb640 1 -- 172.21.15.64:0/1307423117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c0a4790 con 0x557a4d0b85d0 2024-09-19T05:11:32.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc034063740 1 -- 172.21.15.64:0/1307423117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 msgr2=0x557a4d0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.954+0000 7fc034063740 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0b8a10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0ccf00 tx=0x7fc01c005680 comp rx=0 tx=0).stop 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- 172.21.15.64:0/1307423117 shutdown_connections 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0b8a10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 --2- 172.21.15.64:0/1307423117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- 172.21.15.64:0/1307423117 >> 172.21.15.64:0/1307423117 conn(0x557a4d081310 msgr2=0x557a4d0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- 172.21.15.64:0/1307423117 shutdown_connections 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- 172.21.15.64:0/1307423117 wait complete. 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- start start 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 Processor -- start 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0c3370 0x557a4d0c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557a4d0c6010 con 0x557a4d0b85d0 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a4cfd9060 con 0x557a4d0c3370 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a4d0c6190 con 0x557a4d0b9b40 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.955+0000 7fc034063740 10 client.0 _ll_register_callbacks cb 0x557a4cd2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52812/0 (socket says 172.21.15.64:52812) 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 -- 172.21.15.64:0/2217035693 learned_addr learned my addr 172.21.15.64:0/2217035693 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02bfff640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0c3370 0x557a4d0c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02affd640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 -- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 msgr2=0x557a4d0c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 -- 172.21.15.64:0/2217035693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0c3370 msgr2=0x557a4d0c05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0c3370 0x557a4d0c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 -- 172.21.15.64:0/2217035693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4cd8f3c0 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02affd640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c2d50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0047e0 tx=0x7fc01c0983c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9514bd31b4d1aaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc00affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc00affd640 1 -- 172.21.15.64:0/2217035693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c0cd420 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc00affd640 1 -- 172.21.15.64:0/2217035693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fc01c0a4300 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a4d02b860 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc00affd640 1 -- 172.21.15.64:0/2217035693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc01c09c220 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc034063740 10 client.16797 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a4d0c6010 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.956+0000 7fc00affd640 1 -- 172.21.15.64:0/2217035693 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557a4d02b860 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc00affd640 1 -- 172.21.15.64:0/2217035693 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc01c09a9e0 con 0x557a4d0b85d0 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc034063740 10 client.16797 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:32.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc034063740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc034063740 20 client.16797 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc034063740 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d0d2c10 0x557a4d0d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc034063740 4 client.16797 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4d025d70 con 0x557a4d0d2c10 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc02affd640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d0d2c10 0x557a4d0d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc02affd640 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d0d2c10 0x557a4d0d5000 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5224cb6292b78cd4 server_cookie=62c373b010b42e6d in_seq=0 out_seq=0 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.957+0000 7fc00affd640 10 client.16797 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc00affd640 1 -- 172.21.15.64:0/2217035693 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a4d025d70 con 0x557a4d0d2c10 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc00affd640 10 client.16797 handle_command_reply: tid=0 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 client.16797 shutdown 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 msgr2=0x557a4d0c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c2d50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0047e0 tx=0x7fc01c0983c0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 shutdown_connections 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d0d2c10 0x557a4d0d5000 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0c3370 0x557a4d0c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc00affd640 0 client.16797 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 --2- 172.21.15.64:0/2217035693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c2d50 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 >> 172.21.15.64:0/2217035693 conn(0x557a4d081310 msgr2=0x557a4d0ba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 shutdown_connections 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.958+0000 7fc034063740 1 -- 172.21.15.64:0/2217035693 wait complete. 2024-09-19T05:11:32.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.959+0000 7fc034063740 20 client.16797 trim_cache size 0 max 16384 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 Processor -- start 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 -- start start 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1b21e0 con 0x557a4d0b85d0 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1afab0 con 0x557a4d0b9b40 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc034063740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1ad900 con 0x557a4d0c3370 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52816/0 (socket says 172.21.15.64:52816) 2024-09-19T05:11:32.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 -- 172.21.15.64:0/3373912746 learned_addr learned my addr 172.21.15.64:0/3373912746 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02bfff640 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02affd640 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 -- 172.21.15.64:0/3373912746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 msgr2=0x557a4d0bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 -- 172.21.15.64:0/3373912746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 msgr2=0x557a4d0bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0bb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.972+0000 7fc02b7fe640 1 -- 172.21.15.64:0/3373912746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4d0be550 con 0x557a4d0b85d0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.973+0000 7fc02affd640 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.973+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c1f70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc01c09b820 tx=0x7fc01c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38e5f85c20f07f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.973+0000 7fc0117fa640 1 -- 172.21.15.64:0/3373912746 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c020070 con 0x557a4d0b85d0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.973+0000 7fc0117fa640 1 -- 172.21.15.64:0/3373912746 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc01c09ce50 con 0x557a4d0b85d0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.973+0000 7fc0117fa640 1 -- 172.21.15.64:0/3373912746 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c097870 con 0x557a4d0b85d0 2024-09-19T05:11:32.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 -- 172.21.15.64:0/3373912746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 msgr2=0x557a4d0c1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c1f70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc01c09b820 tx=0x7fc01c005680 comp rx=0 tx=0).stop 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 -- 172.21.15.64:0/3373912746 shutdown_connections 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0c3370 0x557a4d0bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b9b40 0x557a4d0bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 --2- 172.21.15.64:0/3373912746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0b85d0 0x557a4d0c1f70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 -- 172.21.15.64:0/3373912746 >> 172.21.15.64:0/3373912746 conn(0x557a4d081310 msgr2=0x557a4d0c1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 -- 172.21.15.64:0/3373912746 shutdown_connections 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 -- 172.21.15.64:0/3373912746 wait complete. 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 -- start start 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.974+0000 7fc034063740 1 Processor -- start 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b85d0 0x557a4d0bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 0x557a4d0bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1adf70 con 0x557a4d0c3370 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1af3e0 con 0x557a4d0b85d0 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1ae640 con 0x557a4d0b9b40 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc034063740 10 client.0 _ll_register_callbacks cb 0x7fc02000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 0x557a4d0bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 0x557a4d0bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52822/0 (socket says 172.21.15.64:52822) 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 -- 172.21.15.64:0/2598502572 learned_addr learned my addr 172.21.15.64:0/2598502572 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02affd640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b85d0 0x557a4d0bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 -- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 msgr2=0x557a4d0bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 -- 172.21.15.64:0/2598502572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b85d0 msgr2=0x557a4d0bd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b85d0 0x557a4d0bd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 -- 172.21.15.64:0/2598502572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4d13c710 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc02bfff640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 0x557a4d0bdb00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc02402fb40 tx=0x7fc02402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d205e5fdba8c40e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc00affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.975+0000 7fc00affd640 1 -- 172.21.15.64:0/2598502572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024057060 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc00affd640 1 -- 172.21.15.64:0/2598502572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fc024004310 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01c09ce50 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc00affd640 1 -- 172.21.15.64:0/2598502572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc024042070 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 10 client.16803 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a4d13d9f0 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc00affd640 1 -- 172.21.15.64:0/2598502572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc01c09ce50 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc00affd640 1 -- 172.21.15.64:0/2598502572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc024030b90 con 0x557a4d0c3370 2024-09-19T05:11:32.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 10 client.16803 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 20 client.16803 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d13e170 0x557a4d140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 4 client.16803 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a4cfa2170 con 0x557a4d13e170 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.976+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d13e170 0x557a4d140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc00affd640 10 client.16803 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc02b7fe640 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d13e170 0x557a4d140590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a94c0accd999c85 server_cookie=f8a2cfdaaf70f77b in_seq=0 out_seq=0 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc00affd640 1 -- 172.21.15.64:0/2598502572 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a4cfa2170 con 0x557a4d13e170 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc00affd640 10 client.16803 handle_command_reply: tid=0 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 client.16803 shutdown 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 msgr2=0x557a4d0bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 0x557a4d0bdb00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc02402fb40 tx=0x7fc02402a950 comp rx=0 tx=0).stop 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 shutdown_connections 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x557a4d13e170 0x557a4d140590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a4d0c3370 0x557a4d0bdb00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a4d0b9b40 0x557a4d0bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc00affd640 0 client.16803 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 --2- 172.21.15.64:0/2598502572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a4d0b85d0 0x557a4d0bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 >> 172.21.15.64:0/2598502572 conn(0x557a4d081310 msgr2=0x557a4d0c2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 shutdown_connections 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 1 -- 172.21.15.64:0/2598502572 wait complete. 2024-09-19T05:11:32.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.977+0000 7fc034063740 20 client.16803 trim_cache size 0 max 16384 2024-09-19T05:11:32.993 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:32.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc0140a33b0 msgr2=0x7fc0140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc0140a33b0 0x7fc0140a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc024006010 tx=0x7fc0240095c0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 msgr2=0x7fc02c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:32.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 0x7fc02c19a560 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00d770 tx=0x7fc01c00dc30 comp rx=0 tx=0).stop 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 shutdown_connections 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc0140a33b0 0x7fc0140a5870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc02c102ef0 0x7fc02c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc02c102570 0x7fc02c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 --2- 172.21.15.64:0/1988814894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc02c100e20 0x7fc02c19a560 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 >> 172.21.15.64:0/1988814894 conn(0x7fc02c0fc7d0 msgr2=0x7fc02c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 shutdown_connections 2024-09-19T05:11:32.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:32.978+0000 7fc031c8a640 1 -- 172.21.15.64:0/1988814894 wait complete. 2024-09-19T05:11:33.135 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:32 smithi103 ceph-mon[34446]: pgmap v172: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:33.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:32 smithi064 ceph-mon[30960]: pgmap v172: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:33.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:32 smithi174 ceph-mon[34488]: pgmap v172: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:33.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:11:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.352+0000 7f587ee28640 1 -- 172.21.15.64:0/583665947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878102ef0 msgr2=0x7f587810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.352+0000 7f587ee28640 1 --2- 172.21.15.64:0/583665947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878102ef0 0x7f587810da40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802a950 comp rx=0 tx=0).stop 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 -- 172.21.15.64:0/583665947 shutdown_connections 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 --2- 172.21.15.64:0/583665947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878102ef0 0x7f587810da40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 --2- 172.21.15.64:0/583665947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5878102570 0x7f58781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 --2- 172.21.15.64:0/583665947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5878100e20 0x7f5878101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 -- 172.21.15.64:0/583665947 >> 172.21.15.64:0/583665947 conn(0x7f58780fc7d0 msgr2=0x7f58780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 -- 172.21.15.64:0/583665947 shutdown_connections 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 -- 172.21.15.64:0/583665947 wait complete. 2024-09-19T05:11:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 Processor -- start 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 -- start start 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 0x7f58781983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.353+0000 7f587ee28640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5878102570 0x7f5878198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587ee28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5878102ef0 0x7f587819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587ee28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5878110e30 con 0x7f5878100e20 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587ee28640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5878110cb0 con 0x7f5878102ef0 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587ee28640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5878110fb0 con 0x7f5878102570 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 0x7f58781983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 0x7f58781983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50824/0 (socket says 172.21.15.64:50824) 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 -- 172.21.15.64:0/3062297825 learned_addr learned my addr 172.21.15.64:0/3062297825 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587d3c5640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5878102ef0 0x7f587819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f5877fff640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5878102570 0x7f5878198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 -- 172.21.15.64:0/3062297825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5878102570 msgr2=0x7f5878198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5878102570 0x7f5878198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 -- 172.21.15.64:0/3062297825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5878102ef0 msgr2=0x7f587819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5878102ef0 0x7f587819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 -- 172.21.15.64:0/3062297825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587819d3a0 con 0x7f5878100e20 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587cbc4640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 0x7f58781983f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f587000d950 tx=0x7f587000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f5875ffb640 1 -- 172.21.15.64:0/3062297825 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870017030 con 0x7f5878100e20 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f5875ffb640 1 -- 172.21.15.64:0/3062297825 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5870004020 con 0x7f5878100e20 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587819d690 con 0x7f5878100e20 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.354+0000 7f5875ffb640 1 -- 172.21.15.64:0/3062297825 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700047a0 con 0x7f5878100e20 2024-09-19T05:11:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.355+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781a4f70 con 0x7f5878100e20 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.357+0000 7f5875ffb640 1 -- 172.21.15.64:0/3062297825 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f58700041d0 con 0x7f5878100e20 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.357+0000 7f5881201740 1 Processor -- start 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.357+0000 7f5881201740 1 -- start start 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5881201740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5881201740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d8ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5881201740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5881201740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6f8cf42f0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5881201740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6f8c9bcf0 con 0x55c6f8d8fb40 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5881201740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6f8cf02b0 con 0x55c6f8d99370 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5875ffb640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f58640a3430 0x7f58640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d8ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d8ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50832/0 (socket says 172.21.15.64:50832) 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 -- 172.21.15.64:0/140912979 learned_addr learned my addr 172.21.15.64:0/140912979 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5877fff640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587d3c5640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f58640a3430 0x7f58640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587d3c5640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5875ffb640 1 -- 172.21.15.64:0/3062297825 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f58700cf770 con 0x7f5878100e20 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 -- 172.21.15.64:0/140912979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 msgr2=0x55c6f8d9b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 -- 172.21.15.64:0/140912979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 msgr2=0x55c6f8d8e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d8e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587cbc4640 1 -- 172.21.15.64:0/140912979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f8c803a0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f5877fff640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.358+0000 7f587d3c5640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f587cbc4640 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d8ea10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5870017000 tx=0x7f5870021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8ed89c7a2363be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f587d3c5640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f58640a3430 0x7f58640a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5868005e30 tx=0x7f5868009800 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f586effd640 1 -- 172.21.15.64:0/140912979 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587009c0f0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f586effd640 1 -- 172.21.15.64:0/140912979 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f587009fda0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f586effd640 1 -- 172.21.15.64:0/140912979 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700cd0e0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 -- 172.21.15.64:0/140912979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 msgr2=0x55c6f8d8ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d8ea10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5870017000 tx=0x7f5870021000 comp rx=0 tx=0).stop 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 -- 172.21.15.64:0/140912979 shutdown_connections 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d8ea10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 --2- 172.21.15.64:0/140912979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.359+0000 7f5881201740 1 -- 172.21.15.64:0/140912979 >> 172.21.15.64:0/140912979 conn(0x55c6f8d57310 msgr2=0x55c6f8d8ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 -- 172.21.15.64:0/140912979 shutdown_connections 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 -- 172.21.15.64:0/140912979 wait complete. 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 -- start start 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 Processor -- start 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8d96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6f8d9bfb0 con 0x55c6f8d99370 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6f8caf060 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6f8d9c130 con 0x55c6f8d8fb40 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f5881201740 10 client.0 _ll_register_callbacks cb 0x55c6f8a028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.360+0000 7f587d3c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8d96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8d96860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50846/0 (socket says 172.21.15.64:50846) 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 -- 172.21.15.64:0/211117089 learned_addr learned my addr 172.21.15.64:0/211117089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587cbc4640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5877fff640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 -- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 msgr2=0x55c6f8d960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 -- 172.21.15.64:0/211117089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 msgr2=0x55c6f8d98e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 -- 172.21.15.64:0/211117089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f8a653c0 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587cbc4640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f587d3c5640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8d96860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f58680098c0 tx=0x7f586802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845061a261347217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5857fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5857fff640 1 -- 172.21.15.64:0/211117089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868034040 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5857fff640 1 -- 172.21.15.64:0/211117089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5868004250 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587009fda0 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5857fff640 1 -- 172.21.15.64:0/211117089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5868004450 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5881201740 10 client.16815 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.361+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6f8da7b20 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5857fff640 1 -- 172.21.15.64:0/211117089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f587009fda0 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5857fff640 1 -- 172.21.15.64:0/211117089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5868048040 con 0x55c6f8d99370 2024-09-19T05:11:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5881201740 10 client.16815 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5881201740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5881201740 20 client.16815 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5881201740 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8da8fe0 0x55c6f8dab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5881201740 4 client.16815 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f8d01860 con 0x55c6f8da8fe0 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.362+0000 7f587cbc4640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8da8fe0 0x55c6f8dab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f587cbc4640 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8da8fe0 0x55c6f8dab440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e1025cb5404eff server_cookie=7fb060cd2a86ef20 in_seq=0 out_seq=0 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5857fff640 10 client.16815 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5857fff640 1 -- 172.21.15.64:0/211117089 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6f8d01860 con 0x55c6f8da8fe0 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5857fff640 10 client.16815 handle_command_reply: tid=0 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 client.16815 shutdown 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 msgr2=0x55c6f8d96860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8d96860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f58680098c0 tx=0x7f586802a950 comp rx=0 tx=0).stop 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 shutdown_connections 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8da8fe0 0x55c6f8dab440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8d96860 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 --2- 172.21.15.64:0/211117089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.363+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 >> 172.21.15.64:0/211117089 conn(0x55c6f8d57310 msgr2=0x55c6f8d90990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.364+0000 7f5857fff640 0 client.16815 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.364+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 shutdown_connections 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.364+0000 7f5881201740 1 -- 172.21.15.64:0/211117089 wait complete. 2024-09-19T05:11:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.364+0000 7f5881201740 20 client.16815 trim_cache size 0 max 16384 2024-09-19T05:11:34.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 Processor -- start 2024-09-19T05:11:34.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 -- start start 2024-09-19T05:11:34.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8db0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8db1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58781b0080 con 0x55c6f8d99370 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58781ad950 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.377+0000 7f5881201740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58781ab7a0 con 0x55c6f8d8fb40 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8db1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8db1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50852/0 (socket says 172.21.15.64:50852) 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 -- 172.21.15.64:0/2693534206 learned_addr learned my addr 172.21.15.64:0/2693534206 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587cbc4640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 -- 172.21.15.64:0/2693534206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 msgr2=0x55c6f8db0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f5877fff640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8db0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8db0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 -- 172.21.15.64:0/2693534206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 msgr2=0x55c6f8d98040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 -- 172.21.15.64:0/2693534206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f8d8f750 con 0x55c6f8d99370 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f5877fff640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8db0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587cbc4640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f587d3c5640 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8db1250 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5868004010 tx=0x7f5868005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cce391c7aed9bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f586e7fc640 1 -- 172.21.15.64:0/2693534206 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680098f0 con 0x55c6f8d99370 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f586e7fc640 1 -- 172.21.15.64:0/2693534206 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f586802f980 con 0x55c6f8d99370 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.378+0000 7f586e7fc640 1 -- 172.21.15.64:0/2693534206 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586805b480 con 0x55c6f8d99370 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 -- 172.21.15.64:0/2693534206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 msgr2=0x55c6f8db1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8db1250 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5868004010 tx=0x7f5868005fb0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 -- 172.21.15.64:0/2693534206 shutdown_connections 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d99370 0x55c6f8db1250 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d8fb40 0x55c6f8db0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 --2- 172.21.15.64:0/2693534206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8d98040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 -- 172.21.15.64:0/2693534206 >> 172.21.15.64:0/2693534206 conn(0x55c6f8d57310 msgr2=0x55c6f8d97930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 -- 172.21.15.64:0/2693534206 shutdown_connections 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 -- 172.21.15.64:0/2693534206 wait complete. 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.379+0000 7f5881201740 1 -- start start 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 Processor -- start 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8da1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d9cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58781abe10 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58781ad280 con 0x55c6f8d8fb40 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58781ac4e0 con 0x55c6f8d99370 2024-09-19T05:11:34.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5881201740 10 client.0 _ll_register_callbacks cb 0x7f586802f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8da1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8da1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50868/0 (socket says 172.21.15.64:50868) 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 -- 172.21.15.64:0/2269379882 learned_addr learned my addr 172.21.15.64:0/2269379882 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f5877fff640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d9cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587d3c5640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 -- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 msgr2=0x55c6f8d9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 -- 172.21.15.64:0/2269379882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 msgr2=0x55c6f8d9cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d9cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.380+0000 7f587cbc4640 1 -- 172.21.15.64:0/2269379882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f8e12ce0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f587cbc4640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8da1c80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f587009b980 tx=0x7f5870097c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a1bd5c5aa8800f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5857fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:34.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5857fff640 1 -- 172.21.15.64:0/2269379882 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700b9060 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5857fff640 1 -- 172.21.15.64:0/2269379882 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f58700860b0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586802f980 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5857fff640 1 -- 172.21.15.64:0/2269379882 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f587009b7c0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 10 client.16821 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6f8e13dd0 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5857fff640 1 -- 172.21.15.64:0/2269379882 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f586802f980 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5857fff640 1 -- 172.21.15.64:0/2269379882 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f58700ba250 con 0x55c6f8d8e5d0 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 10 client.16821 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 20 client.16821 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8e140c0 0x55c6f8e164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 4 client.16821 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:34.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.381+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6f8cfbd70 con 0x55c6f8e140c0 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5877fff640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8e140c0 0x55c6f8e164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5857fff640 10 client.16821 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5877fff640 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8e140c0 0x55c6f8e164b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2327e6b794993ee7 server_cookie=d2b9537dbf4f0510 in_seq=0 out_seq=0 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5857fff640 1 -- 172.21.15.64:0/2269379882 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6f8cfbd70 con 0x55c6f8e140c0 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5857fff640 10 client.16821 handle_command_reply: tid=0 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 client.16821 shutdown 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 msgr2=0x55c6f8da1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8da1c80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f587009b980 tx=0x7f5870097c90 comp rx=0 tx=0).stop 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 shutdown_connections 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6f8d99370 0x55c6f8d9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6f8d8fb40 0x55c6f8d9cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.382+0000 7f5881201740 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c6f8e140c0 0x55c6f8e164b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f5881201740 1 --2- 172.21.15.64:0/2269379882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6f8d8e5d0 0x55c6f8da1c80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f5857fff640 0 client.16821 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:34.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 >> 172.21.15.64:0/2269379882 conn(0x55c6f8d57310 msgr2=0x55c6f8d98150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:34.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 shutdown_connections 2024-09-19T05:11:34.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f5881201740 1 -- 172.21.15.64:0/2269379882 wait complete. 2024-09-19T05:11:34.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f5881201740 20 client.16821 trim_cache size 0 max 16384 2024-09-19T05:11:34.393 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f58640a3430 msgr2=0x7f58640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f58640a3430 0x7f58640a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5868005e30 tx=0x7f5868009800 comp rx=0 tx=0).stop 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 msgr2=0x7f58781983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 0x7f58781983f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f587000d950 tx=0x7f587000de10 comp rx=0 tx=0).stop 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 shutdown_connections 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f58640a3430 0x7f58640a58f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5878102ef0 0x7f587819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5878102570 0x7f5878198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 --2- 172.21.15.64:0/3062297825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5878100e20 0x7f58781983f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:34.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.383+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 >> 172.21.15.64:0/3062297825 conn(0x7f58780fc7d0 msgr2=0x7f5878069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:34.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.384+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 shutdown_connections 2024-09-19T05:11:34.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:34.384+0000 7f587ee28640 1 -- 172.21.15.64:0/3062297825 wait complete. 2024-09-19T05:11:35.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:11:35.144 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:34 smithi064 ceph-mon[30960]: pgmap v173: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:35.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:34 smithi103 ceph-mon[34446]: pgmap v173: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:35.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:34 smithi174 ceph-mon[34488]: pgmap v173: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:35.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 -- 172.21.15.64:0/1042762157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 msgr2=0x7f951410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 --2- 172.21.15.64:0/1042762157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f951410bbb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9504009ca0 tx=0x7f950402a950 comp rx=0 tx=0).stop 2024-09-19T05:11:35.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 -- 172.21.15.64:0/1042762157 shutdown_connections 2024-09-19T05:11:35.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 --2- 172.21.15.64:0/1042762157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951410c180 0x7f951410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 --2- 172.21.15.64:0/1042762157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f951410bbb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 --2- 172.21.15.64:0/1042762157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514102da0 0x7f9514103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 -- 172.21.15.64:0/1042762157 >> 172.21.15.64:0/1042762157 conn(0x7f95140fc7a0 msgr2=0x7f95140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:35.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 -- 172.21.15.64:0/1042762157 shutdown_connections 2024-09-19T05:11:35.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.820+0000 7f951ac87640 1 -- 172.21.15.64:0/1042762157 wait complete. 2024-09-19T05:11:35.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 Processor -- start 2024-09-19T05:11:35.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 -- start start 2024-09-19T05:11:35.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514102da0 0x7f95141a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f95141a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951410c180 0x7f951410ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111e50 con 0x7f95141036c0 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111cd0 con 0x7f951410c180 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f951ac87640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111fd0 con 0x7f9514102da0 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f95141a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f95141a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50896/0 (socket says 172.21.15.64:50896) 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 -- 172.21.15.64:0/218103119 learned_addr learned my addr 172.21.15.64:0/218103119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9518a23640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514102da0 0x7f95141a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9519224640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951410c180 0x7f951410ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 -- 172.21.15.64:0/218103119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514102da0 msgr2=0x7f95141a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514102da0 0x7f95141a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 -- 172.21.15.64:0/218103119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951410c180 msgr2=0x7f951410ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951410c180 0x7f951410ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.821+0000 7f9513fff640 1 -- 172.21.15.64:0/218103119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951410b560 con 0x7f95141036c0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.822+0000 7f9513fff640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f95141a4bf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9504005cc0 tx=0x7f950402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.822+0000 7f9511ffb640 1 -- 172.21.15.64:0/218103119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504034040 con 0x7f95141036c0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.822+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951410b7f0 con 0x7f95141036c0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.822+0000 7f9511ffb640 1 -- 172.21.15.64:0/218103119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95040308f0 con 0x7f95141036c0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.822+0000 7f9511ffb640 1 -- 172.21.15.64:0/218103119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040046b0 con 0x7f95141036c0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.822+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141ab580 con 0x7f95141036c0 2024-09-19T05:11:35.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.824+0000 7f9511ffb640 1 -- 172.21.15.64:0/218103119 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9504037020 con 0x7f95141036c0 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 Processor -- start 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 -- start start 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 0x557da2bff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2bffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c0a370 0x557da2c0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557da2b652f0 con 0x557da2c00b40 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557da2b0ccf0 con 0x557da2bff5d0 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f951d060740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557da2b612b0 con 0x557da2c0a370 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f9519224640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 0x557da2bff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f9519224640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 0x557da2bff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50912/0 (socket says 172.21.15.64:50912) 2024-09-19T05:11:35.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f9519224640 1 -- 172.21.15.64:0/2372451022 learned_addr learned my addr 172.21.15.64:0/2372451022 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f9518a23640 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2bffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f9513fff640 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c0a370 0x557da2c0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.825+0000 7f9511ffb640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f95000a3430 0x7f95000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 -- 172.21.15.64:0/2372451022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c0a370 msgr2=0x557da2c0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c0a370 0x557da2c0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 -- 172.21.15.64:0/2372451022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 msgr2=0x557da2bffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2bffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 -- 172.21.15.64:0/2372451022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557da2af13a0 con 0x557da2c00b40 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9511ffb640 1 -- 172.21.15.64:0/218103119 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f95040f0ad0 con 0x7f95141036c0 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f95000a3430 0x7f95000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 0x557da2bff090 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b880 tx=0x7f950c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59622e7c942c87c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f950affd640 1 -- 172.21.15.64:0/2372451022 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c024070 con 0x557da2c00b40 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f950affd640 1 -- 172.21.15.64:0/2372451022 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f950c0047a0 con 0x557da2c00b40 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f950affd640 1 -- 172.21.15.64:0/2372451022 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c0116a0 con 0x557da2c00b40 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.826+0000 7f9519224640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f95000a3430 0x7f95000a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f950c021000 tx=0x7f950c01e000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 -- 172.21.15.64:0/2372451022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 msgr2=0x557da2bff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 0x557da2bff090 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b880 tx=0x7f950c00bd40 comp rx=0 tx=0).stop 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 -- 172.21.15.64:0/2372451022 shutdown_connections 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c0a370 0x557da2c0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2bffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 --2- 172.21.15.64:0/2372451022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c00b40 0x557da2bff090 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 -- 172.21.15.64:0/2372451022 >> 172.21.15.64:0/2372451022 conn(0x557da2bc8310 msgr2=0x557da2bfbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:35.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 -- 172.21.15.64:0/2372451022 shutdown_connections 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 -- 172.21.15.64:0/2372451022 wait complete. 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 -- start start 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.827+0000 7f951d060740 1 Processor -- start 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 0x557da2c09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c0a370 0x557da2c077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557da2c0d070 con 0x557da2bff5d0 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557da2b20060 con 0x557da2c0a370 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557da2c0d1f0 con 0x557da2c00b40 2024-09-19T05:11:35.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f951d060740 10 client.0 _ll_register_callbacks cb 0x557da28738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 0x557da2c09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 0x557da2c09de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50928/0 (socket says 172.21.15.64:50928) 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 -- 172.21.15.64:0/626574221 learned_addr learned my addr 172.21.15.64:0/626574221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9513fff640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9519224640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c0a370 0x557da2c077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 -- 172.21.15.64:0/626574221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 msgr2=0x557da2c07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c07090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 -- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c0a370 msgr2=0x557da2c077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c0a370 0x557da2c077d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9518a23640 1 -- 172.21.15.64:0/626574221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557da28d63c0 con 0x557da2bff5d0 2024-09-19T05:11:35.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.828+0000 7f9513fff640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f9518a23640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 0x557da2c09de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f94fc011540 tx=0x7f94fc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11e7609293872a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f94f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f94f3fff640 1 -- 172.21.15.64:0/626574221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00f6d0 con 0x557da2bff5d0 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f94f3fff640 1 -- 172.21.15.64:0/626574221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f94fc00f870 con 0x557da2bff5d0 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f950c0047a0 con 0x557da2bff5d0 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f94f3fff640 1 -- 172.21.15.64:0/626574221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94fc00fab0 con 0x557da2bff5d0 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 10 client.16833 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:35.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557da2c18c00 con 0x557da2bff5d0 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f94f3fff640 1 -- 172.21.15.64:0/626574221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f950c0047a0 con 0x557da2bff5d0 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f94f3fff640 1 -- 172.21.15.64:0/626574221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f94fc01d400 con 0x557da2bff5d0 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 20 client.16833 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c1a0c0 0x557da2c1c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 4 client.16833 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.829+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557da2b72860 con 0x557da2c1a0c0 2024-09-19T05:11:35.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.830+0000 7f9513fff640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c1a0c0 0x557da2c1c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.830+0000 7f9513fff640 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c1a0c0 0x557da2c1c510 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aefde3fa81586a server_cookie=9d6c9af394b2138e in_seq=0 out_seq=0 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.830+0000 7f94f3fff640 10 client.16833 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f94f3fff640 1 -- 172.21.15.64:0/626574221 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557da2b72860 con 0x557da2c1a0c0 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f94f3fff640 10 client.16833 handle_command_reply: tid=0 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 client.16833 shutdown 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 msgr2=0x557da2c09de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 0x557da2c09de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f94fc011540 tx=0x7f94fc011a00 comp rx=0 tx=0).stop 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 shutdown_connections 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c0a370 0x557da2c077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c1a0c0 0x557da2c1c510 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 --2- 172.21.15.64:0/626574221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2bff5d0 0x557da2c09de0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f94f3fff640 0 client.16833 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 >> 172.21.15.64:0/626574221 conn(0x557da2bc8310 msgr2=0x557da2c01960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 shutdown_connections 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 1 -- 172.21.15.64:0/626574221 wait complete. 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.831+0000 7f951d060740 20 client.16833 trim_cache size 0 max 16384 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.844+0000 7f951d060740 1 Processor -- start 2024-09-19T05:11:35.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.844+0000 7f951d060740 1 -- start start 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f951d060740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2bff5d0 0x557da2c093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f951d060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c00b40 0x557da2c02120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f951d060740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c02660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f951d060740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f95141b6700 con 0x557da2c0a370 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f951d060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95141b3fd0 con 0x557da2c00b40 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f951d060740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95141b1e20 con 0x557da2bff5d0 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c02660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c02660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50930/0 (socket says 172.21.15.64:50930) 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 -- 172.21.15.64:0/676323712 learned_addr learned my addr 172.21.15.64:0/676323712 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:35.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9513fff640 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c00b40 0x557da2c02120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 -- 172.21.15.64:0/676323712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2bff5d0 msgr2=0x557da2c093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9518a23640 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2bff5d0 0x557da2c093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2bff5d0 0x557da2c093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 -- 172.21.15.64:0/676323712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c00b40 msgr2=0x557da2c02120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c00b40 0x557da2c02120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 -- 172.21.15.64:0/676323712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557da2bfc950 con 0x557da2c0a370 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9513fff640 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c00b40 0x557da2c02120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f9519224640 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c02660 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f950c003d20 tx=0x7f950c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1557c61295bb1064 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:35.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.845+0000 7f950a7fc640 1 -- 172.21.15.64:0/676323712 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c024040 con 0x557da2c0a370 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f950a7fc640 1 -- 172.21.15.64:0/676323712 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f950c011df0 con 0x557da2c0a370 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f950a7fc640 1 -- 172.21.15.64:0/676323712 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c0052a0 con 0x557da2c0a370 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 -- 172.21.15.64:0/676323712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 msgr2=0x557da2c02660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c02660 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f950c003d20 tx=0x7f950c003d50 comp rx=0 tx=0).stop 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 -- 172.21.15.64:0/676323712 shutdown_connections 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c02660 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2c00b40 0x557da2c02120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 --2- 172.21.15.64:0/676323712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2bff5d0 0x557da2c093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 -- 172.21.15.64:0/676323712 >> 172.21.15.64:0/676323712 conn(0x557da2bc8310 msgr2=0x557da2bfad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:35.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 -- 172.21.15.64:0/676323712 shutdown_connections 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.846+0000 7f951d060740 1 -- 172.21.15.64:0/676323712 wait complete. 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 -- start start 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 Processor -- start 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2c0e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c0e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c0ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f95141b2490 con 0x557da2c0a370 2024-09-19T05:11:35.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95141b3900 con 0x557da2bff5d0 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95141b2b60 con 0x557da2c00b40 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f951d060740 10 client.0 _ll_register_callbacks cb 0x7f94fc009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.847+0000 7f9519224640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c0ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2c0e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9519224640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c0ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50938/0 (socket says 172.21.15.64:50938) 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9519224640 1 -- 172.21.15.64:0/1627465416 learned_addr learned my addr 172.21.15.64:0/1627465416 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9513fff640 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c0e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 -- 172.21.15.64:0/1627465416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 msgr2=0x557da2c0e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c0e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 -- 172.21.15.64:0/1627465416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 msgr2=0x557da2c0ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:35.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c0ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 -- 172.21.15.64:0/1627465416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557da2c84000 con 0x557da2bff5d0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f9518a23640 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2c0e410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94fc011540 tx=0x7f94fc00f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf64c5b9e4584ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f94f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f94f3fff640 1 -- 172.21.15.64:0/1627465416 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc02a070 con 0x557da2bff5d0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f94f3fff640 1 -- 172.21.15.64:0/1627465416 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f94fc00fc50 con 0x557da2bff5d0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.848+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f950c011df0 con 0x557da2bff5d0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f94f3fff640 1 -- 172.21.15.64:0/1627465416 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94fc01de60 con 0x557da2bff5d0 2024-09-19T05:11:35.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 10 client.26366 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557da2c852c0 con 0x557da2bff5d0 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f94f3fff640 1 -- 172.21.15.64:0/1627465416 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f950c011df0 con 0x557da2bff5d0 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f94f3fff640 1 -- 172.21.15.64:0/1627465416 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f94fc014c40 con 0x557da2bff5d0 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 10 client.26366 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 20 client.26366 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c859b0 0x557da2c87dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 4 client.26366 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557da2b6cd70 con 0x557da2c859b0 2024-09-19T05:11:35.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.849+0000 7f9513fff640 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c859b0 0x557da2c87dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f9513fff640 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c859b0 0x557da2c87dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7cf96319be7193 server_cookie=b70d7fdad256cf43 in_seq=0 out_seq=0 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f94f3fff640 10 client.26366 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f94f3fff640 1 -- 172.21.15.64:0/1627465416 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557da2b6cd70 con 0x557da2c859b0 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f94f3fff640 10 client.26366 handle_command_reply: tid=0 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 client.26366 shutdown 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 msgr2=0x557da2c0e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2c0e410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94fc011540 tx=0x7f94fc00f270 comp rx=0 tx=0).stop 2024-09-19T05:11:35.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 shutdown_connections 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x557da2c859b0 0x557da2c87dd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557da2c0a370 0x557da2c0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557da2c00b40 0x557da2c0e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 --2- 172.21.15.64:0/1627465416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557da2bff5d0 0x557da2c0e410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 >> 172.21.15.64:0/1627465416 conn(0x557da2bc8310 msgr2=0x557da2bfa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f94f3fff640 0 client.26366 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 shutdown_connections 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 1 -- 172.21.15.64:0/1627465416 wait complete. 2024-09-19T05:11:35.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.850+0000 7f951d060740 20 client.26366 trim_cache size 0 max 16384 2024-09-19T05:11:35.875 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f95000a3430 msgr2=0x7f95000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f95000a3430 0x7f95000a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f950c021000 tx=0x7f950c01e000 comp rx=0 tx=0).stop 2024-09-19T05:11:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 msgr2=0x7f95141a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f95141a4bf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9504005cc0 tx=0x7f950402f650 comp rx=0 tx=0).stop 2024-09-19T05:11:35.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 shutdown_connections 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f95000a3430 0x7f95000a58f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951410c180 0x7f951410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95141036c0 0x7f95141a4bf0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 --2- 172.21.15.64:0/218103119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9514102da0 0x7f95141a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 >> 172.21.15.64:0/218103119 conn(0x7f95140fc7a0 msgr2=0x7f9514101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 shutdown_connections 2024-09-19T05:11:35.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:35.851+0000 7f951ac87640 1 -- 172.21.15.64:0/218103119 wait complete. 2024-09-19T05:11:37.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:36 smithi064 ceph-mon[30960]: pgmap v174: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:37.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:36 smithi103 ceph-mon[34446]: pgmap v174: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:37.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:36 smithi174 ceph-mon[34488]: pgmap v174: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:37.647 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:11:37.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:11:37.920 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:38.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 -- 172.21.15.64:0/274075842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 msgr2=0x7f8600106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 --2- 172.21.15.64:0/274075842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f8600106e60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009930 tx=0x7f85f002a420 comp rx=0 tx=0).stop 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 -- 172.21.15.64:0/274075842 shutdown_connections 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 --2- 172.21.15.64:0/274075842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600101420 0x7f860010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 --2- 172.21.15.64:0/274075842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8600100a80 0x7f8600100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 --2- 172.21.15.64:0/274075842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f8600106e60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 -- 172.21.15.64:0/274075842 >> 172.21.15.64:0/274075842 conn(0x7f86000fc7a0 msgr2=0x7f86000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 -- 172.21.15.64:0/274075842 shutdown_connections 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.371+0000 7f8606692640 1 -- 172.21.15.64:0/274075842 wait complete. 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 Processor -- start 2024-09-19T05:11:38.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 -- start start 2024-09-19T05:11:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600100a80 0x7f8600198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8600101420 0x7f8600198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f860019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8600110d50 con 0x7f8600106a80 2024-09-19T05:11:38.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8600110bd0 con 0x7f8600101420 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8606692640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8600110ed0 con 0x7f8600100a80 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8604c2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f860019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8604c2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f860019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50964/0 (socket says 172.21.15.64:50964) 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f8604c2f640 1 -- 172.21.15.64:0/214398589 learned_addr learned my addr 172.21.15.64:0/214398589 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.372+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8600101420 0x7f8600198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f85fffff640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600100a80 0x7f8600198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8604c2f640 1 -- 172.21.15.64:0/214398589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600100a80 msgr2=0x7f8600198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8604c2f640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600100a80 0x7f8600198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8604c2f640 1 -- 172.21.15.64:0/214398589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8600101420 msgr2=0x7f8600198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8604c2f640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8600101420 0x7f8600198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8604c2f640 1 -- 172.21.15.64:0/214398589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860019d2e0 con 0x7f8600106a80 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f85fffff640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600100a80 0x7f8600198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8604c2f640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f860019cc00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b970 tx=0x7f85f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f85fd7fa640 1 -- 172.21.15.64:0/214398589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8019070 con 0x7f8600106a80 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f85fd7fa640 1 -- 172.21.15.64:0/214398589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f8014440 con 0x7f8600106a80 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860019d5d0 con 0x7f8600106a80 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f85fd7fa640 1 -- 172.21.15.64:0/214398589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80075d0 con 0x7f8600106a80 2024-09-19T05:11:38.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.373+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001a4eb0 con 0x7f8600106a80 2024-09-19T05:11:38.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f85fd7fa640 1 -- 172.21.15.64:0/214398589 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f85f801c070 con 0x7f8600106a80 2024-09-19T05:11:38.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 Processor -- start 2024-09-19T05:11:38.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 -- start start 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84ac360 0x5637e84bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84bb150 0x5637e84bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f85fd7fa640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f85e80a3430 0x7f85e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5637e8411280 con 0x5637e84aba40 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5637e83b8b10 con 0x5637e84ac360 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f8608a6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637e840d240 con 0x5637e84bb150 2024-09-19T05:11:38.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.376+0000 7f85ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84abe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50968/0 (socket says 172.21.15.64:50968) 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 -- 172.21.15.64:0/1829332614 learned_addr learned my addr 172.21.15.64:0/1829332614 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85fffff640 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84ac360 0x5637e84bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f85e80a3430 0x7f85e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85fd7fa640 1 -- 172.21.15.64:0/214398589 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f85f80cf180 con 0x7f8600106a80 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f8604c2f640 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84bb150 0x5637e84bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 -- 172.21.15.64:0/1829332614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84bb150 msgr2=0x5637e84bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84bb150 0x5637e84bd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 -- 172.21.15.64:0/1829332614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84ac360 msgr2=0x5637e84bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84ac360 0x5637e84bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 -- 172.21.15.64:0/1829332614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e83c51f0 con 0x5637e84aba40 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84abe20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f85f4011ab0 tx=0x7f85f4011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8cd2a2b07f5f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f85e80a3430 0x7f85e80a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85f40058c0 tx=0x7f85f401c000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85e67fc640 1 -- 172.21.15.64:0/1829332614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4011050 con 0x5637e84aba40 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85e67fc640 1 -- 172.21.15.64:0/1829332614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f400aa50 con 0x5637e84aba40 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.377+0000 7f85e67fc640 1 -- 172.21.15.64:0/1829332614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40107b0 con 0x5637e84aba40 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 -- 172.21.15.64:0/1829332614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 msgr2=0x5637e84abe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84abe20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f85f4011ab0 tx=0x7f85f4011f70 comp rx=0 tx=0).stop 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 -- 172.21.15.64:0/1829332614 shutdown_connections 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84bb150 0x5637e84bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84ac360 0x5637e84bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1829332614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84abe20 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 -- 172.21.15.64:0/1829332614 >> 172.21.15.64:0/1829332614 conn(0x5637e84a80f0 msgr2=0x5637e84a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 -- 172.21.15.64:0/1829332614 shutdown_connections 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 -- 172.21.15.64:0/1829332614 wait complete. 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.378+0000 7f8608a6b740 1 -- start start 2024-09-19T05:11:38.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 Processor -- start 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5637e84bdd20 con 0x5637e84bb150 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5637e83cbf10 con 0x5637e84aba40 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637e84bdea0 con 0x5637e84ac360 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8608a6b740 10 client.0 _ll_register_callbacks cb 0x5637e8394a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8604c2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8604c2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50972/0 (socket says 172.21.15.64:50972) 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f8604c2f640 1 -- 172.21.15.64:0/817081384 learned_addr learned my addr 172.21.15.64:0/817081384 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f85fffff640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.379+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8604c2f640 1 -- 172.21.15.64:0/817081384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 msgr2=0x5637e84b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8604c2f640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8604c2f640 1 -- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 msgr2=0x5637e84b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8604c2f640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8604c2f640 1 -- 172.21.15.64:0/817081384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e81826a0 con 0x5637e84bb150 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85fffff640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8604c2f640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b8b80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f85f800bfd0 tx=0x7f85f809bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7682bc552b0d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85df7fe640 1 -- 172.21.15.64:0/817081384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80cc060 con 0x5637e84bb150 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85df7fe640 1 -- 172.21.15.64:0/817081384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f85f80c78d0 con 0x5637e84bb150 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f400aa50 con 0x5637e84bb150 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85df7fe640 1 -- 172.21.15.64:0/817081384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85f80a41e0 con 0x5637e84bb150 2024-09-19T05:11:38.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8608a6b740 10 client.16851 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637e84c9aa0 con 0x5637e84bb150 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85df7fe640 1 -- 172.21.15.64:0/817081384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f400aa50 con 0x5637e84bb150 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f85df7fe640 1 -- 172.21.15.64:0/817081384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85f809a9c0 con 0x5637e84bb150 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8608a6b740 10 client.16851 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.380+0000 7f8608a6b740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f8608a6b740 20 client.16851 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:38.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f8608a6b740 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e84caeb0 0x5637e84cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f8608a6b740 4 client.16851 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e841e7f0 con 0x5637e84caeb0 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f85fffff640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e84caeb0 0x5637e84cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f85fffff640 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e84caeb0 0x5637e84cd2a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1780dc79bfec02 server_cookie=4f15046a1d56dc5b in_seq=0 out_seq=0 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.381+0000 7f85df7fe640 10 client.16851 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f85df7fe640 1 -- 172.21.15.64:0/817081384 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637e841e7f0 con 0x5637e84caeb0 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f85df7fe640 10 client.16851 handle_command_reply: tid=0 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 client.16851 shutdown 2024-09-19T05:11:38.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 msgr2=0x5637e84b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b8b80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f85f800bfd0 tx=0x7f85f809bd30 comp rx=0 tx=0).stop 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 shutdown_connections 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e84caeb0 0x5637e84cd2a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b8b80 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 --2- 172.21.15.64:0/817081384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 >> 172.21.15.64:0/817081384 conn(0x5637e84a80f0 msgr2=0x5637e84ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f85df7fe640 0 client.16851 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.382+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 shutdown_connections 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.383+0000 7f8608a6b740 1 -- 172.21.15.64:0/817081384 wait complete. 2024-09-19T05:11:38.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.383+0000 7f8608a6b740 20 client.16851 trim_cache size 0 max 16384 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 Processor -- start 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 -- start start 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86001add50 con 0x5637e84bb150 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86001ab620 con 0x5637e84aba40 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8608a6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86001a9470 con 0x5637e84ac360 2024-09-19T05:11:38.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8604c2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8604c2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50986/0 (socket says 172.21.15.64:50986) 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.400+0000 7f8604c2f640 1 -- 172.21.15.64:0/1800469959 learned_addr learned my addr 172.21.15.64:0/1800469959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f85fffff640 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f8604c2f640 1 -- 172.21.15.64:0/1800469959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 msgr2=0x5637e84d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f8604c2f640 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84d3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f8604c2f640 1 -- 172.21.15.64:0/1800469959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 msgr2=0x5637e84d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f8604c2f640 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f8604c2f640 1 -- 172.21.15.64:0/1800469959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e84ac190 con 0x5637e84bb150 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f8604c2f640 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b5360 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b940 tx=0x7f85f809faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa41001079d6fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:38.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f85e5ffb640 1 -- 172.21.15.64:0/1800469959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f809b0b0 con 0x5637e84bb150 2024-09-19T05:11:38.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f85e5ffb640 1 -- 172.21.15.64:0/1800469959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f8097be0 con 0x5637e84bb150 2024-09-19T05:11:38.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.401+0000 7f85e5ffb640 1 -- 172.21.15.64:0/1800469959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f809aed0 con 0x5637e84bb150 2024-09-19T05:11:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 -- 172.21.15.64:0/1800469959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 msgr2=0x5637e84b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b5360 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b940 tx=0x7f85f809faa0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 -- 172.21.15.64:0/1800469959 shutdown_connections 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84bb150 0x5637e84b5360 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 --2- 172.21.15.64:0/1800469959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84aba40 0x5637e84d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 -- 172.21.15.64:0/1800469959 >> 172.21.15.64:0/1800469959 conn(0x5637e84a80f0 msgr2=0x5637e84ba070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 -- 172.21.15.64:0/1800469959 shutdown_connections 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 -- 172.21.15.64:0/1800469959 wait complete. 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 -- start start 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.402+0000 7f8608a6b740 1 Processor -- start 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84bb150 0x5637e84bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86001a9ae0 con 0x5637e84aba40 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86001aaf50 con 0x5637e84bb150 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86001aa1b0 con 0x5637e84ac360 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8608a6b740 10 client.0 _ll_register_callbacks cb 0x7f85f809b430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84bf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50996/0 (socket says 172.21.15.64:50996) 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 -- 172.21.15.64:0/2016575905 learned_addr learned my addr 172.21.15.64:0/2016575905 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f8604c2f640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84bb150 0x5637e84bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 -- 172.21.15.64:0/2016575905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 msgr2=0x5637e84bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 -- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84bb150 msgr2=0x5637e84bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84bb150 0x5637e84bfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 -- 172.21.15.64:0/2016575905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e8534d70 con 0x5637e84aba40 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85fffff640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84bf1b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005940 tx=0x7f85f00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff0c22baef67033 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.403+0000 7f85df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f85df7fe640 1 -- 172.21.15.64:0/2016575905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0048070 con 0x5637e84aba40 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f85df7fe640 1 -- 172.21.15.64:0/2016575905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f85f00314a0 con 0x5637e84aba40 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f8097be0 con 0x5637e84aba40 2024-09-19T05:11:38.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f85df7fe640 1 -- 172.21.15.64:0/2016575905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85f0054050 con 0x5637e84aba40 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 10 client.16857 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637e85360b0 con 0x5637e84aba40 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f85df7fe640 1 -- 172.21.15.64:0/2016575905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f8097be0 con 0x5637e84aba40 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f85df7fe640 1 -- 172.21.15.64:0/2016575905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85f0044320 con 0x5637e84aba40 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 10 client.16857 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 20 client.16857 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e8536780 0x5637e8538b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 4 client.16857 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637e8418d00 con 0x5637e8536780 2024-09-19T05:11:38.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.404+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e8536780 0x5637e8538b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.405+0000 7f85ff7fe640 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e8536780 0x5637e8538b70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be902f9a5e0ca2d2 server_cookie=ccbe8dadb1981f28 in_seq=0 out_seq=0 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.405+0000 7f85df7fe640 10 client.16857 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f85df7fe640 1 -- 172.21.15.64:0/2016575905 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637e8418d00 con 0x5637e8536780 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f85df7fe640 10 client.16857 handle_command_reply: tid=0 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 client.16857 shutdown 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 msgr2=0x5637e84bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84bf1b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005940 tx=0x7f85f00058b0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 shutdown_connections 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5637e84bb150 0x5637e84bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e84ac360 0x5637e84bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5637e8536780 0x5637e8538b70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 --2- 172.21.15.64:0/2016575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5637e84aba40 0x5637e84bf1b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.406+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 >> 172.21.15.64:0/2016575905 conn(0x5637e84a80f0 msgr2=0x5637e84a6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f85df7fe640 0 client.16857 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 shutdown_connections 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8608a6b740 1 -- 172.21.15.64:0/2016575905 wait complete. 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8608a6b740 20 client.16857 trim_cache size 0 max 16384 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:11:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "3907d7a8-b66d-41db-8fe2-473fb00df213", 2024-09-19T05:11:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:11:38.415 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f85e80a3430 msgr2=0x7f85e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f85e80a3430 0x7f85e80a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85f40058c0 tx=0x7f85f401c000 comp rx=0 tx=0).stop 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 msgr2=0x7f860019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f860019cc00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b970 tx=0x7f85f800bd70 comp rx=0 tx=0).stop 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 shutdown_connections 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f85e80a3430 0x7f85e80a58f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8600106a80 0x7f860019cc00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8600101420 0x7f8600198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 --2- 172.21.15.64:0/214398589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600100a80 0x7f8600198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 >> 172.21.15.64:0/214398589 conn(0x7f86000fc7a0 msgr2=0x7f860006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 shutdown_connections 2024-09-19T05:11:38.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:38.407+0000 7f8606692640 1 -- 172.21.15.64:0/214398589 wait complete. 2024-09-19T05:11:39.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:38 smithi064 ceph-mon[30960]: pgmap v175: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:39.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:38 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:11:39.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:38 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:39.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:38 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:11:39.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:38 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:39.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:38 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:11:39.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:38 smithi103 ceph-mon[34446]: pgmap v175: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:39.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:38 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:11:39.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:38 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:39.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:38 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:11:39.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:38 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:39.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:38 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:11:39.270 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:39.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:38 smithi174 ceph-mon[34488]: pgmap v175: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:39.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:38 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:11:39.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:38 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:38 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:11:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:38 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:39.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:38 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:11:39.535 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 -- 172.21.15.64:0/1177350225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a010e500 msgr2=0x7f31a0110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1177350225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a010e500 0x7f31a0110910 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3198009900 tx=0x7f319802a410 comp rx=0 tx=0).stop 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 -- 172.21.15.64:0/1177350225 shutdown_connections 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1177350225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a010e500 0x7f31a0110910 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1177350225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0101440 0x7f31a010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1177350225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31a0100b20 0x7f31a0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 -- 172.21.15.64:0/1177350225 >> 172.21.15.64:0/1177350225 conn(0x7f31a00fc7d0 msgr2=0x7f31a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:40.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 -- 172.21.15.64:0/1177350225 shutdown_connections 2024-09-19T05:11:40.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 -- 172.21.15.64:0/1177350225 wait complete. 2024-09-19T05:11:40.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.004+0000 7f31a4aee640 1 Processor -- start 2024-09-19T05:11:40.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 -- start start 2024-09-19T05:11:40.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0100b20 0x7f31a019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 0x7f31a019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31a010e500 0x7f31a019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0114100 con 0x7f31a0101440 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0113f80 con 0x7f31a0100b20 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f31a4aee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0114280 con 0x7f31a010e500 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 0x7f31a019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 0x7f31a019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51006/0 (socket says 172.21.15.64:51006) 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 -- 172.21.15.64:0/1099884173 learned_addr learned my addr 172.21.15.64:0/1099884173 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319e59c640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0100b20 0x7f31a019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 -- 172.21.15.64:0/1099884173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31a010e500 msgr2=0x7f31a019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31a010e500 0x7f31a019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 -- 172.21.15.64:0/1099884173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0100b20 msgr2=0x7f31a019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0100b20 0x7f31a019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 -- 172.21.15.64:0/1099884173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a019f590 con 0x7f31a0101440 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319e59c640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0100b20 0x7f31a019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.005+0000 7f319dd9b640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 0x7f31a019ab20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f319400c970 tx=0x7f319400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.006+0000 7f318f7fe640 1 -- 172.21.15.64:0/1099884173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004220 con 0x7f31a0101440 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.006+0000 7f318f7fe640 1 -- 172.21.15.64:0/1099884173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31940043c0 con 0x7f31a0101440 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.006+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a019f880 con 0x7f31a0101440 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.006+0000 7f318f7fe640 1 -- 172.21.15.64:0/1099884173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004a00 con 0x7f31a0101440 2024-09-19T05:11:40.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.006+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a01a7160 con 0x7f31a0101440 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.008+0000 7f318f7fe640 1 -- 172.21.15.64:0/1099884173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f31940079b0 con 0x7f31a0101440 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.008+0000 7f31a6ec7740 1 Processor -- start 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 -- start start 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f84a0 0x564a549f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a03140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a54a03680 0x564a54a05a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a5495e2e0 con 0x564a54a03680 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a54905ce0 con 0x564a549f84a0 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a5495a2a0 con 0x564a549f8dc0 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a54a03680 0x564a54a05a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a03140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f84a0 0x564a549f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a03140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:46022/0 (socket says 172.21.15.64:46022) 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a54a03680 0x564a54a05a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51016/0 (socket says 172.21.15.64:51016) 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 -- 172.21.15.64:0/3508277625 learned_addr learned my addr 172.21.15.64:0/3508277625 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f84a0 0x564a549f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:42132/0 (socket says 172.21.15.64:42132) 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f318f7fe640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f31880a3430 0x7f31880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319ed9d640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f31880a3430 0x7f31880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 -- 172.21.15.64:0/3508277625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f84a0 msgr2=0x564a549f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f84a0 0x564a549f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f318f7fe640 1 -- 172.21.15.64:0/1099884173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f31940cfa50 con 0x7f31a0101440 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 -- 172.21.15.64:0/3508277625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a54a03680 msgr2=0x564a54a05a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.009+0000 7f319e59c640 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a54a03680 0x564a54a05a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.010+0000 7f319e59c640 1 -- 172.21.15.64:0/3508277625 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a548ea440 con 0x564a549f8dc0 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.010+0000 7f319e59c640 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a03140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3190008f10 tx=0x7f319000d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86e282e666782008 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.010+0000 7f319ed9d640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f31880a3430 0x7f31880a58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3198009230 tx=0x7f3198005fd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.010+0000 7f318cff9640 1 -- 172.21.15.64:0/3508277625 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319000f830 con 0x564a549f8dc0 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.010+0000 7f318cff9640 1 -- 172.21.15.64:0/3508277625 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3190018480 con 0x564a549f8dc0 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.010+0000 7f318cff9640 1 -- 172.21.15.64:0/3508277625 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319000e5b0 con 0x564a549f8dc0 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3508277625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 msgr2=0x564a54a03140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a03140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3190008f10 tx=0x7f319000d710 comp rx=0 tx=0).stop 2024-09-19T05:11:40.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3508277625 shutdown_connections 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a54a03680 0x564a54a05a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a03140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3508277625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f84a0 0x564a549f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3508277625 >> 172.21.15.64:0/3508277625 conn(0x564a549f4a30 msgr2=0x564a549f4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3508277625 shutdown_connections 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3508277625 wait complete. 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 -- start start 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.011+0000 7f31a6ec7740 1 Processor -- start 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a02fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a00280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a54a06250 con 0x564a549f84a0 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a54919050 con 0x564a54a03680 2024-09-19T05:11:40.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a54a063d0 con 0x564a549f8dc0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f31a6ec7740 10 client.0 _ll_register_callbacks cb 0x564a54959160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a02fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a02fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51028/0 (socket says 172.21.15.64:51028) 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 -- 172.21.15.64:0/4077027140 learned_addr learned my addr 172.21.15.64:0/4077027140 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319dd9b640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a00280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319ed9d640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 -- 172.21.15.64:0/4077027140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 msgr2=0x564a54a00280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a00280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 -- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 msgr2=0x564a54a007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 -- 172.21.15.64:0/4077027140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a546cf3c0 con 0x564a549f84a0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f319e59c640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a02fd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3190019fd0 tx=0x7f31900082b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7650975b74337c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:40.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.012+0000 7f3185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f3185ffb640 1 -- 172.21.15.64:0/4077027140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190028060 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3190018480 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f3185ffb640 1 -- 172.21.15.64:0/4077027140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3190004c50 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f3185ffb640 1 -- 172.21.15.64:0/4077027140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3190004e50 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 10 client.16869 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a54a11c80 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f3185ffb640 1 -- 172.21.15.64:0/4077027140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3190018480 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f3185ffb640 1 -- 172.21.15.64:0/4077027140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f31900052a0 con 0x564a549f84a0 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 10 client.16869 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 20 client.16869 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a13140 0x564a54a15530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 4 client.16869 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:40.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.013+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a5496b850 con 0x564a54a13140 2024-09-19T05:11:40.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.014+0000 7f319dd9b640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a13140 0x564a54a15530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.014+0000 7f319dd9b640 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a13140 0x564a54a15530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d47b4fbe880cd2 server_cookie=aa084179dfe716de in_seq=0 out_seq=0 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.014+0000 7f3185ffb640 10 client.16869 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f3185ffb640 1 -- 172.21.15.64:0/4077027140 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a5496b850 con 0x564a54a13140 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f3185ffb640 10 client.16869 handle_command_reply: tid=0 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 client.16869 shutdown 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 msgr2=0x564a54a02fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a02fd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3190019fd0 tx=0x7f31900082b0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 shutdown_connections 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f8dc0 0x564a54a00280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a13140 0x564a54a15530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/4077027140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a02fd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 >> 172.21.15.64:0/4077027140 conn(0x564a549f4a30 msgr2=0x564a549fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:40.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f3185ffb640 0 client.16869 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:40.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 shutdown_connections 2024-09-19T05:11:40.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 1 -- 172.21.15.64:0/4077027140 wait complete. 2024-09-19T05:11:40.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.015+0000 7f31a6ec7740 20 client.16869 trim_cache size 0 max 16384 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 Processor -- start 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 -- start start 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f84a0 0x564a54a0f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 0x564a54a1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a1b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01b23d0 con 0x564a549f8dc0 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01afca0 con 0x564a54a03680 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01adaf0 con 0x564a549f84a0 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.032+0000 7f319dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 0x564a54a1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 0x564a54a1ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51044/0 (socket says 172.21.15.64:51044) 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 -- 172.21.15.64:0/3544058783 learned_addr learned my addr 172.21.15.64:0/3544058783 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319e59c640 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f84a0 0x564a54a0f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319ed9d640 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a1b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 -- 172.21.15.64:0/3544058783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f84a0 msgr2=0x564a54a0f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f84a0 0x564a54a0f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 -- 172.21.15.64:0/3544058783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 msgr2=0x564a54a1b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a1b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 -- 172.21.15.64:0/3544058783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a54a05dd0 con 0x564a549f8dc0 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f319dd9b640 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 0x564a54a1ab30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f31940cd7a0 tx=0x7f31940d1fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b9ec06d3868d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f3187fff640 1 -- 172.21.15.64:0/3544058783 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319401d040 con 0x564a549f8dc0 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f3187fff640 1 -- 172.21.15.64:0/3544058783 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31940a49a0 con 0x564a549f8dc0 2024-09-19T05:11:40.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.033+0000 7f3187fff640 1 -- 172.21.15.64:0/3544058783 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194097a80 con 0x564a549f8dc0 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3544058783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 msgr2=0x564a54a1ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 0x564a54a1ab30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f31940cd7a0 tx=0x7f31940d1fd0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3544058783 shutdown_connections 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a54a03680 0x564a54a1b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f8dc0 0x564a54a1ab30 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3544058783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a549f84a0 0x564a54a0f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3544058783 >> 172.21.15.64:0/3544058783 conn(0x564a549f4a30 msgr2=0x564a549ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3544058783 shutdown_connections 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3544058783 wait complete. 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 -- start start 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.034+0000 7f31a6ec7740 1 Processor -- start 2024-09-19T05:11:40.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a06d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f8dc0 0x564a54a07260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a54a03680 0x564a54a75b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01ae160 con 0x564a549f84a0 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01af5d0 con 0x564a549f8dc0 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31a01ae830 con 0x564a54a03680 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 10 client.0 _ll_register_callbacks cb 0x7f319409bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a06d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a06d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51060/0 (socket says 172.21.15.64:51060) 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 -- 172.21.15.64:0/3402539791 learned_addr learned my addr 172.21.15.64:0/3402539791 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319dd9b640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f8dc0 0x564a54a07260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319ed9d640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a54a03680 0x564a54a75b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 -- 172.21.15.64:0/3402539791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a54a03680 msgr2=0x564a54a75b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a54a03680 0x564a54a75b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 -- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f8dc0 msgr2=0x564a54a07260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f8dc0 0x564a54a07260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 -- 172.21.15.64:0/3402539791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a54a761d0 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f319e59c640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a06d20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3190005860 tx=0x7f3190010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634c06329b275032 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f3185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f3185ffb640 1 -- 172.21.15.64:0/3402539791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190018480 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f3185ffb640 1 -- 172.21.15.64:0/3402539791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f319001f040 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.035+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31940a49a0 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f3185ffb640 1 -- 172.21.15.64:0/3402539791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f319003d050 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 10 client.16875 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a54a7d570 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f3185ffb640 1 -- 172.21.15.64:0/3402539791 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31940a49a0 con 0x564a549f84a0 2024-09-19T05:11:40.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f3185ffb640 1 -- 172.21.15.64:0/3402539791 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3190034070 con 0x564a549f84a0 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 10 client.16875 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 20 client.16875 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a7dc30 0x564a54a80020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 4 client.16875 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a54a15920 con 0x564a54a7dc30 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.036+0000 7f319dd9b640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a7dc30 0x564a54a80020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f3185ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f319dd9b640 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a7dc30 0x564a54a80020 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a112f93ec509b9fa server_cookie=be8c5a50e6cc5f0f in_seq=0 out_seq=0 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f3185ffb640 1 -- 172.21.15.64:0/3402539791 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564a54a15920 con 0x564a54a7dc30 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f3185ffb640 10 client.16875 handle_command_reply: tid=0 2024-09-19T05:11:40.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 client.16875 shutdown 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 msgr2=0x564a54a06d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a06d20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3190005860 tx=0x7f3190010dd0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 shutdown_connections 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a54a03680 0x564a54a75b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a549f8dc0 0x564a54a07260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564a54a7dc30 0x564a54a80020 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 --2- 172.21.15.64:0/3402539791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a549f84a0 0x564a54a06d20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 >> 172.21.15.64:0/3402539791 conn(0x564a549f4a30 msgr2=0x564a549faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f3185ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 shutdown_connections 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 1 -- 172.21.15.64:0/3402539791 wait complete. 2024-09-19T05:11:40.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.037+0000 7f31a6ec7740 20 client.16875 trim_cache size 0 max 16384 2024-09-19T05:11:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:40.046 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:11:40.046 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:11:40.046 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f31880a3430 msgr2=0x7f31880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f31880a3430 0x7f31880a58f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3198009230 tx=0x7f3198005fd0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 msgr2=0x7f31a019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 0x7f31a019ab20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f319400c970 tx=0x7f319400ce30 comp rx=0 tx=0).stop 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 shutdown_connections 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f31880a3430 0x7f31880a58f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31a010e500 0x7f31a019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a0101440 0x7f31a019ab20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 --2- 172.21.15.64:0/1099884173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31a0100b20 0x7f31a019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 >> 172.21.15.64:0/1099884173 conn(0x7f31a00fc7d0 msgr2=0x7f31a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 shutdown_connections 2024-09-19T05:11:40.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:40.038+0000 7f31a4aee640 1 -- 172.21.15.64:0/1099884173 wait complete. 2024-09-19T05:11:40.888 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:11:40.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:11:41.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:40 smithi064 ceph-mon[30960]: pgmap v176: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:41.157 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:41.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:40 smithi103 ceph-mon[34446]: pgmap v176: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:41.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:40 smithi174 ceph-mon[34488]: pgmap v176: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 -- 172.21.15.64:0/103003084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c38103660 msgr2=0x7f0c3810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/103003084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c38103660 0x7f0c3810bb50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c2002a480 comp rx=0 tx=0).stop 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 -- 172.21.15.64:0/103003084 shutdown_connections 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/103003084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c3810c170 0x7f0c3810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/103003084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c38103660 0x7f0c3810bb50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/103003084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38102d40 0x7f0c38103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 -- 172.21.15.64:0/103003084 >> 172.21.15.64:0/103003084 conn(0x7f0c380fc7a0 msgr2=0x7f0c380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:41.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 -- 172.21.15.64:0/103003084 shutdown_connections 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.619+0000 7f0c3cef8640 1 -- 172.21.15.64:0/103003084 wait complete. 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 Processor -- start 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 -- start start 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38102d40 0x7f0c3819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c3819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 0x7f0c3819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111e20 con 0x7f0c3810c170 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111ca0 con 0x7f0c38103660 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.620+0000 7f0c3cef8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111fa0 con 0x7f0c38102d40 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 0x7f0c3819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:41.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 0x7f0c3819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51074/0 (socket says 172.21.15.64:51074) 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 -- 172.21.15.64:0/2065216620 learned_addr learned my addr 172.21.15.64:0/2065216620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c3659c640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38102d40 0x7f0c3819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c35d9b640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c3819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 -- 172.21.15.64:0/2065216620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38102d40 msgr2=0x7f0c3819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38102d40 0x7f0c3819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 -- 172.21.15.64:0/2065216620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 msgr2=0x7f0c3819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c3819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 -- 172.21.15.64:0/2065216620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3819f540 con 0x7f0c3810c170 2024-09-19T05:11:41.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c36d9d640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 0x7f0c3819ee60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000eaa0 tx=0x7f0c3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000cdb0 con 0x7f0c3810c170 2024-09-19T05:11:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c30005020 con 0x7f0c3810c170 2024-09-19T05:11:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3819f830 con 0x7f0c3810c170 2024-09-19T05:11:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.621+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300116d0 con 0x7f0c3810c170 2024-09-19T05:11:41.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.622+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a7110 con 0x7f0c3810c170 2024-09-19T05:11:41.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.622+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38103120 con 0x7f0c3810c170 2024-09-19T05:11:41.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.624+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0c300051d0 con 0x7f0c3810c170 2024-09-19T05:11:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.625+0000 7f0c2f7fe640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c240a3430 0x7f0c240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.625+0000 7f0c3659c640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c240a3430 0x7f0c240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.625+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f0c300cfd60 con 0x7f0c3810c170 2024-09-19T05:11:41.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.625+0000 7f0c3659c640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c240a3430 0x7f0c240a58f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c28005fd0 tx=0x7f0c2800b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:41.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.629+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c30016030 con 0x7f0c3810c170 2024-09-19T05:11:41.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.843+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c381a7400 con 0x7f0c3810c170 2024-09-19T05:11:41.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.844+0000 7f0c2f7fe640 1 -- 172.21.15.64:0/2065216620 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f0c3009c0f0 con 0x7f0c3810c170 2024-09-19T05:11:41.845 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:11:41.846 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:11:41.846 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c240a3430 msgr2=0x7f0c240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c240a3430 0x7f0c240a58f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c28005fd0 tx=0x7f0c2800b390 comp rx=0 tx=0).stop 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 msgr2=0x7f0c3819ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 0x7f0c3819ee60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000eaa0 tx=0x7f0c3000ef60 comp rx=0 tx=0).stop 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 shutdown_connections 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0c240a3430 0x7f0c240a58f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c3810c170 0x7f0c3819ee60 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c3819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 --2- 172.21.15.64:0/2065216620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38102d40 0x7f0c3819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 >> 172.21.15.64:0/2065216620 conn(0x7f0c380fc7a0 msgr2=0x7f0c381008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 shutdown_connections 2024-09-19T05:11:41.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:41.846+0000 7f0c3cef8640 1 -- 172.21.15.64:0/2065216620 wait complete. 2024-09-19T05:11:42.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:41 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2065216620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:42.220 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:41 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2065216620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:42.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:41 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2065216620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:42.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:11:42.963 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:43.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:42 smithi064 ceph-mon[30960]: pgmap v177: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:43.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:42 smithi103 ceph-mon[34446]: pgmap v177: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:43.291 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:42 smithi174 ceph-mon[34488]: pgmap v177: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/785525108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34101de0 msgr2=0x7f5b341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/785525108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34101de0 0x7f5b341021c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009900 tx=0x7f5b3002a410 comp rx=0 tx=0).stop 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/785525108 shutdown_connections 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/785525108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34106f10 0x7f5b3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/785525108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34102790 0x7f5b341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/785525108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34101de0 0x7f5b341021c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/785525108 >> 172.21.15.64:0/785525108 conn(0x7f5b340fd8c0 msgr2=0x7f5b340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/785525108 shutdown_connections 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/785525108 wait complete. 2024-09-19T05:11:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 Processor -- start 2024-09-19T05:11:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.396+0000 7f5b3c6eb640 1 -- start start 2024-09-19T05:11:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3c6eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101de0 0x7f5b341983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3c6eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 0x7f5b34198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3c6eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34106f10 0x7f5b3419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3c6eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34111e60 con 0x7f5b34102790 2024-09-19T05:11:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3c6eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34111ce0 con 0x7f5b34106f10 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3c6eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34111fe0 con 0x7f5b34101de0 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 0x7f5b34198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 0x7f5b34198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51084/0 (socket says 172.21.15.64:51084) 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101de0 0x7f5b341983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 -- 172.21.15.64:0/1742506089 learned_addr learned my addr 172.21.15.64:0/1742506089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34106f10 0x7f5b3419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 -- 172.21.15.64:0/1742506089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101de0 msgr2=0x7f5b341983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101de0 0x7f5b341983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 -- 172.21.15.64:0/1742506089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34106f10 msgr2=0x7f5b3419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34106f10 0x7f5b3419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 -- 172.21.15.64:0/1742506089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3419d390 con 0x7f5b34102790 2024-09-19T05:11:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34106f10 0x7f5b3419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.397+0000 7f5b39c86640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 0x7f5b34198920 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b4a0 tx=0x7f5b2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.398+0000 7f5b2b7fe640 1 -- 172.21.15.64:0/1742506089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24014070 con 0x7f5b34102790 2024-09-19T05:11:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.398+0000 7f5b2b7fe640 1 -- 172.21.15.64:0/1742506089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b24010ac0 con 0x7f5b34102790 2024-09-19T05:11:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.398+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3419d680 con 0x7f5b34102790 2024-09-19T05:11:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.398+0000 7f5b2b7fe640 1 -- 172.21.15.64:0/1742506089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24017c30 con 0x7f5b34102790 2024-09-19T05:11:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.398+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b341a4f60 con 0x7f5b34102790 2024-09-19T05:11:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.400+0000 7f5b2b7fe640 1 -- 172.21.15.64:0/1742506089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5b24017070 con 0x7f5b34102790 2024-09-19T05:11:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.400+0000 7f5b3eac4740 1 Processor -- start 2024-09-19T05:11:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 -- start start 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cefa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d00b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556832c552f0 con 0x556832cef5d0 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556832bfccf0 con 0x556832cfe780 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556832c512b0 con 0x556832cf0b40 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cefa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cefa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51096/0 (socket says 172.21.15.64:51096) 2024-09-19T05:11:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 -- 172.21.15.64:0/2589788640 learned_addr learned my addr 172.21.15.64:0/2589788640 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b2b7fe640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5b200a33b0 0x7f5b200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b39c86640 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d00b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5b200a33b0 0x7f5b200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 -- 172.21.15.64:0/2589788640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 msgr2=0x556832cef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 -- 172.21.15.64:0/2589788640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 msgr2=0x556832d00b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d00b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b3a487640 1 -- 172.21.15.64:0/2589788640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556832be13a0 con 0x556832cef5d0 2024-09-19T05:11:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.401+0000 7f5b39c86640 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d00b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.402+0000 7f5b2b7fe640 1 -- 172.21.15.64:0/1742506089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5b240cf2f0 con 0x7f5b34102790 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.402+0000 7f5b3a487640 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cefa10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009900 tx=0x7f5b30032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fa8102f66e2a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.402+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5b200a33b0 0x7f5b200a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005fd0 tx=0x7f5b2c004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.402+0000 7f5b28ff9640 1 -- 172.21.15.64:0/2589788640 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30040070 con 0x556832cef5d0 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.402+0000 7f5b28ff9640 1 -- 172.21.15.64:0/2589788640 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b300316a0 con 0x556832cef5d0 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.402+0000 7f5b28ff9640 1 -- 172.21.15.64:0/2589788640 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30035040 con 0x556832cef5d0 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2589788640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 msgr2=0x556832cefa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cefa10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009900 tx=0x7f5b30032e40 comp rx=0 tx=0).stop 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2589788640 shutdown_connections 2024-09-19T05:11:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d00b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cefa10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2589788640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2589788640 >> 172.21.15.64:0/2589788640 conn(0x556832cb8310 msgr2=0x556832cebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2589788640 shutdown_connections 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2589788640 wait complete. 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 -- start start 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.403+0000 7f5b3eac4740 1 Processor -- start 2024-09-19T05:11:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cfe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832cfba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556832d01420 con 0x556832cef5d0 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556832c10060 con 0x556832cfe780 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556832d015a0 con 0x556832cf0b40 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 10 client.0 _ll_register_callbacks cb 0x5568329638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cfe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cfe170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51104/0 (socket says 172.21.15.64:51104) 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 -- 172.21.15.64:0/977104345 learned_addr learned my addr 172.21.15.64:0/977104345 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b39c86640 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 -- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 msgr2=0x556832cfb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 -- 172.21.15.64:0/977104345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 msgr2=0x556832cfba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832cfba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 -- 172.21.15.64:0/977104345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568329c63c0 con 0x556832cef5d0 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3a487640 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cfe170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5b30032fd0 tx=0x7f5b30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a06a3d3ebc340d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b1dffb640 1 -- 172.21.15.64:0/977104345 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30035070 con 0x556832cef5d0 2024-09-19T05:11:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b1dffb640 1 -- 172.21.15.64:0/977104345 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5b30004030 con 0x556832cef5d0 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.404+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b300316a0 con 0x556832cef5d0 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b1dffb640 1 -- 172.21.15.64:0/977104345 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b30055050 con 0x556832cef5d0 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 10 client.16893 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556832d0cec0 con 0x556832cef5d0 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b1dffb640 1 -- 172.21.15.64:0/977104345 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b300316a0 con 0x556832cef5d0 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b1dffb640 1 -- 172.21.15.64:0/977104345 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b30004750 con 0x556832cef5d0 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 10 client.16893 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 10 client.16893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 20 client.16893 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d0e210 0x556832d10600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 4 client.16893 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556832c62860 con 0x556832d0e210 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.405+0000 7f5b39c86640 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d0e210 0x556832d10600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.406+0000 7f5b39c86640 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d0e210 0x556832d10600 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=574bc5c3021b5382 server_cookie=ab076fb6be536ae0 in_seq=0 out_seq=0 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.406+0000 7f5b1dffb640 10 client.16893 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b1dffb640 1 -- 172.21.15.64:0/977104345 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556832c62860 con 0x556832d0e210 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b1dffb640 10 client.16893 handle_command_reply: tid=0 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 client.16893 shutdown 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 msgr2=0x556832cfe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cfe170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5b30032fd0 tx=0x7f5b30005fd0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 shutdown_connections 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d0e210 0x556832d10600 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832cfba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b1dffb640 0 client.16893 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/977104345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832cfe170 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 >> 172.21.15.64:0/977104345 conn(0x556832cb8310 msgr2=0x556832cf1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 shutdown_connections 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 1 -- 172.21.15.64:0/977104345 wait complete. 2024-09-19T05:11:43.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.407+0000 7f5b3eac4740 20 client.16893 trim_cache size 0 max 16384 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 Processor -- start 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 -- start start 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d0a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d15bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341adf00 con 0x556832cef5d0 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341ab7d0 con 0x556832cfe780 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341a9620 con 0x556832cf0b40 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d0a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d0a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51112/0 (socket says 172.21.15.64:51112) 2024-09-19T05:11:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.420+0000 7f5b3a487640 1 -- 172.21.15.64:0/1811225062 learned_addr learned my addr 172.21.15.64:0/1811225062 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d15bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b39c86640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3a487640 1 -- 172.21.15.64:0/1811225062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 msgr2=0x556832cfa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3a487640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3a487640 1 -- 172.21.15.64:0/1811225062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 msgr2=0x556832d15bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3a487640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d15bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3a487640 1 -- 172.21.15.64:0/1811225062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556832cfc7a0 con 0x556832cef5d0 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b39c86640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b3a487640 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d0a6c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5b3002a930 tx=0x7f5b30031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877c36c2b9efb23d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b1ffff640 1 -- 172.21.15.64:0/1811225062 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30035030 con 0x556832cef5d0 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b1ffff640 1 -- 172.21.15.64:0/1811225062 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3004c070 con 0x556832cef5d0 2024-09-19T05:11:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.421+0000 7f5b1ffff640 1 -- 172.21.15.64:0/1811225062 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30040040 con 0x556832cef5d0 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- 172.21.15.64:0/1811225062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 msgr2=0x556832d0a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d0a6c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5b3002a930 tx=0x7f5b30031400 comp rx=0 tx=0).stop 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- 172.21.15.64:0/1811225062 shutdown_connections 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cfe780 0x556832d15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cf0b40 0x556832cfa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/1811225062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d0a6c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- 172.21.15.64:0/1811225062 >> 172.21.15.64:0/1811225062 conn(0x556832cb8310 msgr2=0x556832cf9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- 172.21.15.64:0/1811225062 shutdown_connections 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- 172.21.15.64:0/1811225062 wait complete. 2024-09-19T05:11:43.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- start start 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 Processor -- start 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cf0b40 0x556832cf7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cfe780 0x556832cf8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341a9c90 con 0x556832cef5d0 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341ab100 con 0x556832cf0b40 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341aa360 con 0x556832cfe780 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.422+0000 7f5b3eac4740 10 client.0 _ll_register_callbacks cb 0x7f5b2c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d06db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51124/0 (socket says 172.21.15.64:51124) 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 -- 172.21.15.64:0/2734633743 learned_addr learned my addr 172.21.15.64:0/2734633743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3ac88640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cfe780 0x556832cf8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b39c86640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cf0b40 0x556832cf7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 -- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cfe780 msgr2=0x556832cf8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cfe780 0x556832cf8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 -- 172.21.15.64:0/2734633743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cf0b40 msgr2=0x556832cf7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cf0b40 0x556832cf7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 -- 172.21.15.64:0/2734633743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556832cf8a60 con 0x556832cef5d0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3a487640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d06db0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5b300096c0 tx=0x7f5b300054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b2a829bfb47a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b1dffb640 1 -- 172.21.15.64:0/2734633743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3005e060 con 0x556832cef5d0 2024-09-19T05:11:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b1dffb640 1 -- 172.21.15.64:0/2734633743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5b300046c0 con 0x556832cef5d0 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b3004c070 con 0x556832cef5d0 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b1dffb640 1 -- 172.21.15.64:0/2734633743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b30040070 con 0x556832cef5d0 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3eac4740 10 client.16899 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.423+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556832d788c0 con 0x556832cef5d0 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b1dffb640 1 -- 172.21.15.64:0/2734633743 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3004c070 con 0x556832cef5d0 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b1dffb640 1 -- 172.21.15.64:0/2734633743 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b30032530 con 0x556832cef5d0 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b3eac4740 10 client.16899 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b3eac4740 10 client.16899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b3eac4740 20 client.16899 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d78f50 0x556832d7b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b3eac4740 4 client.16899 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556832c5cd70 con 0x556832d78f50 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.424+0000 7f5b39c86640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d78f50 0x556832d7b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b39c86640 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d78f50 0x556832d7b370 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29e41886d47c98a5 server_cookie=47eb8276cdd7aa1f in_seq=0 out_seq=0 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b1dffb640 10 client.16899 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b1dffb640 1 -- 172.21.15.64:0/2734633743 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556832c5cd70 con 0x556832d78f50 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b1dffb640 10 client.16899 handle_command_reply: tid=0 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 client.16899 shutdown 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 msgr2=0x556832d06db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d06db0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5b300096c0 tx=0x7f5b300054a0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 shutdown_connections 2024-09-19T05:11:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556832d78f50 0x556832d7b370 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556832cfe780 0x556832cf8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556832cf0b40 0x556832cf7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 --2- 172.21.15.64:0/2734633743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556832cef5d0 0x556832d06db0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b1dffb640 0 client.16899 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 >> 172.21.15.64:0/2734633743 conn(0x556832cb8310 msgr2=0x556832cfa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 shutdown_connections 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 1 -- 172.21.15.64:0/2734633743 wait complete. 2024-09-19T05:11:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.425+0000 7f5b3eac4740 20 client.16899 trim_cache size 0 max 16384 2024-09-19T05:11:43.434 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5b200a33b0 msgr2=0x7f5b200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5b200a33b0 0x7f5b200a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005fd0 tx=0x7f5b2c004ab0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 msgr2=0x7f5b34198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 0x7f5b34198920 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b4a0 tx=0x7f5b2400b960 comp rx=0 tx=0).stop 2024-09-19T05:11:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 shutdown_connections 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5b200a33b0 0x7f5b200a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b34106f10 0x7f5b3419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b34102790 0x7f5b34198920 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 --2- 172.21.15.64:0/1742506089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101de0 0x7f5b341983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 >> 172.21.15.64:0/1742506089 conn(0x7f5b340fd8c0 msgr2=0x7f5b340fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 shutdown_connections 2024-09-19T05:11:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:43.426+0000 7f5b3c6eb640 1 -- 172.21.15.64:0/1742506089 wait complete. 2024-09-19T05:11:44.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:11:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:44.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 -- 172.21.15.64:0/3862923799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58100a80 msgr2=0x7f3e58100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 --2- 172.21.15.64:0/3862923799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58100a80 0x7f3e58100ee0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009930 tx=0x7f3e5402a420 comp rx=0 tx=0).stop 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 -- 172.21.15.64:0/3862923799 shutdown_connections 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 --2- 172.21.15.64:0/3862923799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e58101420 0x7f3e5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 --2- 172.21.15.64:0/3862923799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58100a80 0x7f3e58100ee0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 --2- 172.21.15.64:0/3862923799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e58106a80 0x7f3e58106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 -- 172.21.15.64:0/3862923799 >> 172.21.15.64:0/3862923799 conn(0x7f3e580fc7a0 msgr2=0x7f3e580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 -- 172.21.15.64:0/3862923799 shutdown_connections 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.870+0000 7f3e60782640 1 -- 172.21.15.64:0/3862923799 wait complete. 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 Processor -- start 2024-09-19T05:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 -- start start 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e58100a80 0x7f3e58198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e58101420 0x7f3e58198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 0x7f3e5819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58110d30 con 0x7f3e58106a80 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58110bb0 con 0x7f3e58101420 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e60782640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58110eb0 con 0x7f3e58100a80 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e5ed1f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 0x7f3e5819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e5ed1f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 0x7f3e5819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37154/0 (socket says 172.21.15.64:37154) 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.871+0000 7f3e5ed1f640 1 -- 172.21.15.64:0/2864195174 learned_addr learned my addr 172.21.15.64:0/2864195174 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e58101420 0x7f3e58198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e58100a80 0x7f3e58198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5ed1f640 1 -- 172.21.15.64:0/2864195174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e58100a80 msgr2=0x7f3e58198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5ed1f640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e58100a80 0x7f3e58198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5ed1f640 1 -- 172.21.15.64:0/2864195174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e58101420 msgr2=0x7f3e58198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5ed1f640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e58101420 0x7f3e58198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5ed1f640 1 -- 172.21.15.64:0/2864195174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5819d300 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e5ed1f640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 0x7f3e5819cc20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000cce0 tx=0x7f3e5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e4f7fe640 1 -- 172.21.15.64:0/2864195174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000f450 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e4f7fe640 1 -- 172.21.15.64:0/2864195174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e5000faa0 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5819d5f0 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e4f7fe640 1 -- 172.21.15.64:0/2864195174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50018b70 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.872+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581a4ed0 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e4f7fe640 1 -- 172.21.15.64:0/2864195174 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3e50018d10 con 0x7f3e58106a80 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 Processor -- start 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 -- start start 2024-09-19T05:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13420090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13420a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13431b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fb133862f0 con 0x55fb1342f780 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb1332dcf0 con 0x55fb134205d0 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.875+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb133822b0 con 0x55fb13421b40 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13431b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13431b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37170/0 (socket says 172.21.15.64:37170) 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/2697132495 learned_addr learned my addr 172.21.15.64:0/2697132495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e4f7fe640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3e440a33b0 0x7f3e440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13420a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13420090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5ed1f640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3e440a33b0 0x7f3e440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/2697132495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 msgr2=0x55fb13420090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13420090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/2697132495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 msgr2=0x55fb13420a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13420a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/2697132495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb133123a0 con 0x55fb1342f780 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e4f7fe640 1 -- 172.21.15.64:0/2864195174 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3e500d0100 con 0x7f3e58106a80 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13431b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009930 tx=0x7f3e54005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e637e88b2841605 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e4cff9640 1 -- 172.21.15.64:0/2697132495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54035070 con 0x55fb1342f780 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e5ed1f640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3e440a33b0 0x7f3e440a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cd900 tx=0x7f3e5006b0d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e4cff9640 1 -- 172.21.15.64:0/2697132495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e540316e0 con 0x55fb1342f780 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.876+0000 7f3e4cff9640 1 -- 172.21.15.64:0/2697132495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540047a0 con 0x55fb1342f780 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 -- 172.21.15.64:0/2697132495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 msgr2=0x55fb13431b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13431b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009930 tx=0x7f3e54005fd0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 -- 172.21.15.64:0/2697132495 shutdown_connections 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13431b70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13420a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/2697132495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13420090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 -- 172.21.15.64:0/2697132495 >> 172.21.15.64:0/2697132495 conn(0x55fb133e9310 msgr2=0x55fb1341cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 -- 172.21.15.64:0/2697132495 shutdown_connections 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.877+0000 7f3e62b5b740 1 -- 172.21.15.64:0/2697132495 wait complete. 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 -- start start 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 Processor -- start 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb134205d0 0x55fb1342f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 0x55fb1342c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb1342cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fb13432420 con 0x55fb13421b40 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb13341060 con 0x55fb1342f780 2024-09-19T05:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb134325a0 con 0x55fb134205d0 2024-09-19T05:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e62b5b740 10 client.0 _ll_register_callbacks cb 0x55fb130948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e5e51e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 0x55fb1342c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.107 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e5e51e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 0x55fb1342c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37172/0 (socket says 172.21.15.64:37172) 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e5e51e640 1 -- 172.21.15.64:0/1760732539 learned_addr learned my addr 172.21.15.64:0/1760732539 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb134205d0 0x55fb1342f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.878+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb1342cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5e51e640 1 -- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb134205d0 msgr2=0x55fb1342f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb134205d0 0x55fb1342f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5e51e640 1 -- 172.21.15.64:0/1760732539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 msgr2=0x55fb1342cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb1342cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5e51e640 1 -- 172.21.15.64:0/1760732539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb130f73c0 con 0x55fb13421b40 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb1342cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 0x55fb1342c460 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800b9f0 tx=0x7f3e4800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8f6379975bf641 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1760732539 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48003e20 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1760732539 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3e48011440 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e540316e0 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1760732539 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e48011640 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e62b5b740 10 client.16911 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb1343df60 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.879+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1760732539 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e540316e0 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1760732539 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3e4801b9d0 con 0x55fb13421b40 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e62b5b740 10 client.16911 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e62b5b740 10 client.16911 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e62b5b740 20 client.16911 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb1343f4c0 0x55fb134418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e62b5b740 4 client.16911 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb13393860 con 0x55fb1343f4c0 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb1343f4c0 0x55fb134418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb1343f4c0 0x55fb134418e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b76a0c3fc8ef13d4 server_cookie=b5265aebfd9fd339 in_seq=0 out_seq=0 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.880+0000 7f3e41ffb640 10 client.16911 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1760732539 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb13393860 con 0x55fb1343f4c0 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e41ffb640 10 client.16911 handle_command_reply: tid=0 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 client.16911 shutdown 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 msgr2=0x55fb1342c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 0x55fb1342c460 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800b9f0 tx=0x7f3e4800beb0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 shutdown_connections 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb1342cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb1343f4c0 0x55fb134418e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb13421b40 0x55fb1342c460 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1760732539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb134205d0 0x55fb1342f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e41ffb640 0 client.16911 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 >> 172.21.15.64:0/1760732539 conn(0x55fb133e9310 msgr2=0x55fb13422990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 shutdown_connections 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1760732539 wait complete. 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.881+0000 7f3e62b5b740 20 client.16911 trim_cache size 0 max 16384 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.894+0000 7f3e62b5b740 1 Processor -- start 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 -- start start 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 0x55fb1341cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb1342b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb13447190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581ade80 con 0x55fb134205d0 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581ab750 con 0x55fb1342f780 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581a95a0 con 0x55fb13421b40 2024-09-19T05:11:45.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 0x55fb1341cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 0x55fb1341cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37182/0 (socket says 172.21.15.64:37182) 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/3066909619 learned_addr learned my addr 172.21.15.64:0/3066909619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb1342b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb13447190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/3066909619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 msgr2=0x55fb1342b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb1342b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/3066909619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 msgr2=0x55fb13447190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb13447190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/3066909619 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1341d960 con 0x55fb134205d0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.895+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 0x55fb1341cd70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3e54046040 tx=0x7f3e54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fe01bbd852cb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.898+0000 7f3e43fff640 1 -- 172.21.15.64:0/3066909619 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54035040 con 0x55fb134205d0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.898+0000 7f3e43fff640 1 -- 172.21.15.64:0/3066909619 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e540316e0 con 0x55fb134205d0 2024-09-19T05:11:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.898+0000 7f3e43fff640 1 -- 172.21.15.64:0/3066909619 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54004290 con 0x55fb134205d0 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 -- 172.21.15.64:0/3066909619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 msgr2=0x55fb1341cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 0x55fb1341cd70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3e54046040 tx=0x7f3e54000bc0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 -- 172.21.15.64:0/3066909619 shutdown_connections 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb1342f780 0x55fb13447190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb1342b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/3066909619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb134205d0 0x55fb1341cd70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 -- 172.21.15.64:0/3066909619 >> 172.21.15.64:0/3066909619 conn(0x55fb133e9310 msgr2=0x55fb1342ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 -- 172.21.15.64:0/3066909619 shutdown_connections 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 -- 172.21.15.64:0/3066909619 wait complete. 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 -- start start 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.899+0000 7f3e62b5b740 1 Processor -- start 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13429400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13429940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13429e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581a9c10 con 0x55fb1342f780 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581ab080 con 0x55fb134205d0 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581aa2e0 con 0x55fb13421b40 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e62b5b740 10 client.0 _ll_register_callbacks cb 0x7f3e48004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13429e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13429e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37198/0 (socket says 172.21.15.64:37198) 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/1064900755 learned_addr learned my addr 172.21.15.64:0/1064900755 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13429400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13429940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 msgr2=0x55fb13429940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13429940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/1064900755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 msgr2=0x55fb13429400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13429400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 -- 172.21.15.64:0/1064900755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1341c4f0 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e5dd1d640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13429e80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x55fb13428c90 tx=0x7f3e54057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1adff0ec5d3df51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.900+0000 7f3e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1064900755 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54035070 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1064900755 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3e54005490 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e540316e0 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1064900755 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e54031cd0 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 10 client.16917 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb134aa1f0 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1064900755 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e540316e0 con 0x55fb1342f780 2024-09-19T05:11:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1064900755 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3e54055050 con 0x55fb1342f780 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 10 client.16917 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 10 client.16917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 20 client.16917 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb134aa420 0x55fb134ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 4 client.16917 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb1338dd70 con 0x55fb134aa420 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.901+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb134aa420 0x55fb134ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e41ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e5e51e640 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb134aa420 0x55fb134ac810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fe77bc26d3f1292 server_cookie=5d0d0b1692735fb2 in_seq=0 out_seq=0 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e41ffb640 1 -- 172.21.15.64:0/1064900755 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb1338dd70 con 0x55fb134aa420 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e41ffb640 10 client.16917 handle_command_reply: tid=0 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 client.16917 shutdown 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 msgr2=0x55fb13429e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13429e80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x55fb13428c90 tx=0x7f3e54057040 comp rx=0 tx=0).stop 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 shutdown_connections 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55fb134aa420 0x55fb134ac810 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb1342f780 0x55fb13429e80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb13421b40 0x55fb13429940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e41ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 --2- 172.21.15.64:0/1064900755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb134205d0 0x55fb13429400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 >> 172.21.15.64:0/1064900755 conn(0x55fb133e9310 msgr2=0x55fb1342b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 shutdown_connections 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 1 -- 172.21.15.64:0/1064900755 wait complete. 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.902+0000 7f3e62b5b740 20 client.16917 trim_cache size 0 max 16384 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3e440a33b0 msgr2=0x7f3e440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3e440a33b0 0x7f3e440a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cd900 tx=0x7f3e5006b0d0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 msgr2=0x7f3e5819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 0x7f3e5819cc20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000cce0 tx=0x7f3e5000a620 comp rx=0 tx=0).stop 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 shutdown_connections 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3e440a33b0 0x7f3e440a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58106a80 0x7f3e5819cc20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e58101420 0x7f3e58198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 --2- 172.21.15.64:0/2864195174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e58100a80 0x7f3e58198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 >> 172.21.15.64:0/2864195174 conn(0x7f3e580fc7a0 msgr2=0x7f3e5806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 shutdown_connections 2024-09-19T05:11:45.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:44.903+0000 7f3e60782640 1 -- 172.21.15.64:0/2864195174 wait complete. 2024-09-19T05:11:45.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:44 smithi064 ceph-mon[30960]: pgmap v178: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:45.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:44 smithi103 ceph-mon[34446]: pgmap v178: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:45.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:44 smithi174 ceph-mon[34488]: pgmap v178: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:45.884 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:11:46.143 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:46.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 -- 172.21.15.64:0/1850971456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640100e20 msgr2=0x7f3640101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 --2- 172.21.15.64:0/1850971456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640100e20 0x7f3640101200 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f363002a480 comp rx=0 tx=0).stop 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 -- 172.21.15.64:0/1850971456 shutdown_connections 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 --2- 172.21.15.64:0/1850971456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102ef0 0x7f364010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 --2- 172.21.15.64:0/1850971456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3640102570 0x7f36401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 --2- 172.21.15.64:0/1850971456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640100e20 0x7f3640101200 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 -- 172.21.15.64:0/1850971456 >> 172.21.15.64:0/1850971456 conn(0x7f36400fc7d0 msgr2=0x7f36400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 -- 172.21.15.64:0/1850971456 shutdown_connections 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.570+0000 7f36469f4640 1 -- 172.21.15.64:0/1850971456 wait complete. 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 Processor -- start 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 -- start start 2024-09-19T05:11:46.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3640100e20 0x7f364019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102570 0x7f364019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 0x7f364019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3640111f50 con 0x7f3640102ef0 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3640111dd0 con 0x7f3640102570 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f36469f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36401120d0 con 0x7f3640100e20 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f3644f91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 0x7f364019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102570 0x7f364019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f3644f91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 0x7f364019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37210/0 (socket says 172.21.15.64:37210) 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.571+0000 7f3644f91640 1 -- 172.21.15.64:0/1167369591 learned_addr learned my addr 172.21.15.64:0/1167369591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f363ffff640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3640100e20 0x7f364019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f3644f91640 1 -- 172.21.15.64:0/1167369591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3640100e20 msgr2=0x7f364019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f3644f91640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3640100e20 0x7f364019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f3644f91640 1 -- 172.21.15.64:0/1167369591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102570 msgr2=0x7f364019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f3644f91640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102570 0x7f364019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f3644f91640 1 -- 172.21.15.64:0/1167369591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364019f590 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f363f7fe640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102570 0x7f364019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f3644f91640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 0x7f364019eeb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f363800cce0 tx=0x7f363800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f363d7fa640 1 -- 172.21.15.64:0/1167369591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800f450 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f363d7fa640 1 -- 172.21.15.64:0/1167369591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f363800faa0 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364019f880 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f363d7fa640 1 -- 172.21.15.64:0/1167369591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638018b70 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.572+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401a7160 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363d7fa640 1 -- 172.21.15.64:0/1167369591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3638018d10 con 0x7f3640102ef0 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 Processor -- start 2024-09-19T05:11:46.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 -- start start 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09b9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09ba8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564e09b012f0 con 0x564e09b9b5d0 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e09aa8cf0 con 0x564e09b9cb40 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f3648dcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e09afd2b0 con 0x564e09ba6370 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09b9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09b9ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37212/0 (socket says 172.21.15.64:37212) 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363ffff640 1 -- 172.21.15.64:0/3643207656 learned_addr learned my addr 172.21.15.64:0/3643207656 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363f7fe640 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363d7fa640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36280a3430 0x7f36280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.575+0000 7f363f7fe640 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09ba8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363ffff640 1 -- 172.21.15.64:0/3643207656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 msgr2=0x564e09ba8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363ffff640 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09ba8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363ffff640 1 -- 172.21.15.64:0/3643207656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 msgr2=0x564e09b9b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363ffff640 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363ffff640 1 -- 172.21.15.64:0/3643207656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e09a8d3a0 con 0x564e09b9b5d0 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f3644f91640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36280a3430 0x7f36280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363f7fe640 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09ba8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:46.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363d7fa640 1 -- 172.21.15.64:0/1167369591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f36380d0080 con 0x7f3640102ef0 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f363ffff640 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09b9ba10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f3630031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32227a8422e7bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f36267fc640 1 -- 172.21.15.64:0/3643207656 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630040070 con 0x564e09b9b5d0 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f36267fc640 1 -- 172.21.15.64:0/3643207656 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3630031630 con 0x564e09b9b5d0 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.576+0000 7f36267fc640 1 -- 172.21.15.64:0/3643207656 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630035040 con 0x564e09b9b5d0 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 -- 172.21.15.64:0/3643207656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 msgr2=0x564e09b9ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3644f91640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36280a3430 0x7f36280a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f36380cddd0 tx=0x7f363806f190 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09b9ba10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f3630031c60 comp rx=0 tx=0).stop 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 -- 172.21.15.64:0/3643207656 shutdown_connections 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09ba8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09b9ba10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3643207656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 -- 172.21.15.64:0/3643207656 >> 172.21.15.64:0/3643207656 conn(0x564e09b64310 msgr2=0x564e09b97c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 -- 172.21.15.64:0/3643207656 shutdown_connections 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 -- 172.21.15.64:0/3643207656 wait complete. 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 -- start start 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.577+0000 7f3648dcd740 1 Processor -- start 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09ba5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09b9cb40 0x564e09ba2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 0x564e09ba3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564e09ba9010 con 0x564e09ba6370 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e09abc060 con 0x564e09b9b5d0 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e09ba9190 con 0x564e09b9cb40 2024-09-19T05:11:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f3648dcd740 10 client.0 _ll_register_callbacks cb 0x564e0980f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 0x564e09ba3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 0x564e09ba3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37218/0 (socket says 172.21.15.64:37218) 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 -- 172.21.15.64:0/2449097858 learned_addr learned my addr 172.21.15.64:0/2449097858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363ffff640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09b9cb40 0x564e09ba2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09ba5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 -- 172.21.15.64:0/2449097858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09b9cb40 msgr2=0x564e09ba2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09b9cb40 0x564e09ba2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 -- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 msgr2=0x564e09ba5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09ba5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 -- 172.21.15.64:0/2449097858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e098723c0 con 0x564e09ba6370 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f363f7fe640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 0x564e09ba3480 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3634003970 tx=0x7f363400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53265bf4aa260e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f361f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:46.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.578+0000 7f361f7fe640 1 -- 172.21.15.64:0/2449097858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363401b040 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f361f7fe640 1 -- 172.21.15.64:0/2449097858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3634012d00 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3630031630 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f361f7fe640 1 -- 172.21.15.64:0/2449097858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3634020420 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 10 client.16929 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e09bb49a0 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f361f7fe640 1 -- 172.21.15.64:0/2449097858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3630031630 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f361f7fe640 1 -- 172.21.15.64:0/2449097858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36340296b0 con 0x564e09ba6370 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 10 client.16929 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 20 client.16929 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09bb5db0 0x564e09bb81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 4 client.16929 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:46.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e09b0e860 con 0x564e09bb5db0 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.579+0000 7f363ffff640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09bb5db0 0x564e09bb81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.580+0000 7f363ffff640 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09bb5db0 0x564e09bb81a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9df4230599ebda server_cookie=94675fc1cbe457d9 in_seq=0 out_seq=0 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.580+0000 7f361f7fe640 10 client.16929 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f361f7fe640 1 -- 172.21.15.64:0/2449097858 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e09b0e860 con 0x564e09bb5db0 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f361f7fe640 10 client.16929 handle_command_reply: tid=0 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 client.16929 shutdown 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 msgr2=0x564e09ba3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 0x564e09ba3480 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3634003970 tx=0x7f363400bc80 comp rx=0 tx=0).stop 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 shutdown_connections 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09bb5db0 0x564e09bb81a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09ba6370 0x564e09ba3480 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09b9cb40 0x564e09ba2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 --2- 172.21.15.64:0/2449097858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09ba5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f361f7fe640 0 client.16929 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:46.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 >> 172.21.15.64:0/2449097858 conn(0x564e09b64310 msgr2=0x564e09b9d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 shutdown_connections 2024-09-19T05:11:46.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 1 -- 172.21.15.64:0/2449097858 wait complete. 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.581+0000 7f3648dcd740 20 client.16929 trim_cache size 0 max 16384 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 Processor -- start 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 -- start start 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09ba4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09b9e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36401b2280 con 0x564e09b9b5d0 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36401afb50 con 0x564e09b9cb40 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f3648dcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36401ad9a0 con 0x564e09ba6370 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09ba4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09ba4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37220/0 (socket says 172.21.15.64:37220) 2024-09-19T05:11:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 -- 172.21.15.64:0/3502650476 learned_addr learned my addr 172.21.15.64:0/3502650476 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363ffff640 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 -- 172.21.15.64:0/3502650476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 msgr2=0x564e09b9e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09b9e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 -- 172.21.15.64:0/3502650476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 msgr2=0x564e09b9e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.594+0000 7f363f7fe640 1 -- 172.21.15.64:0/3502650476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e09ba1ef0 con 0x564e09b9b5d0 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f363f7fe640 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09ba4eb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f363400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90439998cb2610c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3625ffb640 1 -- 172.21.15.64:0/3502650476 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634029ca0 con 0x564e09b9b5d0 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3625ffb640 1 -- 172.21.15.64:0/3502650476 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3634029e40 con 0x564e09b9b5d0 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3625ffb640 1 -- 172.21.15.64:0/3502650476 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634005360 con 0x564e09b9b5d0 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 -- 172.21.15.64:0/3502650476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 msgr2=0x564e09ba4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09ba4eb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f363400be70 comp rx=0 tx=0).stop 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 -- 172.21.15.64:0/3502650476 shutdown_connections 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09b9e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9cb40 0x564e09b9e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 --2- 172.21.15.64:0/3502650476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9b5d0 0x564e09ba4eb0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 -- 172.21.15.64:0/3502650476 >> 172.21.15.64:0/3502650476 conn(0x564e09b64310 msgr2=0x564e09ba47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 -- 172.21.15.64:0/3502650476 shutdown_connections 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.595+0000 7f3648dcd740 1 -- 172.21.15.64:0/3502650476 wait complete. 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 -- start start 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 Processor -- start 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09baa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 0x564e09baa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09baab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36401ae010 con 0x564e09b9cb40 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36401af480 con 0x564e09b9b5d0 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36401ae6e0 con 0x564e09ba6370 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f3648dcd740 10 client.0 _ll_register_callbacks cb 0x7f36340035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f363ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 0x564e09baa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f363ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 0x564e09baa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37234/0 (socket says 172.21.15.64:37234) 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f363ffff640 1 -- 172.21.15.64:0/355041850 learned_addr learned my addr 172.21.15.64:0/355041850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.596+0000 7f363f7fe640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09baa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363ffff640 1 -- 172.21.15.64:0/355041850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 msgr2=0x564e09baab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363ffff640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09baab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363ffff640 1 -- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 msgr2=0x564e09baa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363ffff640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09baa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363ffff640 1 -- 172.21.15.64:0/355041850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e09c1f9f0 con 0x564e09b9cb40 2024-09-19T05:11:46.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363f7fe640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09baa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f363ffff640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 0x564e09baa610 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f3630000c00 tx=0x7f3630031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627ef760f14b90ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f361f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f361f7fe640 1 -- 172.21.15.64:0/355041850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630054060 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f361f7fe640 1 -- 172.21.15.64:0/355041850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f3630005430 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e09b08d70 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f361f7fe640 1 -- 172.21.15.64:0/355041850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3630040030 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 10 client.16935 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e09c20c30 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f361f7fe640 1 -- 172.21.15.64:0/355041850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564e09b08d70 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f361f7fe640 1 -- 172.21.15.64:0/355041850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3630003920 con 0x564e09b9cb40 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 10 client.16935 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:46.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 20 client.16935 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.597+0000 7f3648dcd740 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09c21030 0x564e09c23420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.598+0000 7f3648dcd740 4 client.16935 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.598+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e09a85170 con 0x564e09c21030 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.598+0000 7f363f7fe640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09c21030 0x564e09c23420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.598+0000 7f361f7fe640 10 client.16935 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.598+0000 7f363f7fe640 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09c21030 0x564e09c23420 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a063924ed0a927e6 server_cookie=b649dc6a7d26fe75 in_seq=0 out_seq=0 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f361f7fe640 1 -- 172.21.15.64:0/355041850 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e09a85170 con 0x564e09c21030 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f361f7fe640 10 client.16935 handle_command_reply: tid=0 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 client.16935 shutdown 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 msgr2=0x564e09baa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 0x564e09baa610 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f3630000c00 tx=0x7f3630031f10 comp rx=0 tx=0).stop 2024-09-19T05:11:46.640 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:11:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 shutdown_connections 2024-09-19T05:11:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e09ba6370 0x564e09baab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x564e09c21030 0x564e09c23420 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e09b9cb40 0x564e09baa610 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 --2- 172.21.15.64:0/355041850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e09b9b5d0 0x564e09baa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f361f7fe640 0 client.16935 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 >> 172.21.15.64:0/355041850 conn(0x564e09b64310 msgr2=0x564e09ba4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 shutdown_connections 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 1 -- 172.21.15.64:0/355041850 wait complete. 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f3648dcd740 20 client.16935 trim_cache size 0 max 16384 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.599+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36280a3430 msgr2=0x7f36280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36280a3430 0x7f36280a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f36380cddd0 tx=0x7f363806f190 comp rx=0 tx=0).stop 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 msgr2=0x7f364019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 0x7f364019eeb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f363800cce0 tx=0x7f363800a620 comp rx=0 tx=0).stop 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 shutdown_connections 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f36280a3430 0x7f36280a58f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3640102ef0 0x7f364019eeb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3640102570 0x7f364019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 --2- 172.21.15.64:0/1167369591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3640100e20 0x7f364019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 >> 172.21.15.64:0/1167369591 conn(0x7f36400fc7d0 msgr2=0x7f36400fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 shutdown_connections 2024-09-19T05:11:46.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:46.600+0000 7f36469f4640 1 -- 172.21.15.64:0/1167369591 wait complete. 2024-09-19T05:11:47.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:46 smithi064 ceph-mon[30960]: pgmap v179: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:47.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:46 smithi103 ceph-mon[34446]: pgmap v179: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:47.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:46 smithi174 ceph-mon[34488]: pgmap v179: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:48.503 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:11:48.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:11:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:49.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:48 smithi064 ceph-mon[30960]: pgmap v180: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:49.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:48 smithi103 ceph-mon[34446]: pgmap v180: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 -- 172.21.15.64:0/2306394233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224100e20 msgr2=0x7fe224101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/2306394233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224100e20 0x7fe224101200 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe218009ca0 tx=0x7fe21802a950 comp rx=0 tx=0).stop 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 -- 172.21.15.64:0/2306394233 shutdown_connections 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/2306394233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe224102ef0 0x7fe22410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/2306394233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224102570 0x7fe2241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/2306394233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224100e20 0x7fe224101200 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 -- 172.21.15.64:0/2306394233 >> 172.21.15.64:0/2306394233 conn(0x7fe2240fc7d0 msgr2=0x7fe2240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 -- 172.21.15.64:0/2306394233 shutdown_connections 2024-09-19T05:11:49.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 -- 172.21.15.64:0/2306394233 wait complete. 2024-09-19T05:11:49.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 Processor -- start 2024-09-19T05:11:49.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.241+0000 7fe22c1bf640 1 -- start start 2024-09-19T05:11:49.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22c1bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224100e20 0x7fe2241972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22c1bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 0x7fe224197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22c1bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe224102ef0 0x7fe22419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22c1bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe224110e30 con 0x7fe224102570 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22c1bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe224110cb0 con 0x7fe224100e20 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22c1bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe224110fb0 con 0x7fe224102ef0 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 0x7fe224197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 0x7fe224197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37250/0 (socket says 172.21.15.64:37250) 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 -- 172.21.15.64:0/479045608 learned_addr learned my addr 172.21.15.64:0/479045608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe229f5b640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224100e20 0x7fe2241972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22a75c640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe224102ef0 0x7fe22419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 -- 172.21.15.64:0/479045608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe224102ef0 msgr2=0x7fe22419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe224102ef0 0x7fe22419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 -- 172.21.15.64:0/479045608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224100e20 msgr2=0x7fe2241972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224100e20 0x7fe2241972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 -- 172.21.15.64:0/479045608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22419c280 con 0x7fe224102570 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe229f5b640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224100e20 0x7fe2241972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe22975a640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 0x7fe224197810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe22000bea0 tx=0x7fe22000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.242+0000 7fe216ffd640 1 -- 172.21.15.64:0/479045608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220016070 con 0x7fe224102570 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.243+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22419c570 con 0x7fe224102570 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.243+0000 7fe216ffd640 1 -- 172.21.15.64:0/479045608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe220009c30 con 0x7fe224102570 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.243+0000 7fe216ffd640 1 -- 172.21.15.64:0/479045608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220019730 con 0x7fe224102570 2024-09-19T05:11:49.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.243+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe224069f70 con 0x7fe224102570 2024-09-19T05:11:49.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.245+0000 7fe216ffd640 1 -- 172.21.15.64:0/479045608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe220010560 con 0x7fe224102570 2024-09-19T05:11:49.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.245+0000 7fe22e598740 1 Processor -- start 2024-09-19T05:11:49.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 -- start start 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a28e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a37c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a3a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56049798e280 con 0x560497a29340 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560497935b10 con 0x560497a28a20 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22e598740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56049798a240 con 0x560497a38150 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a37c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a37c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37266/0 (socket says 172.21.15.64:37266) 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 -- 172.21.15.64:0/3581696255 learned_addr learned my addr 172.21.15.64:0/3581696255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:49.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe216ffd640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2100a33b0 0x7fe2100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22975a640 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a3a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22a75c640 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a28e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 -- 172.21.15.64:0/3581696255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 msgr2=0x560497a3a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a3a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 -- 172.21.15.64:0/3581696255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 msgr2=0x560497a28e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe229f5b640 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a28e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.246+0000 7fe22a75c640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2100a33b0 0x7fe2100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe229f5b640 1 -- 172.21.15.64:0/3581696255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604979421f0 con 0x560497a29340 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe216ffd640 1 -- 172.21.15.64:0/479045608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe2200d19d0 con 0x7fe224102570 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe229f5b640 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a37c10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe21802fec0 tx=0x7fe2180093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6166a190ac59516c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:49.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe20ffff640 1 -- 172.21.15.64:0/3581696255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218034040 con 0x560497a29340 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe22a75c640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2100a33b0 0x7fe2100a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005fd0 tx=0x7fe21c0046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe20ffff640 1 -- 172.21.15.64:0/3581696255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21802fb40 con 0x560497a29340 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.247+0000 7fe20ffff640 1 -- 172.21.15.64:0/3581696255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218004570 con 0x560497a29340 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 -- 172.21.15.64:0/3581696255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 msgr2=0x560497a37c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a37c10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe21802fec0 tx=0x7fe2180093f0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 -- 172.21.15.64:0/3581696255 shutdown_connections 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a3a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a37c10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 --2- 172.21.15.64:0/3581696255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a28e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 -- 172.21.15.64:0/3581696255 >> 172.21.15.64:0/3581696255 conn(0x560497a250f0 msgr2=0x560497a254d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 -- 172.21.15.64:0/3581696255 shutdown_connections 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 -- 172.21.15.64:0/3581696255 wait complete. 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 -- start start 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.248+0000 7fe22e598740 1 Processor -- start 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a35b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560497a3acc0 con 0x560497a29340 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560497948f10 con 0x560497a28a20 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560497a3ae40 con 0x560497a38150 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22e598740 10 client.0 _ll_register_callbacks cb 0x560497911a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37274/0 (socket says 172.21.15.64:37274) 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 -- 172.21.15.64:0/3118920097 learned_addr learned my addr 172.21.15.64:0/3118920097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:49.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe229f5b640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22a75c640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a35b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 -- 172.21.15.64:0/3118920097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 msgr2=0x560497a35b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a35b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 -- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 msgr2=0x560497a350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 -- 172.21.15.64:0/3118920097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604976ff6a0 con 0x560497a29340 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22a75c640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a35b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe22975a640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a355e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe2200cf230 tx=0x7fe220068f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7ba61195bba550 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.249+0000 7fe20d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe20d7fa640 1 -- 172.21.15.64:0/3118920097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220009de0 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe20d7fa640 1 -- 172.21.15.64:0/3118920097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe220010aa0 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21802fb40 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe20d7fa640 1 -- 172.21.15.64:0/3118920097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe22009dd60 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 10 client.16947 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560497a46980 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe20d7fa640 1 -- 172.21.15.64:0/3118920097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21802fb40 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe20d7fa640 1 -- 172.21.15.64:0/3118920097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe2200cf340 con 0x560497a29340 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 10 client.16947 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 20 client.16947 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497a47e50 0x560497a4a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 4 client.16947 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.250+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56049799b7f0 con 0x560497a47e50 2024-09-19T05:11:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.251+0000 7fe229f5b640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497a47e50 0x560497a4a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.251+0000 7fe229f5b640 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497a47e50 0x560497a4a240 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589b1e307d4aed01 server_cookie=b17090ba649c089c in_seq=0 out_seq=0 2024-09-19T05:11:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.251+0000 7fe20d7fa640 10 client.16947 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe20d7fa640 1 -- 172.21.15.64:0/3118920097 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56049799b7f0 con 0x560497a47e50 2024-09-19T05:11:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe20d7fa640 10 client.16947 handle_command_reply: tid=0 2024-09-19T05:11:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 client.16947 shutdown 2024-09-19T05:11:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 msgr2=0x560497a355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a355e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe2200cf230 tx=0x7fe220068f00 comp rx=0 tx=0).stop 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 shutdown_connections 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a35b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497a47e50 0x560497a4a240 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a29340 0x560497a355e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 --2- 172.21.15.64:0/3118920097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a28a20 0x560497a350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 >> 172.21.15.64:0/3118920097 conn(0x560497a250f0 msgr2=0x560497a2ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe20d7fa640 0 client.16947 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 shutdown_connections 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 1 -- 172.21.15.64:0/3118920097 wait complete. 2024-09-19T05:11:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.252+0000 7fe22e598740 20 client.16947 trim_cache size 0 max 16384 2024-09-19T05:11:49.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.269+0000 7fe22e598740 1 Processor -- start 2024-09-19T05:11:49.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.269+0000 7fe22e598740 1 -- start start 2024-09-19T05:11:49.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22e598740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a36f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22e598740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a29340 0x560497a4fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22e598740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a50040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22e598740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241adeb0 con 0x560497a28a20 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22e598740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241ab780 con 0x560497a29340 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22e598740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241a95d0 con 0x560497a38150 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a36f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a36f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37282/0 (socket says 172.21.15.64:37282) 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 -- 172.21.15.64:0/2707155160 learned_addr learned my addr 172.21.15.64:0/2707155160 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22975a640 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a29340 0x560497a4fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe22a75c640 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a50040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 -- 172.21.15.64:0/2707155160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 msgr2=0x560497a50040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a50040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 -- 172.21.15.64:0/2707155160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a29340 msgr2=0x560497a4fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a29340 0x560497a4fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 -- 172.21.15.64:0/2707155160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560497a1f680 con 0x560497a28a20 2024-09-19T05:11:49.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe229f5b640 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a36f00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe218004120 tx=0x7fe21802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7c45c631a120d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe20f7fe640 1 -- 172.21.15.64:0/2707155160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218034040 con 0x560497a28a20 2024-09-19T05:11:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe20f7fe640 1 -- 172.21.15.64:0/2707155160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21802f9c0 con 0x560497a28a20 2024-09-19T05:11:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.270+0000 7fe20f7fe640 1 -- 172.21.15.64:0/2707155160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218004420 con 0x560497a28a20 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 -- 172.21.15.64:0/2707155160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 msgr2=0x560497a36f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a36f00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe218004120 tx=0x7fe21802a950 comp rx=0 tx=0).stop 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 -- 172.21.15.64:0/2707155160 shutdown_connections 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a38150 0x560497a50040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a29340 0x560497a4fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 --2- 172.21.15.64:0/2707155160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a36f00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 -- 172.21.15.64:0/2707155160 >> 172.21.15.64:0/2707155160 conn(0x560497a250f0 msgr2=0x560497a367f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 -- 172.21.15.64:0/2707155160 shutdown_connections 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 -- 172.21.15.64:0/2707155160 wait complete. 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.271+0000 7fe22e598740 1 -- start start 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 Processor -- start 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a3c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a29340 0x560497a3ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a38150 0x560497ab1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241a9c40 con 0x560497a28a20 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241ab0b0 con 0x560497a38150 2024-09-19T05:11:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241aa310 con 0x560497a29340 2024-09-19T05:11:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22e598740 10 client.0 _ll_register_callbacks cb 0x7fe22009a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a3c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a3c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37298/0 (socket says 172.21.15.64:37298) 2024-09-19T05:11:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 -- 172.21.15.64:0/746666119 learned_addr learned my addr 172.21.15.64:0/746666119 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22a75c640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a38150 0x560497ab1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22975a640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a29340 0x560497a3ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 -- 172.21.15.64:0/746666119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a29340 msgr2=0x560497a3ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a29340 0x560497a3ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 -- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a38150 msgr2=0x560497ab1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a38150 0x560497ab1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe229f5b640 1 -- 172.21.15.64:0/746666119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560497ab2010 con 0x560497a28a20 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22975a640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a29340 0x560497a3ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.272+0000 7fe22a75c640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a38150 0x560497ab1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe229f5b640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a3c4c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe21802fef0 tx=0x7fe218009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a10d85aa79784e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe20d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe20d7fa640 1 -- 172.21.15.64:0/746666119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21805b020 con 0x560497a28a20 2024-09-19T05:11:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe20d7fa640 1 -- 172.21.15.64:0/746666119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fe218004150 con 0x560497a28a20 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21802f9c0 con 0x560497a28a20 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe20d7fa640 1 -- 172.21.15.64:0/746666119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe218066050 con 0x560497a28a20 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 10 client.16953 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560497ab32b0 con 0x560497a28a20 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe20d7fa640 1 -- 172.21.15.64:0/746666119 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21802f9c0 con 0x560497a28a20 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe20d7fa640 1 -- 172.21.15.64:0/746666119 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe218042070 con 0x560497a28a20 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 10 client.16953 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 20 client.16953 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497ab35f0 0x560497ab59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.273+0000 7fe22e598740 4 client.16953 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.274+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560497995d00 con 0x560497ab35f0 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.274+0000 7fe22975a640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497ab35f0 0x560497ab59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.274+0000 7fe22975a640 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497ab35f0 0x560497ab59e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d57b4003c775dee server_cookie=37eb3a9408f96770 in_seq=0 out_seq=0 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.274+0000 7fe20d7fa640 10 client.16953 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe20d7fa640 1 -- 172.21.15.64:0/746666119 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560497995d00 con 0x560497ab35f0 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe20d7fa640 10 client.16953 handle_command_reply: tid=0 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 client.16953 shutdown 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 msgr2=0x560497a3c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a3c4c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe21802fef0 tx=0x7fe218009a20 comp rx=0 tx=0).stop 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 shutdown_connections 2024-09-19T05:11:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560497a38150 0x560497ab1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560497a29340 0x560497a3ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560497ab35f0 0x560497ab59e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 --2- 172.21.15.64:0/746666119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560497a28a20 0x560497a3c4c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe20d7fa640 0 client.16953 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 >> 172.21.15.64:0/746666119 conn(0x560497a250f0 msgr2=0x560497a24270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 shutdown_connections 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.275+0000 7fe22e598740 1 -- 172.21.15.64:0/746666119 wait complete. 2024-09-19T05:11:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22e598740 20 client.16953 trim_cache size 0 max 16384 2024-09-19T05:11:49.284 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:49.284 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:11:49.284 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "23428ee8-1609-492c-9f44-e0ef067d4f21", 2024-09-19T05:11:49.284 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:11:49.284 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2100a33b0 msgr2=0x7fe2100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2100a33b0 0x7fe2100a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005fd0 tx=0x7fe21c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 msgr2=0x7fe224197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 0x7fe224197810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe22000bea0 tx=0x7fe22000bf90 comp rx=0 tx=0).stop 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 shutdown_connections 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2100a33b0 0x7fe2100a5870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe224102ef0 0x7fe22419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224102570 0x7fe224197810 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 --2- 172.21.15.64:0/479045608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe224100e20 0x7fe2241972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 >> 172.21.15.64:0/479045608 conn(0x7fe2240fc7d0 msgr2=0x7fe2240fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 shutdown_connections 2024-09-19T05:11:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:49.276+0000 7fe22c1bf640 1 -- 172.21.15.64:0/479045608 wait complete. 2024-09-19T05:11:49.286 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:48 smithi174 ceph-mon[34488]: pgmap v180: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:50.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:50.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:49 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:11:50.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:49 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:50.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:49 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:11:50.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:49 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:50.277 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:50.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:49 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:11:50.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:49 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:50.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:49 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:11:50.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:49 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:50.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:49 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:11:50.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:49 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:50.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:49 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:11:50.398 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:49 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 -- 172.21.15.64:0/1174565890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438102790 msgr2=0x7f04381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 --2- 172.21.15.64:0/1174565890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438102790 0x7f04381067e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0420009900 tx=0x7f042002a410 comp rx=0 tx=0).stop 2024-09-19T05:11:50.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 -- 172.21.15.64:0/1174565890 shutdown_connections 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 --2- 172.21.15.64:0/1174565890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0438106f10 0x7f043810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 --2- 172.21.15.64:0/1174565890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438102790 0x7f04381067e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 --2- 172.21.15.64:0/1174565890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0438101de0 0x7f04381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 -- 172.21.15.64:0/1174565890 >> 172.21.15.64:0/1174565890 conn(0x7f04380fd8c0 msgr2=0x7f04380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 -- 172.21.15.64:0/1174565890 shutdown_connections 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 -- 172.21.15.64:0/1174565890 wait complete. 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 Processor -- start 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.755+0000 7f043d659640 1 -- start start 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f043d659640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0438101de0 0x7f04381983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f043d659640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0438102790 0x7f0438198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f043d659640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 0x7f043819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f043d659640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0438111e40 con 0x7f0438106f10 2024-09-19T05:11:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f043d659640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0438111cc0 con 0x7f0438102790 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f043d659640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0438111fc0 con 0x7f0438101de0 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 0x7f043819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 0x7f043819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37316/0 (socket says 172.21.15.64:37316) 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 -- 172.21.15.64:0/2521728732 learned_addr learned my addr 172.21.15.64:0/2521728732 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04367fc640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0438102790 0x7f0438198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f0436ffd640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0438101de0 0x7f04381983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 -- 172.21.15.64:0/2521728732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0438101de0 msgr2=0x7f04381983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0438101de0 0x7f04381983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 -- 172.21.15.64:0/2521728732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0438102790 msgr2=0x7f0438198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0438102790 0x7f0438198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 -- 172.21.15.64:0/2521728732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043819d370 con 0x7f0438106f10 2024-09-19T05:11:50.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.756+0000 7f04377fe640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 0x7f043819cc90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f042800b970 tx=0x7f042800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.757+0000 7f042ffff640 1 -- 172.21.15.64:0/2521728732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428019070 con 0x7f0438106f10 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.757+0000 7f042ffff640 1 -- 172.21.15.64:0/2521728732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0428014440 con 0x7f0438106f10 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.757+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043819d660 con 0x7f0438106f10 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.757+0000 7f042ffff640 1 -- 172.21.15.64:0/2521728732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04280075d0 con 0x7f0438106f10 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.757+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a4f40 con 0x7f0438106f10 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.759+0000 7f042ffff640 1 -- 172.21.15.64:0/2521728732 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f042801c070 con 0x7f0438106f10 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.759+0000 7f043fa32740 1 Processor -- start 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.759+0000 7f043fa32740 1 -- start start 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f043fa32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e54a0 0x558dc61e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f043fa32740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e5dc0 0x558dc61f0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f043fa32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 0x558dc61f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f043fa32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558dc614b2e0 con 0x558dc61f0680 2024-09-19T05:11:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f043fa32740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558dc60f2ce0 con 0x558dc61e5dc0 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f043fa32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558dc61472a0 con 0x558dc61e54a0 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 0x558dc61f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 0x558dc61f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37322/0 (socket says 172.21.15.64:37322) 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 -- 172.21.15.64:0/1259716452 learned_addr learned my addr 172.21.15.64:0/1259716452 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e54a0 0x558dc61e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f0436ffd640 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e5dc0 0x558dc61f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 -- 172.21.15.64:0/1259716452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e54a0 msgr2=0x558dc61e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e54a0 0x558dc61e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 -- 172.21.15.64:0/1259716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e5dc0 msgr2=0x558dc61f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e5dc0 0x558dc61f0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f042ffff640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04240a33b0 0x7f04240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 -- 172.21.15.64:0/1259716452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc60d7440 con 0x558dc61f0680 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04377fe640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04240a33b0 0x7f04240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.760+0000 7f04367fc640 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 0x558dc61f2a70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f042002a930 tx=0x7f0420005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b611b41db066de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f042d7fa640 1 -- 172.21.15.64:0/1259716452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420035070 con 0x558dc61f0680 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f042ffff640 1 -- 172.21.15.64:0/2521728732 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f04280cf0f0 con 0x7f0438106f10 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f042d7fa640 1 -- 172.21.15.64:0/1259716452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04200035f0 con 0x558dc61f0680 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f042d7fa640 1 -- 172.21.15.64:0/1259716452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200044e0 con 0x558dc61f0680 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f043fa32740 1 -- 172.21.15.64:0/1259716452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 msgr2=0x558dc61f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f043fa32740 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 0x558dc61f2a70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f042002a930 tx=0x7f0420005db0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.761+0000 7f04377fe640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04240a33b0 0x7f04240a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f04280ccaa0 tx=0x7f0428004b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- 172.21.15.64:0/1259716452 shutdown_connections 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61f0680 0x558dc61f2a70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e5dc0 0x558dc61f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 --2- 172.21.15.64:0/1259716452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e54a0 0x558dc61e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- 172.21.15.64:0/1259716452 >> 172.21.15.64:0/1259716452 conn(0x558dc61e1a30 msgr2=0x558dc61e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- 172.21.15.64:0/1259716452 shutdown_connections 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- 172.21.15.64:0/1259716452 wait complete. 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- start start 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 Processor -- start 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e54a0 0x558dc61eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 0x558dc61ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61f0680 0x558dc61ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558dc61f3250 con 0x558dc61e5dc0 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558dc6106050 con 0x558dc61e54a0 2024-09-19T05:11:50.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558dc61f33d0 con 0x558dc61f0680 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f043fa32740 10 client.0 _ll_register_callbacks cb 0x558dc6146160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 0x558dc61ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 0x558dc61ed200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37332/0 (socket says 172.21.15.64:37332) 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.762+0000 7f0436ffd640 1 -- 172.21.15.64:0/2692438294 learned_addr learned my addr 172.21.15.64:0/2692438294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f04367fc640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e54a0 0x558dc61eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f04367fc640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61f0680 0x558dc61ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f0436ffd640 1 -- 172.21.15.64:0/2692438294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61f0680 msgr2=0x558dc61ed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f0436ffd640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61f0680 0x558dc61ed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f0436ffd640 1 -- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e54a0 msgr2=0x558dc61eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f0436ffd640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e54a0 0x558dc61eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f0436ffd640 1 -- 172.21.15.64:0/2692438294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc5ebc3c0 con 0x558dc61e5dc0 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f0436ffd640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 0x558dc61ed200 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f043000bed0 tx=0x7f04300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3ca397073f44e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f041e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f041e7fc640 1 -- 172.21.15.64:0/2692438294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430004220 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f041e7fc640 1 -- 172.21.15.64:0/2692438294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f04300043c0 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.763+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04200035f0 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f041e7fc640 1 -- 172.21.15.64:0/2692438294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04300035f0 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 10 client.16965 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558dc61fec00 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f041e7fc640 1 -- 172.21.15.64:0/2692438294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04200035f0 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f041e7fc640 1 -- 172.21.15.64:0/2692438294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f043001b8c0 con 0x558dc61e5dc0 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 10 client.16965 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 20 client.16965 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc62000c0 0x558dc6202510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 4 client.16965 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc6158850 con 0x558dc62000c0 2024-09-19T05:11:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.764+0000 7f04367fc640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc62000c0 0x558dc6202510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.765+0000 7f041e7fc640 10 client.16965 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.765+0000 7f04367fc640 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc62000c0 0x558dc6202510 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8cd0de7998988e server_cookie=a1bcf88d73d2b2d9 in_seq=0 out_seq=0 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.765+0000 7f041e7fc640 1 -- 172.21.15.64:0/2692438294 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dc6158850 con 0x558dc62000c0 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.765+0000 7f041e7fc640 10 client.16965 handle_command_reply: tid=0 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 client.16965 shutdown 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 msgr2=0x558dc61ed200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 0x558dc61ed200 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f043000bed0 tx=0x7f04300117b0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 shutdown_connections 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61f0680 0x558dc61ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc62000c0 0x558dc6202510 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e5dc0 0x558dc61ed200 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 --2- 172.21.15.64:0/2692438294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e54a0 0x558dc61eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f041e7fc640 0 client.16965 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 >> 172.21.15.64:0/2692438294 conn(0x558dc61e1a30 msgr2=0x558dc61e7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 shutdown_connections 2024-09-19T05:11:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 1 -- 172.21.15.64:0/2692438294 wait complete. 2024-09-19T05:11:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.766+0000 7f043fa32740 20 client.16965 trim_cache size 0 max 16384 2024-09-19T05:11:50.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 Processor -- start 2024-09-19T05:11:50.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 -- start start 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04381b01f0 con 0x558dc61e54a0 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04381adac0 con 0x558dc61e1a30 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f043fa32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04381ab910 con 0x558dc61e5dc0 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37346/0 (socket says 172.21.15.64:37346) 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.783+0000 7f0436ffd640 1 -- 172.21.15.64:0/3358215899 learned_addr learned my addr 172.21.15.64:0/3358215899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f04367fc640 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f04367fc640 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f0436ffd640 1 -- 172.21.15.64:0/3358215899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 msgr2=0x558dc61e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f0436ffd640 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f0436ffd640 1 -- 172.21.15.64:0/3358215899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 msgr2=0x558dc61fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f0436ffd640 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61fc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f0436ffd640 1 -- 172.21.15.64:0/3358215899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc61e1860 con 0x558dc61e54a0 2024-09-19T05:11:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f04367fc640 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:11:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f0436ffd640 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61e8900 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f043001bed0 tx=0x7f0430005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c488d354d6391d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f042cff9640 1 -- 172.21.15.64:0/3358215899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430028040 con 0x558dc61e54a0 2024-09-19T05:11:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f042cff9640 1 -- 172.21.15.64:0/3358215899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0430032070 con 0x558dc61e54a0 2024-09-19T05:11:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.784+0000 7f042cff9640 1 -- 172.21.15.64:0/3358215899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300048e0 con 0x558dc61e54a0 2024-09-19T05:11:50.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 -- 172.21.15.64:0/3358215899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 msgr2=0x558dc61e8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61e8900 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f043001bed0 tx=0x7f0430005fd0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 -- 172.21.15.64:0/3358215899 shutdown_connections 2024-09-19T05:11:50.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61e8900 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 --2- 172.21.15.64:0/3358215899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 -- 172.21.15.64:0/3358215899 >> 172.21.15.64:0/3358215899 conn(0x558dc61ed8a0 msgr2=0x558dc61e0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 -- 172.21.15.64:0/3358215899 shutdown_connections 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 -- 172.21.15.64:0/3358215899 wait complete. 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 -- start start 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.785+0000 7f043fa32740 1 Processor -- start 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04381abf80 con 0x558dc61e54a0 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04381ad3f0 con 0x558dc61e1a30 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04381ac650 con 0x558dc61e5dc0 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f043fa32740 10 client.0 _ll_register_callbacks cb 0x7f043000b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37358/0 (socket says 172.21.15.64:37358) 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 -- 172.21.15.64:0/161363874 learned_addr learned my addr 172.21.15.64:0/161363874 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f04367fc640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f04367fc640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 -- 172.21.15.64:0/161363874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 msgr2=0x558dc61f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 -- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 msgr2=0x558dc61f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 -- 172.21.15.64:0/161363874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc61f4e20 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f04367fc640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f0436ffd640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61f40e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f043001cf30 tx=0x7f0430005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907e2c91d549f7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f041e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.786+0000 7f041e7fc640 1 -- 172.21.15.64:0/161363874 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430028070 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f041e7fc640 1 -- 172.21.15.64:0/161363874 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f043001b680 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0430032070 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f041e7fc640 1 -- 172.21.15.64:0/161363874 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0430005100 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 10 client.16971 fetch_fsmap learned FSMap version 25 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558dc626ad60 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f041e7fc640 1 -- 172.21.15.64:0/161363874 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0430032070 con 0x558dc61e54a0 2024-09-19T05:11:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f041e7fc640 1 -- 172.21.15.64:0/161363874 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0430012450 con 0x558dc61e54a0 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 10 client.16971 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 20 client.16971 populate_metadata read hostname 'smithi064' 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc61f0680 0x558dc626d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 4 client.16971 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dc6152d60 con 0x558dc61f0680 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.787+0000 7f04367fc640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc61f0680 0x558dc626d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f04367fc640 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc61f0680 0x558dc626d440 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e1b3156fe2825c server_cookie=1a0cefe92682325f in_seq=0 out_seq=0 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f041e7fc640 10 client.16971 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f041e7fc640 1 -- 172.21.15.64:0/161363874 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558dc6152d60 con 0x558dc61f0680 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f041e7fc640 10 client.16971 handle_command_reply: tid=0 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 client.16971 shutdown 2024-09-19T05:11:50.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 msgr2=0x558dc61f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61f40e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f043001cf30 tx=0x7f0430005970 comp rx=0 tx=0).stop 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 shutdown_connections 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dc61e5dc0 0x558dc61f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558dc61f0680 0x558dc626d440 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc61e54a0 0x558dc61f40e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 --2- 172.21.15.64:0/161363874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc61e1a30 0x558dc61f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 >> 172.21.15.64:0/161363874 conn(0x558dc61ed8a0 msgr2=0x558dc61ef280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f041e7fc640 0 client.16971 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 shutdown_connections 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 1 -- 172.21.15.64:0/161363874 wait complete. 2024-09-19T05:11:50.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.788+0000 7f043fa32740 20 client.16971 trim_cache size 0 max 16384 2024-09-19T05:11:50.797 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:11:50.797 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:11:50.797 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:11:50.797 INFO:teuthology.orchestra.run.smithi064.stdout: "23428ee8-1609-492c-9f44-e0ef067d4f21": { 2024-09-19T05:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "23428ee8-1609-492c-9f44-e0ef067d4f21", 2024-09-19T05:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:11:50.798 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04240a33b0 msgr2=0x7f04240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04240a33b0 0x7f04240a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f04280ccaa0 tx=0x7f0428004b50 comp rx=0 tx=0).stop 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 msgr2=0x7f043819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 0x7f043819cc90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f042800b970 tx=0x7f042800be30 comp rx=0 tx=0).stop 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 shutdown_connections 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04240a33b0 0x7f04240a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0438106f10 0x7f043819cc90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0438102790 0x7f0438198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 --2- 172.21.15.64:0/2521728732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0438101de0 0x7f04381983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 >> 172.21.15.64:0/2521728732 conn(0x7f04380fd8c0 msgr2=0x7f04380fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 shutdown_connections 2024-09-19T05:11:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:11:50.789+0000 7f043d659640 1 -- 172.21.15.64:0/2521728732 wait complete. 2024-09-19T05:11:51.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:50 smithi064 ceph-mon[30960]: pgmap v181: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:51.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:50 smithi103 ceph-mon[34446]: pgmap v181: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:51.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:50 smithi174 ceph-mon[34488]: pgmap v181: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:11:51.487 INFO:tasks.cephfs.filesystem:scrub status for tag:23428ee8-1609-492c-9f44-e0ef067d4f21 - {'path': '/', 'tag': '23428ee8-1609-492c-9f44-e0ef067d4f21', 'options': 'recursive,force'} 2024-09-19T05:11:53.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:52 smithi064 ceph-mon[30960]: pgmap v182: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:53.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:52 smithi103 ceph-mon[34446]: pgmap v182: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:53.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:52 smithi174 ceph-mon[34488]: pgmap v182: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:54.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:53 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:11:54.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:53 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:11:54.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:53 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:11:55.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:54 smithi103 ceph-mon[34446]: pgmap v183: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:55.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:54 smithi174 ceph-mon[34488]: pgmap v183: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:55.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:54 smithi064 ceph-mon[30960]: pgmap v183: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:57.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:56 smithi103 ceph-mon[34446]: pgmap v184: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:57.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:56 smithi174 ceph-mon[34488]: pgmap v184: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:57.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:56 smithi064 ceph-mon[30960]: pgmap v184: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:11:59.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:11:58 smithi103 ceph-mon[34446]: pgmap v185: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:59.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:11:58 smithi174 ceph-mon[34488]: pgmap v185: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:11:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:11:58 smithi064 ceph-mon[30960]: pgmap v185: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:01.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:00 smithi103 ceph-mon[34446]: pgmap v186: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:12:01.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:00 smithi174 ceph-mon[34488]: pgmap v186: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:12:01.405 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:00 smithi064 ceph-mon[30960]: pgmap v186: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:12:03.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:02 smithi103 ceph-mon[34446]: pgmap v187: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:03.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:02 smithi174 ceph-mon[34488]: pgmap v187: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:03.284 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:02 smithi064 ceph-mon[30960]: pgmap v187: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:05.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:04 smithi103 ceph-mon[34446]: pgmap v188: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:05.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:04 smithi174 ceph-mon[34488]: pgmap v188: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:05.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:04 smithi064 ceph-mon[30960]: pgmap v188: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:07.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:06 smithi103 ceph-mon[34446]: pgmap v189: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:07.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:06 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:12:07.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:06 smithi174 ceph-mon[34488]: pgmap v189: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:07.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:06 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:12:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:06 smithi064 ceph-mon[30960]: pgmap v189: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:06 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:12:08.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:07 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:12:08.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:07 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:12:08.219 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:07 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:12:08.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:07 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:12:08.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:07 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:12:08.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:07 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:12:08.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:07 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:12:08.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:07 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:12:08.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:07 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:12:09.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:09 smithi064 ceph-mon[30960]: pgmap v190: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:09.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:09 smithi103 ceph-mon[34446]: pgmap v190: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:09 smithi174 ceph-mon[34488]: pgmap v190: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:11.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:11 smithi064 ceph-mon[30960]: pgmap v191: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:11.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:11 smithi103 ceph-mon[34446]: pgmap v191: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:11.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:11 smithi174 ceph-mon[34488]: pgmap v191: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:13.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:13 smithi064 ceph-mon[30960]: pgmap v192: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:13 smithi103 ceph-mon[34446]: pgmap v192: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:13.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:13 smithi174 ceph-mon[34488]: pgmap v192: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:15.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:15 smithi064 ceph-mon[30960]: pgmap v193: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:15 smithi103 ceph-mon[34446]: pgmap v193: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:15 smithi174 ceph-mon[34488]: pgmap v193: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-19T05:12:17.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:17 smithi064 ceph-mon[30960]: pgmap v194: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:17 smithi103 ceph-mon[34446]: pgmap v194: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:17.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:17 smithi174 ceph-mon[34488]: pgmap v194: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T05:12:19.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:19 smithi064 ceph-mon[30960]: pgmap v195: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:12:19.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:19 smithi103 ceph-mon[34446]: pgmap v195: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:12:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:19 smithi174 ceph-mon[34488]: pgmap v195: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T05:12:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:21 smithi064 ceph-mon[30960]: pgmap v196: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:12:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:21 smithi103 ceph-mon[34446]: pgmap v196: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:12:21.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:12:21.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:21 smithi174 ceph-mon[34488]: pgmap v196: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:12:21.763 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 -- 172.21.15.64:0/3490396948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 msgr2=0x7ff8e81095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 --2- 172.21.15.64:0/3490396948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e81095d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009900 tx=0x7ff8d802a480 comp rx=0 tx=0).stop 2024-09-19T05:12:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 -- 172.21.15.64:0/3490396948 shutdown_connections 2024-09-19T05:12:22.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 --2- 172.21.15.64:0/3490396948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8e8114ab0 0x7ff8e8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 --2- 172.21.15.64:0/3490396948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8e8109b10 0x7ff8e8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 --2- 172.21.15.64:0/3490396948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e81095d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 -- 172.21.15.64:0/3490396948 >> 172.21.15.64:0/3490396948 conn(0x7ff8e806e140 msgr2=0x7ff8e806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 -- 172.21.15.64:0/3490396948 shutdown_connections 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.308+0000 7ff8ee472640 1 -- 172.21.15.64:0/3490396948 wait complete. 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 Processor -- start 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 -- start start 2024-09-19T05:12:22.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8e8109b10 0x7ff8e810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8e8114ab0 0x7ff8e810ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e811a670 con 0x7ff8e81091f0 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e811a4f0 con 0x7ff8e8114ab0 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8ee472640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e811a7f0 con 0x7ff8e8109b10 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8e7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e810a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39604/0 (socket says 172.21.15.64:39604) 2024-09-19T05:12:22.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8e7fff640 1 -- 172.21.15.64:0/420238869 learned_addr learned my addr 172.21.15.64:0/420238869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8e8109b10 0x7ff8e810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.309+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8e8114ab0 0x7ff8e810ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e7fff640 1 -- 172.21.15.64:0/420238869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8e8109b10 msgr2=0x7ff8e810a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e7fff640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8e8109b10 0x7ff8e810a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e7fff640 1 -- 172.21.15.64:0/420238869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8e8114ab0 msgr2=0x7ff8e810ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e7fff640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8e8114ab0 0x7ff8e810ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e7fff640 1 -- 172.21.15.64:0/420238869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e81af010 con 0x7ff8e81091f0 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e7fff640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e810a3d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005940 tx=0x7ff8d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:22.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e57fa640 1 -- 172.21.15.64:0/420238869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d803f030 con 0x7ff8e81091f0 2024-09-19T05:12:22.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e57fa640 1 -- 172.21.15.64:0/420238869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8d8003aa0 con 0x7ff8e81091f0 2024-09-19T05:12:22.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e81af2a0 con 0x7ff8e81091f0 2024-09-19T05:12:22.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8e57fa640 1 -- 172.21.15.64:0/420238869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8035070 con 0x7ff8e81091f0 2024-09-19T05:12:22.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.310+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e81af7d0 con 0x7ff8e81091f0 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 Processor -- start 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 -- start start 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526dadc0 0x55ba526e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526e5680 0x55ba526e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ba526402e0 con 0x55ba526da4a0 2024-09-19T05:12:22.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ba525e7ce0 con 0x55ba526dadc0 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8f084b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba5263c2a0 con 0x55ba526e5680 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526da880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39610/0 (socket says 172.21.15.64:39610) 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.313+0000 7ff8eca0f640 1 -- 172.21.15.64:0/4080871975 learned_addr learned my addr 172.21.15.64:0/4080871975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8e57fa640 1 -- 172.21.15.64:0/420238869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff8d8042030 con 0x7ff8e81091f0 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526e5680 0x55ba526e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8eca0f640 1 -- 172.21.15.64:0/4080871975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526e5680 msgr2=0x55ba526e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526e5680 0x55ba526e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8eca0f640 1 -- 172.21.15.64:0/4080871975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526dadc0 msgr2=0x55ba526e5140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526dadc0 0x55ba526e5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8eca0f640 1 -- 172.21.15.64:0/4080871975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba525cc440 con 0x55ba526da4a0 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526da880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009f10 tx=0x7ff8e0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df13c03d97227fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:22.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8ce7fc640 1 -- 172.21.15.64:0/4080871975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0016070 con 0x55ba526da4a0 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8ce7fc640 1 -- 172.21.15.64:0/4080871975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e001bb10 con 0x55ba526da4a0 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.314+0000 7ff8ce7fc640 1 -- 172.21.15.64:0/4080871975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0012c30 con 0x55ba526da4a0 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8e57fa640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8d00a3430 0x7ff8d00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8e7fff640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8d00a3430 0x7ff8d00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8e57fa640 1 -- 172.21.15.64:0/420238869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff8d80efe60 con 0x7ff8e81091f0 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 -- 172.21.15.64:0/4080871975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 msgr2=0x55ba526da880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526da880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009f10 tx=0x7ff8e0010740 comp rx=0 tx=0).stop 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8e7fff640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8d00a3430 0x7ff8d00a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55ba526d7890 tx=0x7ff8d8039f00 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 -- 172.21.15.64:0/4080871975 shutdown_connections 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526e5680 0x55ba526e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526dadc0 0x55ba526e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 --2- 172.21.15.64:0/4080871975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526da880 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.315+0000 7ff8f084b740 1 -- 172.21.15.64:0/4080871975 >> 172.21.15.64:0/4080871975 conn(0x55ba526d6a30 msgr2=0x55ba526d6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 -- 172.21.15.64:0/4080871975 shutdown_connections 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 -- 172.21.15.64:0/4080871975 wait complete. 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 -- start start 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 Processor -- start 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ba526e8250 con 0x55ba526e5680 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ba525fb050 con 0x55ba526da4a0 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.316+0000 7ff8f084b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba526e83d0 con 0x55ba526dadc0 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8f084b740 10 client.0 _ll_register_callbacks cb 0x55ba5263b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39622/0 (socket says 172.21.15.64:39622) 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 -- 172.21.15.64:0/943373922 learned_addr learned my addr 172.21.15.64:0/943373922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 -- 172.21.15.64:0/943373922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 msgr2=0x55ba526e2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 -- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 msgr2=0x55ba526e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526e5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 -- 172.21.15.64:0/943373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba523b13c0 con 0x55ba526e5680 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e2910 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0012730 tx=0x7ff8e0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fba4af3e06342b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.317+0000 7ff8c77fe640 1 -- 172.21.15.64:0/943373922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0016070 con 0x55ba526e5680 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8c77fe640 1 -- 172.21.15.64:0/943373922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff8e0003b80 con 0x55ba526e5680 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e001bb10 con 0x55ba526e5680 2024-09-19T05:12:22.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8c77fe640 1 -- 172.21.15.64:0/943373922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8e0004b40 con 0x55ba526e5680 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 10 client.16983 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba526f3ce0 con 0x55ba526e5680 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8c77fe640 1 -- 172.21.15.64:0/943373922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e001bb10 con 0x55ba526e5680 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8c77fe640 1 -- 172.21.15.64:0/943373922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8e0028020 con 0x55ba526e5680 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 10 client.16983 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 20 client.16983 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba526f54c0 0x55ba526f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 4 client.16983 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.318+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba5264d850 con 0x55ba526f54c0 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.319+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba526f54c0 0x55ba526f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.319+0000 7ff8c77fe640 10 client.16983 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.319+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba526f54c0 0x55ba526f78e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff1616820a0bf41 server_cookie=9a1daaf13b0f67f0 in_seq=0 out_seq=0 2024-09-19T05:12:22.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8c77fe640 1 -- 172.21.15.64:0/943373922 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba5264d850 con 0x55ba526f54c0 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8c77fe640 10 client.16983 handle_command_reply: tid=0 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 client.16983 shutdown 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 msgr2=0x55ba526e2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e2910 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0012730 tx=0x7ff8e0009800 comp rx=0 tx=0).stop 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 shutdown_connections 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba526f54c0 0x55ba526f78e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e2910 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 --2- 172.21.15.64:0/943373922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 >> 172.21.15.64:0/943373922 conn(0x55ba526d6a30 msgr2=0x55ba526dc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8c77fe640 0 client.16983 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 shutdown_connections 2024-09-19T05:12:22.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.320+0000 7ff8f084b740 1 -- 172.21.15.64:0/943373922 wait complete. 2024-09-19T05:12:22.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.321+0000 7ff8f084b740 20 client.16983 trim_cache size 0 max 16384 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 Processor -- start 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 -- start start 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526e5680 0x55ba526e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81baab0 con 0x55ba526da4a0 2024-09-19T05:12:22.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81b8380 con 0x55ba526e5680 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8f084b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81b61d0 con 0x55ba526dadc0 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526f14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39638/0 (socket says 172.21.15.64:39638) 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.338+0000 7ff8eca0f640 1 -- 172.21.15.64:0/3280489207 learned_addr learned my addr 172.21.15.64:0/3280489207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526e5680 0x55ba526e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 -- 172.21.15.64:0/3280489207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 msgr2=0x55ba526e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 -- 172.21.15.64:0/3280489207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526e5680 msgr2=0x55ba526e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526e5680 0x55ba526e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 -- 172.21.15.64:0/3280489207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba526dabf0 con 0x55ba526da4a0 2024-09-19T05:12:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526f14e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009770 tx=0x7ff8e0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58f7b356789cbfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8cdffb640 1 -- 172.21.15.64:0/3280489207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0016070 con 0x55ba526da4a0 2024-09-19T05:12:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8cdffb640 1 -- 172.21.15.64:0/3280489207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e0025040 con 0x55ba526da4a0 2024-09-19T05:12:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.339+0000 7ff8cdffb640 1 -- 172.21.15.64:0/3280489207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e001b6c0 con 0x55ba526da4a0 2024-09-19T05:12:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 -- 172.21.15.64:0/3280489207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 msgr2=0x55ba526f14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526f14e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009770 tx=0x7ff8e0010740 comp rx=0 tx=0).stop 2024-09-19T05:12:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 -- 172.21.15.64:0/3280489207 shutdown_connections 2024-09-19T05:12:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526e5680 0x55ba526e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 --2- 172.21.15.64:0/3280489207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526da4a0 0x55ba526f14e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 -- 172.21.15.64:0/3280489207 >> 172.21.15.64:0/3280489207 conn(0x55ba526d6a30 msgr2=0x55ba526e22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 -- 172.21.15.64:0/3280489207 shutdown_connections 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 -- 172.21.15.64:0/3280489207 wait complete. 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.340+0000 7ff8f084b740 1 -- start start 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 Processor -- start 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81b6840 con 0x55ba526e5680 2024-09-19T05:12:22.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81b7cb0 con 0x55ba526da4a0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81b6f10 con 0x55ba526dadc0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8f084b740 10 client.0 _ll_register_callbacks cb 0x7ff8e0009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8eca0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39654/0 (socket says 172.21.15.64:39654) 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8eca0f640 1 -- 172.21.15.64:0/1676370597 learned_addr learned my addr 172.21.15.64:0/1676370597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.341+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8eca0f640 1 -- 172.21.15.64:0/1676370597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 msgr2=0x55ba526e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8eca0f640 1 -- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 msgr2=0x55ba526edc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526edc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8eca0f640 1 -- 172.21.15.64:0/1676370597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba526e91c0 con 0x55ba526e5680 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8eca0f640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e8b50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009880 tx=0x7ff8e0005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfd054884d9fda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:22.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8c77fe640 1 -- 172.21.15.64:0/1676370597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e003e040 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8c77fe640 1 -- 172.21.15.64:0/1676370597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7ff8e001b660 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e0025040 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8c77fe640 1 -- 172.21.15.64:0/1676370597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8e0016040 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8f084b740 10 client.16989 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba5275f030 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.342+0000 7ff8c77fe640 1 -- 172.21.15.64:0/1676370597 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e0025040 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8c77fe640 1 -- 172.21.15.64:0/1676370597 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff8e0044070 con 0x55ba526e5680 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8f084b740 10 client.16989 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8f084b740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8f084b740 20 client.16989 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8f084b740 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba5275f5f0 0x55ba527619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8f084b740 4 client.16989 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba52647d60 con 0x55ba5275f5f0 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba5275f5f0 0x55ba527619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8e77fe640 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba5275f5f0 0x55ba527619e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab93299ee6d92225 server_cookie=9b98ec683b303adb in_seq=0 out_seq=0 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.343+0000 7ff8c77fe640 10 client.16989 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8c77fe640 1 -- 172.21.15.64:0/1676370597 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba52647d60 con 0x55ba5275f5f0 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8c77fe640 10 client.16989 handle_command_reply: tid=0 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 client.16989 shutdown 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 msgr2=0x55ba526e8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e8b50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009880 tx=0x7ff8e0005cf0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 shutdown_connections 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ba5275f5f0 0x55ba527619e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba526e5680 0x55ba526e8b50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba526dadc0 0x55ba526e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8c77fe640 0 client.16989 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 --2- 172.21.15.64:0/1676370597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba526da4a0 0x55ba526edc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 >> 172.21.15.64:0/1676370597 conn(0x55ba526d6a30 msgr2=0x55ba526e0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 shutdown_connections 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 1 -- 172.21.15.64:0/1676370597 wait complete. 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.344+0000 7ff8f084b740 20 client.16989 trim_cache size 0 max 16384 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:12:22.352 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:12:22.353 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:12:22.353 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:12:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8d00a3430 msgr2=0x7ff8d00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8d00a3430 0x7ff8d00a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55ba526d7890 tx=0x7ff8d8039f00 comp rx=0 tx=0).stop 2024-09-19T05:12:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 msgr2=0x7ff8e810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e810a3d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005940 tx=0x7ff8d8004270 comp rx=0 tx=0).stop 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 shutdown_connections 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8d00a3430 0x7ff8d00a58f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8e8114ab0 0x7ff8e810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8e8109b10 0x7ff8e810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 --2- 172.21.15.64:0/420238869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8e81091f0 0x7ff8e810a3d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 >> 172.21.15.64:0/420238869 conn(0x7ff8e806e140 msgr2=0x7ff8e8070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 shutdown_connections 2024-09-19T05:12:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:22.345+0000 7ff8ee472640 1 -- 172.21.15.64:0/420238869 wait complete. 2024-09-19T05:12:23.200 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:12:23.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:12:23.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:23 smithi064 ceph-mon[30960]: pgmap v197: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:23.462 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:23.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:23 smithi103 ceph-mon[34446]: pgmap v197: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:23.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:23 smithi174 ceph-mon[34488]: pgmap v197: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 -- 172.21.15.64:0/3360960104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102ef0 msgr2=0x7fef2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 --2- 172.21.15.64:0/3360960104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102ef0 0x7fef2010da40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fef10009ca0 tx=0x7fef1002a950 comp rx=0 tx=0).stop 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 -- 172.21.15.64:0/3360960104 shutdown_connections 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 --2- 172.21.15.64:0/3360960104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102ef0 0x7fef2010da40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 --2- 172.21.15.64:0/3360960104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20102570 0x7fef201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 --2- 172.21.15.64:0/3360960104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20100e20 0x7fef20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 -- 172.21.15.64:0/3360960104 >> 172.21.15.64:0/3360960104 conn(0x7fef200fc7d0 msgr2=0x7fef200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 -- 172.21.15.64:0/3360960104 shutdown_connections 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.964+0000 7fef274fe640 1 -- 172.21.15.64:0/3360960104 wait complete. 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 Processor -- start 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 -- start start 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 0x7fef200fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 0x7fef200fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20102ef0 0x7fef200fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fef20110e30 con 0x7fef20102570 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef20110cb0 con 0x7fef20100e20 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef274fe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef20110fb0 con 0x7fef20102ef0 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 0x7fef200fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 0x7fef200fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39680/0 (socket says 172.21.15.64:39680) 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 -- 172.21.15.64:0/4152143650 learned_addr learned my addr 172.21.15.64:0/4152143650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef25a9b640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20102ef0 0x7fef200fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef2529a640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 0x7fef200fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 -- 172.21.15.64:0/4152143650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20102ef0 msgr2=0x7fef200fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20102ef0 0x7fef200fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 -- 172.21.15.64:0/4152143650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 msgr2=0x7fef200fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 0x7fef200fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.965+0000 7fef24a99640 1 -- 172.21.15.64:0/4152143650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef201a6840 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.966+0000 7fef24a99640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 0x7fef200fd1c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fef1400d950 tx=0x7fef1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.966+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14017070 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.966+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef14009c30 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.966+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1400f620 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.966+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef201a6b30 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.966+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef201a7020 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.967+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef20101200 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.969+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fef140109a0 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.969+0000 7fef1e7fc640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fef080a33b0 0x7fef080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.969+0000 7fef2529a640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fef080a33b0 0x7fef080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.969+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fef140cf960 con 0x7fef20102570 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.970+0000 7fef2529a640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fef080a33b0 0x7fef080a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef18005e10 tx=0x7fef18005da0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:23.973+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef14009de0 con 0x7fef20102570 2024-09-19T05:12:24.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.189+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef2010a0a0 con 0x7fef20102570 2024-09-19T05:12:24.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.189+0000 7fef1e7fc640 1 -- 172.21.15.64:0/4152143650 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fef1409c340 con 0x7fef20102570 2024-09-19T05:12:24.192 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:12:24.192 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:12:24.193 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:12:24.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fef080a33b0 msgr2=0x7fef080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:24.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fef080a33b0 0x7fef080a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef18005e10 tx=0x7fef18005da0 comp rx=0 tx=0).stop 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 msgr2=0x7fef200fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 0x7fef200fd1c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fef1400d950 tx=0x7fef1400de10 comp rx=0 tx=0).stop 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 shutdown_connections 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fef080a33b0 0x7fef080a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef20102ef0 0x7fef200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef20102570 0x7fef200fd1c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 --2- 172.21.15.64:0/4152143650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef20100e20 0x7fef200fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 >> 172.21.15.64:0/4152143650 conn(0x7fef200fc7d0 msgr2=0x7fef2006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 shutdown_connections 2024-09-19T05:12:24.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:24.192+0000 7fef274fe640 1 -- 172.21.15.64:0/4152143650 wait complete. 2024-09-19T05:12:24.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:12:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:25.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:25 smithi064 ceph-mon[30960]: pgmap v198: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:25.366 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:25 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4152143650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:25.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:25 smithi103 ceph-mon[34446]: pgmap v198: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:25 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4152143650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:25.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:25 smithi174 ceph-mon[34488]: pgmap v198: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:25.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:25 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4152143650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.738+0000 7f62bdcd3640 1 -- 172.21.15.64:0/766454477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b8103700 msgr2=0x7f62b810bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.738+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/766454477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b8103700 0x7f62b810bbf0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009900 tx=0x7f62ac02a480 comp rx=0 tx=0).stop 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 -- 172.21.15.64:0/766454477 shutdown_connections 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/766454477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b810c180 0x7f62b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/766454477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b8103700 0x7f62b810bbf0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/766454477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b8102de0 0x7f62b81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 -- 172.21.15.64:0/766454477 >> 172.21.15.64:0/766454477 conn(0x7f62b80fc7a0 msgr2=0x7f62b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 -- 172.21.15.64:0/766454477 shutdown_connections 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 -- 172.21.15.64:0/766454477 wait complete. 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 Processor -- start 2024-09-19T05:12:25.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.739+0000 7f62bdcd3640 1 -- start start 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62bdcd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b8102de0 0x7f62b8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62bdcd3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b8103700 0x7f62b81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62bdcd3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 0x7f62b819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62bdcd3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8111e20 con 0x7f62b810c180 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62bdcd3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8111ca0 con 0x7f62b8103700 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62bdcd3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8111fa0 con 0x7f62b8102de0 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 0x7f62b819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 0x7f62b819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40814/0 (socket says 172.21.15.64:40814) 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 -- 172.21.15.64:0/2664947292 learned_addr learned my addr 172.21.15.64:0/2664947292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b77fe640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b8102de0 0x7f62b8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b8103700 0x7f62b81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 -- 172.21.15.64:0/2664947292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b8102de0 msgr2=0x7f62b8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b8102de0 0x7f62b8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 -- 172.21.15.64:0/2664947292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b8103700 msgr2=0x7f62b81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b8103700 0x7f62b81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 -- 172.21.15.64:0/2664947292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b819d310 con 0x7f62b810c180 2024-09-19T05:12:25.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b7fff640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 0x7f62b819cc30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f62b000eaa0 tx=0x7f62b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.740+0000 7f62b4ff9640 1 -- 172.21.15.64:0/2664947292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000cdb0 con 0x7f62b810c180 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.741+0000 7f62b4ff9640 1 -- 172.21.15.64:0/2664947292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b0005020 con 0x7f62b810c180 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.741+0000 7f62b4ff9640 1 -- 172.21.15.64:0/2664947292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00116d0 con 0x7f62b810c180 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.741+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b819d600 con 0x7f62b810c180 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.741+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b81a4ee0 con 0x7f62b810c180 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.743+0000 7f62b4ff9640 1 -- 172.21.15.64:0/2664947292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f62b00051d0 con 0x7f62b810c180 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.743+0000 7f62c00ac740 1 Processor -- start 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.743+0000 7f62c00ac740 1 -- start start 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712aa7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712ab4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62c00ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556712a0d2f0 con 0x556712ab2370 2024-09-19T05:12:25.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62c00ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567129b4cf0 con 0x556712aa8b40 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62c00ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556712a092b0 con 0x556712aa75d0 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b4ff9640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62a00a33b0 0x7f62a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712ab4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712ab4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40816/0 (socket says 172.21.15.64:40816) 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 -- 172.21.15.64:0/1444979664 learned_addr learned my addr 172.21.15.64:0/1444979664 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b77fe640 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712aa7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b7fff640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62a00a33b0 0x7f62a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b4ff9640 1 -- 172.21.15.64:0/2664947292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f62b00cfeb0 con 0x7f62b810c180 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 -- 172.21.15.64:0/1444979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 msgr2=0x556712aa7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712aa7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 -- 172.21.15.64:0/1444979664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 msgr2=0x556712aa7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aa7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.744+0000 7f62b6ffd640 1 -- 172.21.15.64:0/1444979664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567129993a0 con 0x556712ab2370 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.745+0000 7f62b7fff640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62a00a33b0 0x7f62a00a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62b009c120 tx=0x7f62b006efc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.745+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712ab4760 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0096c0 tx=0x7f62ac005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38bdfa239e235f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.745+0000 7f629dffb640 1 -- 172.21.15.64:0/1444979664 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac035040 con 0x556712ab2370 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.745+0000 7f629dffb640 1 -- 172.21.15.64:0/1444979664 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62ac0318e0 con 0x556712ab2370 2024-09-19T05:12:25.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.745+0000 7f629dffb640 1 -- 172.21.15.64:0/1444979664 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac003a10 con 0x556712ab2370 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 -- 172.21.15.64:0/1444979664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 msgr2=0x556712ab4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712ab4760 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0096c0 tx=0x7f62ac005b50 comp rx=0 tx=0).stop 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 -- 172.21.15.64:0/1444979664 shutdown_connections 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712ab4760 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712aa7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 --2- 172.21.15.64:0/1444979664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 -- 172.21.15.64:0/1444979664 >> 172.21.15.64:0/1444979664 conn(0x556712a70310 msgr2=0x556712aa3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 -- 172.21.15.64:0/1444979664 shutdown_connections 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 -- 172.21.15.64:0/1444979664 wait complete. 2024-09-19T05:12:25.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 -- start start 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.746+0000 7f62c00ac740 1 Processor -- start 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712ab1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aaf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712aaf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556712ab5010 con 0x556712ab2370 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567129c8060 con 0x556712aa8b40 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556712ab5190 con 0x556712aa75d0 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 10 client.0 _ll_register_callbacks cb 0x55671271b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712aaf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712aaf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40828/0 (socket says 172.21.15.64:40828) 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 -- 172.21.15.64:0/154602691 learned_addr learned my addr 172.21.15.64:0/154602691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712ab1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b77fe640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aaf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 -- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 msgr2=0x556712ab1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712ab1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 -- 172.21.15.64:0/154602691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 msgr2=0x556712aaf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aaf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 -- 172.21.15.64:0/154602691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671277e3c0 con 0x556712ab2370 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712aaf7f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009330 tx=0x7f62ac0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffa4ae552b08d13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f6296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f6296ffd640 1 -- 172.21.15.64:0/154602691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac035070 con 0x556712ab2370 2024-09-19T05:12:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f6296ffd640 1 -- 172.21.15.64:0/154602691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f62ac0316e0 con 0x556712ab2370 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.747+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556712a1a860 con 0x556712ab2370 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f6296ffd640 1 -- 172.21.15.64:0/154602691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62ac04a420 con 0x556712ab2370 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 10 client.17007 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556712ac0b00 con 0x556712ab2370 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f6296ffd640 1 -- 172.21.15.64:0/154602691 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556712a1a860 con 0x556712ab2370 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f6296ffd640 1 -- 172.21.15.64:0/154602691 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62ac03a040 con 0x556712ab2370 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 10 client.17007 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 10 client.17007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 20 client.17007 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712ac4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 4 client.17007 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.748+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556712a14d70 con 0x556712ac1e50 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.749+0000 7f62b77fe640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712ac4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.749+0000 7f6296ffd640 10 client.17007 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.749+0000 7f62b77fe640 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712ac4240 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea4588f5cd32f9e7 server_cookie=dcc5e3cb14c86e46 in_seq=0 out_seq=0 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f6296ffd640 1 -- 172.21.15.64:0/154602691 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556712a14d70 con 0x556712ac1e50 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f6296ffd640 10 client.17007 handle_command_reply: tid=0 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 client.17007 shutdown 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 msgr2=0x556712aaf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712aaf7f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009330 tx=0x7f62ac0035f0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 shutdown_connections 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712ac4240 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712ab2370 0x556712aaf7f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa8b40 0x556712aaf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f6296ffd640 0 client.17007 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:12:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 --2- 172.21.15.64:0/154602691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa75d0 0x556712ab1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 >> 172.21.15.64:0/154602691 conn(0x556712a70310 msgr2=0x556712aa9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 shutdown_connections 2024-09-19T05:12:25.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 1 -- 172.21.15.64:0/154602691 wait complete. 2024-09-19T05:12:25.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.750+0000 7f62c00ac740 20 client.17007 trim_cache size 0 max 16384 2024-09-19T05:12:25.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 Processor -- start 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 -- start start 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 0x556712aae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa8b40 0x556712aaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712ab2370 0x556712aaa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81b0170 con 0x556712aa75d0 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81ada40 con 0x556712ab2370 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62c00ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81ab890 con 0x556712aa8b40 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 0x556712aae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 0x556712aae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40838/0 (socket says 172.21.15.64:40838) 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.763+0000 7f62b6ffd640 1 -- 172.21.15.64:0/3509124700 learned_addr learned my addr 172.21.15.64:0/3509124700 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712ab2370 0x556712aaa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 -- 172.21.15.64:0/3509124700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa8b40 msgr2=0x556712aaa120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa8b40 0x556712aaa120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 -- 172.21.15.64:0/3509124700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712ab2370 msgr2=0x556712aaa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712ab2370 0x556712aaa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 -- 172.21.15.64:0/3509124700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556712aad5b0 con 0x556712aa75d0 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 0x556712aae920 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f62ac033000 tx=0x7f62ac005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a964887568f0d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f629d7fa640 1 -- 172.21.15.64:0/3509124700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac035070 con 0x556712aa75d0 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f629d7fa640 1 -- 172.21.15.64:0/3509124700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62ac0315d0 con 0x556712aa75d0 2024-09-19T05:12:25.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.764+0000 7f629d7fa640 1 -- 172.21.15.64:0/3509124700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac039990 con 0x556712aa75d0 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 -- 172.21.15.64:0/3509124700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 msgr2=0x556712aae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 0x556712aae920 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f62ac033000 tx=0x7f62ac005b50 comp rx=0 tx=0).stop 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 -- 172.21.15.64:0/3509124700 shutdown_connections 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712ab2370 0x556712aaa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712aa8b40 0x556712aaa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 --2- 172.21.15.64:0/3509124700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa75d0 0x556712aae920 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 -- 172.21.15.64:0/3509124700 >> 172.21.15.64:0/3509124700 conn(0x556712a70310 msgr2=0x556712aa2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 -- 172.21.15.64:0/3509124700 shutdown_connections 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 -- 172.21.15.64:0/3509124700 wait complete. 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 -- start start 2024-09-19T05:12:25.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.765+0000 7f62c00ac740 1 Processor -- start 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa75d0 0x556712ac4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa8b40 0x556712ac4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712ab2370 0x556712ac7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81abf00 con 0x556712aa8b40 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81ad370 con 0x556712aa75d0 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b81ac5d0 con 0x556712ab2370 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62c00ac740 10 client.0 _ll_register_callbacks cb 0x7f62ac032cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa8b40 0x556712ac4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa75d0 0x556712ac4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa75d0 0x556712ac4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:55252/0 (socket says 172.21.15.64:55252) 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b6ffd640 1 -- 172.21.15.64:0/4268284405 learned_addr learned my addr 172.21.15.64:0/4268284405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712ab2370 0x556712ac7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 -- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712ab2370 msgr2=0x556712ac7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712ab2370 0x556712ac7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 -- 172.21.15.64:0/4268284405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa75d0 msgr2=0x556712ac4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa75d0 0x556712ac4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 -- 172.21.15.64:0/4268284405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556712ac76b0 con 0x556712aa8b40 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f62b77fe640 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa8b40 0x556712ac4fc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x556712ab5180 tx=0x7f62a800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2683bb448c69c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.766+0000 7f6296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f6296ffd640 1 -- 172.21.15.64:0/4268284405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8003f30 con 0x556712aa8b40 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f6296ffd640 1 -- 172.21.15.64:0/4268284405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f62a8005020 con 0x556712aa8b40 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ac0315d0 con 0x556712aa8b40 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f6296ffd640 1 -- 172.21.15.64:0/4268284405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62a8005220 con 0x556712aa8b40 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 10 client.17013 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556712b30780 con 0x556712aa8b40 2024-09-19T05:12:25.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f6296ffd640 1 -- 172.21.15.64:0/4268284405 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62ac0315d0 con 0x556712aa8b40 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f6296ffd640 1 -- 172.21.15.64:0/4268284405 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62a8011440 con 0x556712aa8b40 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 10 client.17013 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 10 client.17013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 20 client.17013 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712b32e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 4 client.17013 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.767+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556712991170 con 0x556712ac1e50 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.768+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712b32e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.768+0000 7f62b6ffd640 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712b32e90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4903d2e290062e77 server_cookie=9aae9a5c0f785e8 in_seq=0 out_seq=0 2024-09-19T05:12:25.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.768+0000 7f6296ffd640 10 client.17013 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:12:25.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.768+0000 7f6296ffd640 1 -- 172.21.15.64:0/4268284405 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556712991170 con 0x556712ac1e50 2024-09-19T05:12:25.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.768+0000 7f6296ffd640 10 client.17013 handle_command_reply: tid=0 2024-09-19T05:12:25.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.768+0000 7f62c00ac740 1 client.17013 shutdown 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa8b40 msgr2=0x556712ac4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa8b40 0x556712ac4fc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x556712ab5180 tx=0x7f62a800bcd0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 shutdown_connections 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x556712ac1e50 0x556712b32e90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556712ab2370 0x556712ac7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556712aa8b40 0x556712ac4fc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f6296ffd640 0 client.17013 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 --2- 172.21.15.64:0/4268284405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556712aa75d0 0x556712ac4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 >> 172.21.15.64:0/4268284405 conn(0x556712a70310 msgr2=0x556712aad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 shutdown_connections 2024-09-19T05:12:25.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 1 -- 172.21.15.64:0/4268284405 wait complete. 2024-09-19T05:12:25.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62c00ac740 20 client.17013 trim_cache size 0 max 16384 2024-09-19T05:12:25.777 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:12:25.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.769+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62a00a33b0 msgr2=0x7f62a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62a00a33b0 0x7f62a00a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62b009c120 tx=0x7f62b006efc0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 msgr2=0x7f62b819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 0x7f62b819cc30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f62b000eaa0 tx=0x7f62b000ef60 comp rx=0 tx=0).stop 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 shutdown_connections 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62a00a33b0 0x7f62a00a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b810c180 0x7f62b819cc30 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b8103700 0x7f62b81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 --2- 172.21.15.64:0/2664947292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b8102de0 0x7f62b8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 >> 172.21.15.64:0/2664947292 conn(0x7f62b80fc7a0 msgr2=0x7f62b80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 shutdown_connections 2024-09-19T05:12:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:25.770+0000 7f62bdcd3640 1 -- 172.21.15.64:0/2664947292 wait complete. 2024-09-19T05:12:26.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:12:26.892 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:27.269 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:27 smithi064 ceph-mon[30960]: pgmap v199: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:27.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:27 smithi103 ceph-mon[34446]: pgmap v199: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:27.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 -- 172.21.15.64:0/1552311121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb810c170 msgr2=0x7f9fb810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 --2- 172.21.15.64:0/1552311121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb810c170 0x7f9fb810e610 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009ca0 tx=0x7f9fb002a950 comp rx=0 tx=0).stop 2024-09-19T05:12:27.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 -- 172.21.15.64:0/1552311121 shutdown_connections 2024-09-19T05:12:27.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 --2- 172.21.15.64:0/1552311121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb810c170 0x7f9fb810e610 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 --2- 172.21.15.64:0/1552311121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8103660 0x7f9fb810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 --2- 172.21.15.64:0/1552311121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb8102d40 0x7f9fb8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 -- 172.21.15.64:0/1552311121 >> 172.21.15.64:0/1552311121 conn(0x7f9fb80fc7a0 msgr2=0x7f9fb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 -- 172.21.15.64:0/1552311121 shutdown_connections 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.490+0000 7f9fbded4640 1 -- 172.21.15.64:0/1552311121 wait complete. 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 Processor -- start 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 -- start start 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8102d40 0x7f9fb8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 0x7f9fb8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb810c170 0x7f9fb819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111e20 con 0x7f9fb8103660 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111ca0 con 0x7f9fb8102d40 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.491+0000 7f9fbded4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111fa0 con 0x7f9fb810c170 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 0x7f9fb8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 0x7f9fb8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40864/0 (socket says 172.21.15.64:40864) 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3046161887 learned_addr learned my addr 172.21.15.64:0/3046161887 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb810c170 0x7f9fb819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8102d40 0x7f9fb8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3046161887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb810c170 msgr2=0x7f9fb819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb810c170 0x7f9fb819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3046161887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8102d40 msgr2=0x7f9fb8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8102d40 0x7f9fb8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3046161887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb819d2f0 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8102d40 0x7f9fb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb810c170 0x7f9fb819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 0x7f9fb8198880 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d950 tx=0x7f9fac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb4ff9640 1 -- 172.21.15.64:0/3046161887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac017030 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb4ff9640 1 -- 172.21.15.64:0/3046161887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac004020 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fb4ff9640 1 -- 172.21.15.64:0/3046161887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0055e0 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb819d5e0 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.492+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81a4ec0 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.495+0000 7f9fb4ff9640 1 -- 172.21.15.64:0/3046161887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9fac0041d0 con 0x7f9fb8103660 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.495+0000 7f9fc02ad740 1 Processor -- start 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.495+0000 7f9fc02ad740 1 -- start start 2024-09-19T05:12:27.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559488b6d2f0 con 0x559488c12370 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559488b14cf0 con 0x559488c075d0 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559488b692b0 con 0x559488c08b40 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c14760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40876/0 (socket says 172.21.15.64:40876) 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3055374964 learned_addr learned my addr 172.21.15.64:0/3055374964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb4ff9640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9fa00a33b0 0x7f9fa00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9fa00a33b0 0x7f9fa00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb4ff9640 1 -- 172.21.15.64:0/3046161887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f9fac0cf680 con 0x7f9fb8103660 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3055374964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 msgr2=0x559488c07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c07090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3055374964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 msgr2=0x559488c07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/3055374964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559488af93a0 con 0x559488c12370 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.496+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.497+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c14760 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9fac09b890 tx=0x7f9fac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71032cc64ad6410c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:27.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.497+0000 7f9f9dffb640 1 -- 172.21.15.64:0/3055374964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac09b440 con 0x559488c12370 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.497+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9fa00a33b0 0x7f9fa00a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005fc0 tx=0x7f9fb00095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.497+0000 7f9f9dffb640 1 -- 172.21.15.64:0/3055374964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac0980f0 con 0x559488c12370 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.497+0000 7f9f9dffb640 1 -- 172.21.15.64:0/3055374964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac09b260 con 0x559488c12370 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.497+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3055374964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 msgr2=0x559488c14760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c14760 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9fac09b890 tx=0x7f9fac000bc0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3055374964 shutdown_connections 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c14760 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3055374964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3055374964 >> 172.21.15.64:0/3055374964 conn(0x559488bd0310 msgr2=0x559488c03c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3055374964 shutdown_connections 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.498+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3055374964 wait complete. 2024-09-19T05:12:27.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 -- start start 2024-09-19T05:12:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 Processor -- start 2024-09-19T05:12:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c11d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c0f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c0f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559488c14fb0 con 0x559488c12370 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559488b28060 con 0x559488c075d0 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559488c15130 con 0x559488c08b40 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fc02ad740 10 client.0 _ll_register_callbacks cb 0x55948887b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c0f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c0f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40892/0 (socket says 172.21.15.64:40892) 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fb7fff640 1 -- 172.21.15.64:0/1167811980 learned_addr learned my addr 172.21.15.64:0/1167811980 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c11d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.499+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c0f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb7fff640 1 -- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 msgr2=0x559488c0f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c0f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb7fff640 1 -- 172.21.15.64:0/1167811980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 msgr2=0x559488c11d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c11d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb7fff640 1 -- 172.21.15.64:0/1167811980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594888de3c0 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c0f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c0f650 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0035000 tx=0x7f9fb002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c3c28e1f268b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9f96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9f96ffd640 1 -- 172.21.15.64:0/1167811980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb004c030 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9f96ffd640 1 -- 172.21.15.64:0/1167811980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f9fb0004250 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559488b7a860 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9f96ffd640 1 -- 172.21.15.64:0/1167811980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fb0034040 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fc02ad740 10 client.17025 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.500+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559488c20a80 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9f96ffd640 1 -- 172.21.15.64:0/1167811980 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559488b7a860 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9f96ffd640 1 -- 172.21.15.64:0/1167811980 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9fb0047040 con 0x559488c12370 2024-09-19T05:12:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fc02ad740 10 client.17025 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fc02ad740 10 client.17025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fc02ad740 20 client.17025 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c21e90 0x559488c24310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fc02ad740 4 client.17025 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559488b74d70 con 0x559488c21e90 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c21e90 0x559488c24310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.502+0000 7f9f96ffd640 10 client.17025 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.501+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c21e90 0x559488c24310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f54717c1c7b8f1cf server_cookie=f8b182c9b940fda0 in_seq=0 out_seq=0 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.502+0000 7f9f96ffd640 1 -- 172.21.15.64:0/1167811980 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559488b74d70 con 0x559488c21e90 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.502+0000 7f9f96ffd640 10 client.17025 handle_command_reply: tid=0 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.502+0000 7f9fc02ad740 1 client.17025 shutdown 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 msgr2=0x559488c0f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c0f650 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0035000 tx=0x7f9fb002a950 comp rx=0 tx=0).stop 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 shutdown_connections 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c21e90 0x559488c24310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c12370 0x559488c0f650 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c08b40 0x559488c0f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/1167811980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c11d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 >> 172.21.15.64:0/1167811980 conn(0x559488bd0310 msgr2=0x559488c09960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9f96ffd640 0 client.17025 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 shutdown_connections 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 1 -- 172.21.15.64:0/1167811980 wait complete. 2024-09-19T05:12:27.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.503+0000 7f9fc02ad740 20 client.17025 trim_cache size 0 max 16384 2024-09-19T05:12:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 Processor -- start 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 -- start start 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 0x559488c1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c08b40 0x559488c03700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c03c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81b0160 con 0x559488c075d0 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ada30 con 0x559488c08b40 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ab880 con 0x559488c12370 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 0x559488c1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 0x559488c1e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40900/0 (socket says 172.21.15.64:40900) 2024-09-19T05:12:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fb77fe640 1 -- 172.21.15.64:0/3983724927 learned_addr learned my addr 172.21.15.64:0/3983724927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.516+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c08b40 0x559488c03700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c03c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb77fe640 1 -- 172.21.15.64:0/3983724927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 msgr2=0x559488c03c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c03c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb77fe640 1 -- 172.21.15.64:0/3983724927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c08b40 msgr2=0x559488c03700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c08b40 0x559488c03700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb77fe640 1 -- 172.21.15.64:0/3983724927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559488c04180 con 0x559488c075d0 2024-09-19T05:12:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c08b40 0x559488c03700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c03c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 0x559488c1e250 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8004700 tx=0x7f9fa800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d57c3c0448b2b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9f9d7fa640 1 -- 172.21.15.64:0/3983724927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa801d670 con 0x559488c075d0 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9f9d7fa640 1 -- 172.21.15.64:0/3983724927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa801dcc0 con 0x559488c075d0 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.517+0000 7f9f9d7fa640 1 -- 172.21.15.64:0/3983724927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80103d0 con 0x559488c075d0 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3983724927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 msgr2=0x559488c1e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 0x559488c1e250 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8004700 tx=0x7f9fa800bd90 comp rx=0 tx=0).stop 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3983724927 shutdown_connections 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c03c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c08b40 0x559488c03700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/3983724927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c075d0 0x559488c1e250 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3983724927 >> 172.21.15.64:0/3983724927 conn(0x559488bd0310 msgr2=0x559488c0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:27.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3983724927 shutdown_connections 2024-09-19T05:12:27.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 -- 172.21.15.64:0/3983724927 wait complete. 2024-09-19T05:12:27.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 -- start start 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.518+0000 7f9fc02ad740 1 Processor -- start 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c1a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 0x559488c0bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c0bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81abef0 con 0x559488c08b40 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ad360 con 0x559488c075d0 2024-09-19T05:12:27.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81ac5c0 con 0x559488c12370 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fc02ad740 10 client.0 _ll_register_callbacks cb 0x7f9fb0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 0x559488c0bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 0x559488c0bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40904/0 (socket says 172.21.15.64:40904) 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/793460395 learned_addr learned my addr 172.21.15.64:0/793460395 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb7fff640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c0bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c1a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 msgr2=0x559488c0bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c0bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/793460395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 msgr2=0x559488c1a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c1a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.519+0000 7f9fb6ffd640 1 -- 172.21.15.64:0/793460395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559488c0c530 con 0x559488c08b40 2024-09-19T05:12:27.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9fb6ffd640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 0x559488c0bab0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9fac09b890 tx=0x7f9fac0a4640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a012e61f713b1358 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9f96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9f96ffd640 1 -- 172.21.15.64:0/793460395 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0b4060 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9f96ffd640 1 -- 172.21.15.64:0/793460395 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f9fac09af50 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559488af1170 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9f96ffd640 1 -- 172.21.15.64:0/793460395 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fac09b260 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9fc02ad740 10 client.17031 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559488c8c800 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9f96ffd640 1 -- 172.21.15.64:0/793460395 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559488af1170 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.520+0000 7f9f96ffd640 1 -- 172.21.15.64:0/793460395 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9fac060bc0 con 0x559488c08b40 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fc02ad740 10 client.17031 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fc02ad740 10 client.17031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fc02ad740 20 client.17031 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:27.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c8cae0 0x559488c8eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fc02ad740 4 client.17031 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559488b3e9b0 con 0x559488c8cae0 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c8cae0 0x559488c8eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9f96ffd640 10 client.17031 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.521+0000 7f9fb77fe640 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c8cae0 0x559488c8eed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8bf7aab8322967b server_cookie=2099cb9c9fa6957a in_seq=0 out_seq=0 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9f96ffd640 1 -- 172.21.15.64:0/793460395 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559488b3e9b0 con 0x559488c8cae0 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9f96ffd640 10 client.17031 handle_command_reply: tid=0 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 client.17031 shutdown 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 msgr2=0x559488c0bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 0x559488c0bab0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9fac09b890 tx=0x7f9fac0a4640 comp rx=0 tx=0).stop 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 shutdown_connections 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559488c12370 0x559488c0bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x559488c8cae0 0x559488c8eed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559488c08b40 0x559488c0bab0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 --2- 172.21.15.64:0/793460395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559488c075d0 0x559488c1a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9f96ffd640 0 client.17031 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 >> 172.21.15.64:0/793460395 conn(0x559488bd0310 msgr2=0x559488c0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 shutdown_connections 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 1 -- 172.21.15.64:0/793460395 wait complete. 2024-09-19T05:12:27.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.522+0000 7f9fc02ad740 20 client.17031 trim_cache size 0 max 16384 2024-09-19T05:12:27.531 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:12:27.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9fa00a33b0 msgr2=0x7f9fa00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9fa00a33b0 0x7f9fa00a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005fc0 tx=0x7f9fb00095c0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 msgr2=0x7f9fb8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:27.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 0x7f9fb8198880 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d950 tx=0x7f9fac00de10 comp rx=0 tx=0).stop 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 shutdown_connections 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9fa00a33b0 0x7f9fa00a5870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb810c170 0x7f9fb819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fb8103660 0x7f9fb8198880 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 --2- 172.21.15.64:0/3046161887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9fb8102d40 0x7f9fb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 >> 172.21.15.64:0/3046161887 conn(0x7f9fb80fc7a0 msgr2=0x7f9fb80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 shutdown_connections 2024-09-19T05:12:27.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:27.523+0000 7f9fbded4640 1 -- 172.21.15.64:0/3046161887 wait complete. 2024-09-19T05:12:27.532 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:27 smithi174 ceph-mon[34488]: pgmap v199: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:28.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:12:28.451 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 -- 172.21.15.64:0/2367153205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 msgr2=0x7fb63010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 --2- 172.21.15.64:0/2367153205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63010ea90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb628009900 tx=0x7fb62802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 -- 172.21.15.64:0/2367153205 shutdown_connections 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 --2- 172.21.15.64:0/2367153205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63010ea90 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 --2- 172.21.15.64:0/2367153205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb630102790 0x7fb6301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 --2- 172.21.15.64:0/2367153205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb630101de0 0x7fb6301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 -- 172.21.15.64:0/2367153205 >> 172.21.15.64:0/2367153205 conn(0x7fb6300fd8c0 msgr2=0x7fb6300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 -- 172.21.15.64:0/2367153205 shutdown_connections 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.923+0000 7fb634902640 1 -- 172.21.15.64:0/2367153205 wait complete. 2024-09-19T05:12:28.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 Processor -- start 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 -- start start 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb630101de0 0x7fb630198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb630102790 0x7fb6301988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb630111e60 con 0x7fb630106f10 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb630111ce0 con 0x7fb630102790 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb634902640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb630111fe0 con 0x7fb630101de0 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.924+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40934/0 (socket says 172.21.15.64:40934) 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 -- 172.21.15.64:0/1321558456 learned_addr learned my addr 172.21.15.64:0/1321558456 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb630102790 0x7fb6301988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 -- 172.21.15.64:0/1321558456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb630101de0 msgr2=0x7fb630198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb630101de0 0x7fb630198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 -- 172.21.15.64:0/1321558456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb630102790 msgr2=0x7fb6301988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb630102790 0x7fb6301988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 -- 172.21.15.64:0/1321558456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63019d330 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63019cc50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb628009230 tx=0x7fb628004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb6277fe640 1 -- 172.21.15.64:0/1321558456 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62803f030 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb6277fe640 1 -- 172.21.15.64:0/1321558456 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb628003aa0 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63019d5c0 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb6277fe640 1 -- 172.21.15.64:0/1321558456 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628035070 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.925+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301a4f00 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.928+0000 7fb636cdb740 1 Processor -- start 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.928+0000 7fb6277fe640 1 -- 172.21.15.64:0/1321558456 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fb628042030 con 0x7fb630106f10 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 -- start start 2024-09-19T05:12:28.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 0x560b421aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421b9780 0x560b421bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b421102f0 con 0x560b421aa5d0 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b420b7cf0 con 0x560b421abb40 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb636cdb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b4210c2b0 con 0x560b421b9780 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 0x560b421aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 0x560b421aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40950/0 (socket says 172.21.15.64:40950) 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 -- 172.21.15.64:0/152621366 learned_addr learned my addr 172.21.15.64:0/152621366 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421b9780 0x560b421bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 -- 172.21.15.64:0/152621366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421b9780 msgr2=0x560b421bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421b9780 0x560b421bbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 -- 172.21.15.64:0/152621366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 msgr2=0x560b421aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62e59c640 1 -- 172.21.15.64:0/152621366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4209c3a0 con 0x560b421aa5d0 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421b9780 0x560b421bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.929+0000 7fb6277fe640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb61c0a33b0 0x7fb61c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.930+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb61c0a33b0 0x7fb61c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.930+0000 7fb62e59c640 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 0x560b421aaa10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb620009da0 tx=0x7fb62000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e709adac3caef6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.930+0000 7fb6277fe640 1 -- 172.21.15.64:0/1321558456 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb6280f0470 con 0x7fb630106f10 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.930+0000 7fb624ff9640 1 -- 172.21.15.64:0/152621366 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000bd40 con 0x560b421aa5d0 2024-09-19T05:12:28.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.930+0000 7fb624ff9640 1 -- 172.21.15.64:0/152621366 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb620017070 con 0x560b421aa5d0 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.930+0000 7fb624ff9640 1 -- 172.21.15.64:0/152621366 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620011630 con 0x560b421aa5d0 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 -- 172.21.15.64:0/152621366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 msgr2=0x560b421aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 0x560b421aaa10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb620009da0 tx=0x7fb62000b420 comp rx=0 tx=0).stop 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb61c0a33b0 0x7fb61c0a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x560b421ab070 tx=0x7fb618014040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 -- 172.21.15.64:0/152621366 shutdown_connections 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421b9780 0x560b421bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421aa5d0 0x560b421aaa10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 --2- 172.21.15.64:0/152621366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 -- 172.21.15.64:0/152621366 >> 172.21.15.64:0/152621366 conn(0x560b42173310 msgr2=0x560b421a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 -- 172.21.15.64:0/152621366 shutdown_connections 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 -- 172.21.15.64:0/152621366 wait complete. 2024-09-19T05:12:28.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 -- start start 2024-09-19T05:12:28.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.931+0000 7fb636cdb740 1 Processor -- start 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421aa5d0 0x560b421b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b421bc480 con 0x560b421b9780 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b420cb060 con 0x560b421abb40 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b421bc600 con 0x560b421aa5d0 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 10 client.0 _ll_register_callbacks cb 0x560b41e1e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40952/0 (socket says 172.21.15.64:40952) 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 -- 172.21.15.64:0/4121161668 learned_addr learned my addr 172.21.15.64:0/4121161668 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:28.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 -- 172.21.15.64:0/4121161668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421aa5d0 msgr2=0x560b421b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421aa5d0 0x560b421b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 -- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 msgr2=0x560b421b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 -- 172.21.15.64:0/4121161668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b41e813c0 con 0x560b421b9780 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b6900 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb6280c4e90 tx=0x7fb6280bc0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff19b0b3e2dd07d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb615ffb640 1 -- 172.21.15.64:0/4121161668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628049070 con 0x560b421b9780 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb615ffb640 1 -- 172.21.15.64:0/4121161668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb62803bcd0 con 0x560b421b9780 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.932+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb620017070 con 0x560b421b9780 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb615ffb640 1 -- 172.21.15.64:0/4121161668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6280c0b00 con 0x560b421b9780 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 10 client.17043 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:28.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b421c7e00 con 0x560b421b9780 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb615ffb640 1 -- 172.21.15.64:0/4121161668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb620017070 con 0x560b421b9780 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb615ffb640 1 -- 172.21.15.64:0/4121161668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6280c4f30 con 0x560b421b9780 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 10 client.17043 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 20 client.17043 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b421cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 4 client.17043 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b4211d860 con 0x560b421c9240 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.933+0000 7fb62e59c640 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b421cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.934+0000 7fb62e59c640 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b421cb700 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805cc78a94efdcbf server_cookie=5a3eef9a4aceb554 in_seq=0 out_seq=0 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.934+0000 7fb615ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:28.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.934+0000 7fb615ffb640 1 -- 172.21.15.64:0/4121161668 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b4211d860 con 0x560b421c9240 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.934+0000 7fb615ffb640 10 client.17043 handle_command_reply: tid=0 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 client.17043 shutdown 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 msgr2=0x560b421b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b6900 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb6280c4e90 tx=0x7fb6280bc0b0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 shutdown_connections 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b421cb700 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b6900 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421abb40 0x560b421b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb615ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 --2- 172.21.15.64:0/4121161668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421aa5d0 0x560b421b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 >> 172.21.15.64:0/4121161668 conn(0x560b42173310 msgr2=0x560b421ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 shutdown_connections 2024-09-19T05:12:28.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 1 -- 172.21.15.64:0/4121161668 wait complete. 2024-09-19T05:12:28.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.935+0000 7fb636cdb740 20 client.17043 trim_cache size 0 max 16384 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 Processor -- start 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 -- start start 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301adde0 con 0x560b421b9780 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ab6b0 con 0x560b421aa5d0 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb636cdb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301a9500 con 0x560b421abb40 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.948+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40966/0 (socket says 172.21.15.64:40966) 2024-09-19T05:12:28.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2276153676 learned_addr learned my addr 172.21.15.64:0/2276153676 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62e59c640 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2276153676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 msgr2=0x560b421a6700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421a6700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2276153676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 msgr2=0x560b421c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2276153676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b421a7180 con 0x560b421b9780 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62e59c640 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421a6c40 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb6280c0cc0 tx=0x7fb6280bc8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c44fcee19653bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb617fff640 1 -- 172.21.15.64:0/2276153676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628049070 con 0x560b421b9780 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb617fff640 1 -- 172.21.15.64:0/2276153676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6280c4f30 con 0x560b421b9780 2024-09-19T05:12:28.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.949+0000 7fb617fff640 1 -- 172.21.15.64:0/2276153676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6280b8dc0 con 0x560b421b9780 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 -- 172.21.15.64:0/2276153676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 msgr2=0x560b421a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421a6c40 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb6280c0cc0 tx=0x7fb6280bc8d0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 -- 172.21.15.64:0/2276153676 shutdown_connections 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421a6c40 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2276153676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 -- 172.21.15.64:0/2276153676 >> 172.21.15.64:0/2276153676 conn(0x560b42173310 msgr2=0x560b421b6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 -- 172.21.15.64:0/2276153676 shutdown_connections 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 -- 172.21.15.64:0/2276153676 wait complete. 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 -- start start 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.950+0000 7fb636cdb740 1 Processor -- start 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301a9b70 con 0x560b421b9780 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301aafe0 con 0x560b421aa5d0 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301aa240 con 0x560b421abb40 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 10 client.0 _ll_register_callbacks cb 0x7fb628009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:28.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40978/0 (socket says 172.21.15.64:40978) 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2498245151 learned_addr learned my addr 172.21.15.64:0/2498245151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62e59c640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2498245151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 msgr2=0x560b421b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 msgr2=0x560b421c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 -- 172.21.15.64:0/2498245151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b421b3e40 con 0x560b421b9780 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62dd9b640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb62ed9d640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b3610 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb62803d040 tx=0x7fb628003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4dffd7a477e97c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:28.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb615ffb640 1 -- 172.21.15.64:0/2498245151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628049070 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb615ffb640 1 -- 172.21.15.64:0/2498245151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fb6280bbcf0 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.951+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6280c4f30 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb615ffb640 1 -- 172.21.15.64:0/2498245151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb62803be50 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 10 client.17049 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b421b3e40 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb615ffb640 1 -- 172.21.15.64:0/2498245151 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6280c4f30 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb615ffb640 1 -- 172.21.15.64:0/2498245151 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6280b85a0 con 0x560b421b9780 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 10 client.17049 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 20 client.17049 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b422362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 4 client.17049 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:28.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b42117d70 con 0x560b421c9240 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.952+0000 7fb62e59c640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b422362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb62e59c640 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b422362c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e781bf6053edcf server_cookie=45667533fd98cad1 in_seq=0 out_seq=0 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb615ffb640 10 client.17049 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb615ffb640 1 -- 172.21.15.64:0/2498245151 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b42117d70 con 0x560b421c9240 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb615ffb640 10 client.17049 handle_command_reply: tid=0 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 client.17049 shutdown 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 msgr2=0x560b421b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b3610 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb62803d040 tx=0x7fb628003f50 comp rx=0 tx=0).stop 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 shutdown_connections 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560b421c9240 0x560b422362c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b421b9780 0x560b421b3610 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b421abb40 0x560b421b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 --2- 172.21.15.64:0/2498245151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b421aa5d0 0x560b421c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 >> 172.21.15.64:0/2498245151 conn(0x560b42173310 msgr2=0x560b421b6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb615ffb640 0 client.17049 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 shutdown_connections 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 1 -- 172.21.15.64:0/2498245151 wait complete. 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.953+0000 7fb636cdb740 20 client.17049 trim_cache size 0 max 16384 2024-09-19T05:12:28.961 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb61c0a33b0 msgr2=0x7fb61c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb61c0a33b0 0x7fb61c0a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x560b421ab070 tx=0x7fb618014040 comp rx=0 tx=0).stop 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 msgr2=0x7fb63019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63019cc50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb628009230 tx=0x7fb628004270 comp rx=0 tx=0).stop 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 shutdown_connections 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fb61c0a33b0 0x7fb61c0a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb630106f10 0x7fb63019cc50 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb630102790 0x7fb6301988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 --2- 172.21.15.64:0/1321558456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb630101de0 0x7fb630198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 >> 172.21.15.64:0/1321558456 conn(0x7fb6300fd8c0 msgr2=0x7fb6300fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 shutdown_connections 2024-09-19T05:12:28.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:28.954+0000 7fb634902640 1 -- 172.21.15.64:0/1321558456 wait complete. 2024-09-19T05:12:29.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:29 smithi064 ceph-mon[30960]: pgmap v200: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:29.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:29 smithi103 ceph-mon[34446]: pgmap v200: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:29 smithi174 ceph-mon[34488]: pgmap v200: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:30.667 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:12:30.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:12:30.943 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:31.330 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:31 smithi174 ceph-mon[34488]: pgmap v201: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:31.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:31 smithi064 ceph-mon[30960]: pgmap v201: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 -- 172.21.15.64:0/312790212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8103660 msgr2=0x7fadc810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 --2- 172.21.15.64:0/312790212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8103660 0x7fadc810bb50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009900 tx=0x7fadbc02a480 comp rx=0 tx=0).stop 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 -- 172.21.15.64:0/312790212 shutdown_connections 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 --2- 172.21.15.64:0/312790212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 0x7fadc810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 --2- 172.21.15.64:0/312790212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8103660 0x7fadc810bb50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 --2- 172.21.15.64:0/312790212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8102d40 0x7fadc8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 -- 172.21.15.64:0/312790212 >> 172.21.15.64:0/312790212 conn(0x7fadc80fc7a0 msgr2=0x7fadc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.382+0000 7fadcc933640 1 -- 172.21.15.64:0/312790212 shutdown_connections 2024-09-19T05:12:31.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.383+0000 7fadcc933640 1 -- 172.21.15.64:0/312790212 wait complete. 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.383+0000 7fadcc933640 1 Processor -- start 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.383+0000 7fadcc933640 1 -- start start 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 0x7fadc81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8103660 0x7fadc81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 0x7fadc819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8111e20 con 0x7fadc8102d40 2024-09-19T05:12:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8111ca0 con 0x7fadc8103660 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8111fa0 con 0x7fadc810c170 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 0x7fadc81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 0x7fadc81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41004/0 (socket says 172.21.15.64:41004) 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 -- 172.21.15.64:0/1301491495 learned_addr learned my addr 172.21.15.64:0/1301491495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8103660 0x7fadc81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 0x7fadc819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 -- 172.21.15.64:0/1301491495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 msgr2=0x7fadc819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 0x7fadc819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 -- 172.21.15.64:0/1301491495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8103660 msgr2=0x7fadc81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8103660 0x7fadc81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 -- 172.21.15.64:0/1301491495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc819d350 con 0x7fadc8102d40 2024-09-19T05:12:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 0x7fadc819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8103660 0x7fadc81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadc659c640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 0x7fadc81983a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d950 tx=0x7fadb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadb77fe640 1 -- 172.21.15.64:0/1301491495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8017070 con 0x7fadc8102d40 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc819d640 con 0x7fadc8102d40 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.384+0000 7fadb77fe640 1 -- 172.21.15.64:0/1301491495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadb8009c30 con 0x7fadc8102d40 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.385+0000 7fadb77fe640 1 -- 172.21.15.64:0/1301491495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800f620 con 0x7fadc8102d40 2024-09-19T05:12:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.385+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81a4f20 con 0x7fadc8102d40 2024-09-19T05:12:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadb77fe640 1 -- 172.21.15.64:0/1301491495 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fadb80109a0 con 0x7fadc8102d40 2024-09-19T05:12:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 Processor -- start 2024-09-19T05:12:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 -- start start 2024-09-19T05:12:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e066d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e0662a280 con 0x555e066c5380 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555e065d1b10 con 0x555e066d4150 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadced0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e06626240 con 0x555e066c4a60 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.388+0000 7fadc659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41008/0 (socket says 172.21.15.64:41008) 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 -- 172.21.15.64:0/779733749 learned_addr learned my addr 172.21.15.64:0/779733749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e066d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadb77fe640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fadb00a33b0 0x7fadb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 -- 172.21.15.64:0/779733749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 msgr2=0x555e066c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 -- 172.21.15.64:0/779733749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 msgr2=0x555e066d6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e066d6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 -- 172.21.15.64:0/779733749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e065de1f0 con 0x555e066c5380 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fadb00a33b0 0x7fadb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadb77fe640 1 -- 172.21.15.64:0/1301491495 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fadb80cfa20 con 0x7fadc8102d40 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc659c640 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d3c10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fadb80cd0e0 tx=0x7fadb800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d0c3f7c4a3f66c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadb4ff9640 1 -- 172.21.15.64:0/779733749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020030 con 0x555e066c5380 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadb4ff9640 1 -- 172.21.15.64:0/779733749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadb80d8070 con 0x555e066c5380 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadb4ff9640 1 -- 172.21.15.64:0/779733749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb809c400 con 0x555e066c5380 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.389+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fadb00a33b0 0x7fadb00a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005fd0 tx=0x7fadc0004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 -- 172.21.15.64:0/779733749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 msgr2=0x555e066d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d3c10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fadb80cd0e0 tx=0x7fadb800bf60 comp rx=0 tx=0).stop 2024-09-19T05:12:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 -- 172.21.15.64:0/779733749 shutdown_connections 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e066d6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d3c10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 --2- 172.21.15.64:0/779733749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 -- 172.21.15.64:0/779733749 >> 172.21.15.64:0/779733749 conn(0x555e066c10f0 msgr2=0x555e066c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 -- 172.21.15.64:0/779733749 shutdown_connections 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.390+0000 7fadced0c740 1 -- 172.21.15.64:0/779733749 wait complete. 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 -- start start 2024-09-19T05:12:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 Processor -- start 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c4a60 0x555e066d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066d4150 0x555e066d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e066d6d20 con 0x555e066c5380 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555e065e4f10 con 0x555e066c4a60 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e066d6ea0 con 0x555e066d4150 2024-09-19T05:12:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadced0c740 10 client.0 _ll_register_callbacks cb 0x555e065ada40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadc5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadc5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41012/0 (socket says 172.21.15.64:41012) 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadc5d9b640 1 -- 172.21.15.64:0/227320238 learned_addr learned my addr 172.21.15.64:0/227320238 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.391+0000 7fadc659c640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c4a60 0x555e066d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066d4150 0x555e066d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc5d9b640 1 -- 172.21.15.64:0/227320238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066d4150 msgr2=0x555e066d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066d4150 0x555e066d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc5d9b640 1 -- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c4a60 msgr2=0x555e066d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c4a60 0x555e066d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc5d9b640 1 -- 172.21.15.64:0/227320238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0639b6a0 con 0x555e066c5380 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066d4150 0x555e066d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d1590 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fadbc033000 tx=0x7fadbc0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c8ab44e3a68f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadadffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadadffb640 1 -- 172.21.15.64:0/227320238 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc035040 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadadffb640 1 -- 172.21.15.64:0/227320238 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fadbc004250 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb80d8070 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadadffb640 1 -- 172.21.15.64:0/227320238 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadbc004450 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadced0c740 10 client.17061 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e066e2990 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.392+0000 7fadadffb640 1 -- 172.21.15.64:0/227320238 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadb80d8070 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadadffb640 1 -- 172.21.15.64:0/227320238 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fadbc004a60 con 0x555e066c5380 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadced0c740 10 client.17061 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadced0c740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadced0c740 20 client.17061 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadced0c740 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e066e3e10 0x555e066e6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadced0c740 4 client.17061 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e066377f0 con 0x555e066e3e10 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadc659c640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e066e3e10 0x555e066e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadc659c640 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e066e3e10 0x555e066e6250 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca339988dac8b8 server_cookie=e8a410dab4106f6d in_seq=0 out_seq=0 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.393+0000 7fadadffb640 10 client.17061 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadadffb640 1 -- 172.21.15.64:0/227320238 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e066377f0 con 0x555e066e3e10 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadadffb640 10 client.17061 handle_command_reply: tid=0 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 client.17061 shutdown 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 msgr2=0x555e066d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d1590 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fadbc033000 tx=0x7fadbc0314a0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 shutdown_connections 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066d4150 0x555e066d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e066e3e10 0x555e066e6250 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadadffb640 0 client.17061 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d1590 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 --2- 172.21.15.64:0/227320238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c4a60 0x555e066d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.394+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 >> 172.21.15.64:0/227320238 conn(0x555e066c10f0 msgr2=0x555e066c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.395+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 shutdown_connections 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.395+0000 7fadced0c740 1 -- 172.21.15.64:0/227320238 wait complete. 2024-09-19T05:12:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.395+0000 7fadced0c740 20 client.17061 trim_cache size 0 max 16384 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 Processor -- start 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 -- start start 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c5380 0x555e066ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 0x555e066ec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81adfb0 con 0x555e066d4150 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81ab880 con 0x555e066c5380 2024-09-19T05:12:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadced0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81a96d0 con 0x555e066c4a60 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 0x555e066ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadc6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 0x555e066ec060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41022/0 (socket says 172.21.15.64:41022) 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.412+0000 7fadc6d9d640 1 -- 172.21.15.64:0/3595681283 learned_addr learned my addr 172.21.15.64:0/3595681283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c5380 0x555e066ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc659c640 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc6d9d640 1 -- 172.21.15.64:0/3595681283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 msgr2=0x555e066d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc6d9d640 1 -- 172.21.15.64:0/3595681283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c5380 msgr2=0x555e066ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c5380 0x555e066ebb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc6d9d640 1 -- 172.21.15.64:0/3595681283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e066c0f20 con 0x555e066d4150 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc659c640 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 0x555e066ec060 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fadc001d620 tx=0x7fadc0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733f2902223f9f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadaffff640 1 -- 172.21.15.64:0/3595681283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011a50 con 0x555e066d4150 2024-09-19T05:12:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadaffff640 1 -- 172.21.15.64:0/3595681283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadc0011bf0 con 0x555e066d4150 2024-09-19T05:12:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.413+0000 7fadaffff640 1 -- 172.21.15.64:0/3595681283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc001a640 con 0x555e066d4150 2024-09-19T05:12:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 -- 172.21.15.64:0/3595681283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 msgr2=0x555e066ec060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 0x555e066ec060 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fadc001d620 tx=0x7fadc0012000 comp rx=0 tx=0).stop 2024-09-19T05:12:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 -- 172.21.15.64:0/3595681283 shutdown_connections 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066d4150 0x555e066ec060 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066c5380 0x555e066ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 --2- 172.21.15.64:0/3595681283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 -- 172.21.15.64:0/3595681283 >> 172.21.15.64:0/3595681283 conn(0x555e066c10f0 msgr2=0x555e066d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 -- 172.21.15.64:0/3595681283 shutdown_connections 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 -- 172.21.15.64:0/3595681283 wait complete. 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 -- start start 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.414+0000 7fadced0c740 1 Processor -- start 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e06747ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81a9d40 con 0x555e066c5380 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81ab1b0 con 0x555e066d4150 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81aa410 con 0x555e066c4a60 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 10 client.0 _ll_register_callbacks cb 0x7fadbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41026/0 (socket says 172.21.15.64:41026) 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 -- 172.21.15.64:0/707475957 learned_addr learned my addr 172.21.15.64:0/707475957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 -- 172.21.15.64:0/707475957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 msgr2=0x555e066d86d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc6d9d640 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e06747ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d86d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 -- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 msgr2=0x555e06747ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e06747ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 -- 172.21.15.64:0/707475957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e067480b0 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadc5d9b640 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d8c10 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fadbc004f00 tx=0x7fadbc031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eea6851a324e18a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadadffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadadffb640 1 -- 172.21.15.64:0/707475957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc035040 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadadffb640 1 -- 172.21.15.64:0/707475957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7fadbc0048c0 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.415+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc0011bf0 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadadffb640 1 -- 172.21.15.64:0/707475957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadbc05e050 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 10 client.17067 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e0674f3e0 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadadffb640 1 -- 172.21.15.64:0/707475957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadc0011bf0 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadadffb640 1 -- 172.21.15.64:0/707475957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fadbc056020 con 0x555e066c5380 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 10 client.17067 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 20 client.17067 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e0674fb80 0x555e06751fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 4 client.17067 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555e06631d00 con 0x555e0674fb80 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.416+0000 7fadc659c640 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e0674fb80 0x555e06751fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.417+0000 7fadadffb640 10 client.17067 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.417+0000 7fadc659c640 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e0674fb80 0x555e06751fa0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec15849ec119359d server_cookie=ee51af39304a9811 in_seq=0 out_seq=0 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadadffb640 1 -- 172.21.15.64:0/707475957 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555e06631d00 con 0x555e0674fb80 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadadffb640 10 client.17067 handle_command_reply: tid=0 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 client.17067 shutdown 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 msgr2=0x555e066d8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d8c10 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fadbc004f00 tx=0x7fadbc031ba0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 shutdown_connections 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e066d4150 0x555e06747ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555e0674fb80 0x555e06751fa0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e066c5380 0x555e066d8c10 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 --2- 172.21.15.64:0/707475957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e066c4a60 0x555e066d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadadffb640 0 client.17067 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 >> 172.21.15.64:0/707475957 conn(0x555e066c10f0 msgr2=0x555e066d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 shutdown_connections 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 1 -- 172.21.15.64:0/707475957 wait complete. 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.418+0000 7fadced0c740 20 client.17067 trim_cache size 0 max 16384 2024-09-19T05:12:31.425 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:12:31.426 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:12:31.426 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "9ccc1e20-ce0e-45d4-81b0-d12a748bc317", 2024-09-19T05:12:31.426 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:12:31.426 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:12:31.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fadb00a33b0 msgr2=0x7fadb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fadb00a33b0 0x7fadb00a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005fd0 tx=0x7fadc0004a40 comp rx=0 tx=0).stop 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 msgr2=0x7fadc81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 0x7fadc81983a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d950 tx=0x7fadb800de10 comp rx=0 tx=0).stop 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 shutdown_connections 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fadb00a33b0 0x7fadb00a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc810c170 0x7fadc819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadc8103660 0x7fadc81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 --2- 172.21.15.64:0/1301491495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadc8102d40 0x7fadc81983a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 >> 172.21.15.64:0/1301491495 conn(0x7fadc80fc7a0 msgr2=0x7fadc80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 shutdown_connections 2024-09-19T05:12:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:31.419+0000 7fadcc933640 1 -- 172.21.15.64:0/1301491495 wait complete. 2024-09-19T05:12:31.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:31 smithi103 ceph-mon[34446]: pgmap v201: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:32.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:12:32.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:32 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:12:32.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:32 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:32.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:32 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:12:32.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:32 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:32.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:32 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:12:32.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:32 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:32.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:32 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:12:32.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:32 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:32.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:32 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:12:32.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:32 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:32.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:32 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:12:32.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:32 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:33.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 -- 172.21.15.64:0/3958918366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01019e0 msgr2=0x7febd0101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 --2- 172.21.15.64:0/3958918366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01019e0 0x7febd0101e40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7febc4009930 tx=0x7febc402a420 comp rx=0 tx=0).stop 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 -- 172.21.15.64:0/3958918366 shutdown_connections 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 --2- 172.21.15.64:0/3958918366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd0102380 0x7febd010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 --2- 172.21.15.64:0/3958918366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01019e0 0x7febd0101e40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 --2- 172.21.15.64:0/3958918366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0107af0 0x7febd0107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 -- 172.21.15.64:0/3958918366 >> 172.21.15.64:0/3958918366 conn(0x7febd00fd700 msgr2=0x7febd00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 -- 172.21.15.64:0/3958918366 shutdown_connections 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.002+0000 7febcffff640 1 -- 172.21.15.64:0/3958918366 wait complete. 2024-09-19T05:12:33.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.003+0000 7febcffff640 1 Processor -- start 2024-09-19T05:12:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.003+0000 7febcffff640 1 -- start start 2024-09-19T05:12:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.003+0000 7febcffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01019e0 0x7febd0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.003+0000 7febcffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0102380 0x7febd01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 0x7febd019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febd0111dc0 con 0x7febd0107af0 2024-09-19T05:12:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcffff640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febd0111c40 con 0x7febd01019e0 2024-09-19T05:12:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcffff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd0111f40 con 0x7febd0102380 2024-09-19T05:12:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 0x7febd019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 0x7febd019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41032/0 (socket says 172.21.15.64:41032) 2024-09-19T05:12:33.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 -- 172.21.15.64:0/372815903 learned_addr learned my addr 172.21.15.64:0/372815903 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:33.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febceffd640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01019e0 0x7febd0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febce7fc640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0102380 0x7febd01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 -- 172.21.15.64:0/372815903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0102380 msgr2=0x7febd01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0102380 0x7febd01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 -- 172.21.15.64:0/372815903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01019e0 msgr2=0x7febd0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01019e0 0x7febd0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 -- 172.21.15.64:0/372815903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd019d310 con 0x7febd0107af0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febce7fc640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0102380 0x7febd01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febcf7fe640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 0x7febd019cc30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7febc800eaa0 tx=0x7febc800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.004+0000 7febd4802640 1 -- 172.21.15.64:0/372815903 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8004d20 con 0x7febd0107af0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.005+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd019d600 con 0x7febd0107af0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.005+0000 7febd4802640 1 -- 172.21.15.64:0/372815903 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc8005020 con 0x7febd0107af0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.005+0000 7febd4802640 1 -- 172.21.15.64:0/372815903 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80116b0 con 0x7febd0107af0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.005+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a4ee0 con 0x7febd0107af0 2024-09-19T05:12:33.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.007+0000 7febd4802640 1 -- 172.21.15.64:0/372815903 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7febc80051d0 con 0x7febd0107af0 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.007+0000 7febd7bdd740 1 Processor -- start 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.007+0000 7febd7bdd740 1 -- start start 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152c670 0x56061152ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x560611537310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560611537850 0x560611539c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd7bdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5606114922e0 con 0x56061152cf90 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd7bdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560611439ce0 con 0x56061152c670 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd7bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56061148e2a0 con 0x560611537850 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x560611537310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x560611537310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41046/0 (socket says 172.21.15.64:41046) 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 -- 172.21.15.64:0/3445331139 learned_addr learned my addr 172.21.15.64:0/3445331139 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd4802640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febce7fc640 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152c670 0x56061152ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febce7fc640 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560611537850 0x560611539c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 -- 172.21.15.64:0/3445331139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560611537850 msgr2=0x560611539c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560611537850 0x560611539c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 -- 172.21.15.64:0/3445331139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152c670 msgr2=0x56061152ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152c670 0x56061152ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 -- 172.21.15.64:0/3445331139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061141e440 con 0x56061152cf90 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febd4802640 1 -- 172.21.15.64:0/372815903 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7febc80cff90 con 0x7febd0107af0 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febce7fc640 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152c670 0x56061152ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febcf7fe640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.008+0000 7febceffd640 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x560611537310 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7febc0009a70 tx=0x7febc0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86410dc498dbdd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febb5ffb640 1 -- 172.21.15.64:0/3445331139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0017070 con 0x56061152cf90 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febb5ffb640 1 -- 172.21.15.64:0/3445331139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc0013470 con 0x56061152cf90 2024-09-19T05:12:33.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febb5ffb640 1 -- 172.21.15.64:0/3445331139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00125f0 con 0x56061152cf90 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 -- 172.21.15.64:0/3445331139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 msgr2=0x560611537310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febcf7fe640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febc809bf70 tx=0x7febc8005750 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x560611537310 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7febc0009a70 tx=0x7febc0009f30 comp rx=0 tx=0).stop 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 -- 172.21.15.64:0/3445331139 shutdown_connections 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560611537850 0x560611539c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x560611537310 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3445331139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152c670 0x56061152ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.009+0000 7febd7bdd740 1 -- 172.21.15.64:0/3445331139 >> 172.21.15.64:0/3445331139 conn(0x560611528c00 msgr2=0x560611528fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 -- 172.21.15.64:0/3445331139 shutdown_connections 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 -- 172.21.15.64:0/3445331139 wait complete. 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 -- start start 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 Processor -- start 2024-09-19T05:12:33.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 0x5606115370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152cf90 0x560611534420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x560611534960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56061153a420 con 0x56061152c670 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56061144d050 con 0x560611537850 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56061153a5a0 con 0x56061152cf90 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febd7bdd740 10 client.0 _ll_register_callbacks cb 0x56061148d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febce7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 0x5606115370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febce7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 0x5606115370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41062/0 (socket says 172.21.15.64:41062) 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.010+0000 7febce7fc640 1 -- 172.21.15.64:0/184382621 learned_addr learned my addr 172.21.15.64:0/184382621 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febceffd640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152cf90 0x560611534420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x560611534960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 -- 172.21.15.64:0/184382621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152cf90 msgr2=0x560611534420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152cf90 0x560611534420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 -- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 msgr2=0x560611534960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x560611534960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 -- 172.21.15.64:0/184382621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606112033c0 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febce7fc640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 0x5606115370d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7febc4009930 tx=0x7febc402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaa0933a1dfc741 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febaeffd640 1 -- 172.21.15.64:0/184382621 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4035040 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febaeffd640 1 -- 172.21.15.64:0/184382621 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7febc4004450 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc0013470 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febaeffd640 1 -- 172.21.15.64:0/184382621 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febc40035f0 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febd7bdd740 10 client.17079 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560611545d30 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.011+0000 7febaeffd640 1 -- 172.21.15.64:0/184382621 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc0013470 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febaeffd640 1 -- 172.21.15.64:0/184382621 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febc403d580 con 0x56061152c670 2024-09-19T05:12:33.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febd7bdd740 10 client.17079 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febd7bdd740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febd7bdd740 20 client.17079 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febd7bdd740 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560611547140 0x560611549570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febd7bdd740 4 client.17079 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56061149f850 con 0x560611547140 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febceffd640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560611547140 0x560611549570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febceffd640 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560611547140 0x560611549570 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d13a48df4e3a11e server_cookie=8ebc48009b476f99 in_seq=0 out_seq=0 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.012+0000 7febaeffd640 10 client.17079 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febaeffd640 1 -- 172.21.15.64:0/184382621 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56061149f850 con 0x560611547140 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febaeffd640 10 client.17079 handle_command_reply: tid=0 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 client.17079 shutdown 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 msgr2=0x5606115370d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 0x5606115370d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7febc4009930 tx=0x7febc402a420 comp rx=0 tx=0).stop 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 shutdown_connections 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x560611534960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152cf90 0x560611534420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560611547140 0x560611549570 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 --2- 172.21.15.64:0/184382621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152c670 0x5606115370d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febaeffd640 0 client.17079 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.013+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 >> 172.21.15.64:0/184382621 conn(0x560611528c00 msgr2=0x56061152e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.014+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 shutdown_connections 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.014+0000 7febd7bdd740 1 -- 172.21.15.64:0/184382621 wait complete. 2024-09-19T05:12:33.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.014+0000 7febd7bdd740 20 client.17079 trim_cache size 0 max 16384 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 Processor -- start 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 -- start start 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x560611533360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152cf90 0x560611534840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 0x56061154ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febd01affd0 con 0x560611537850 2024-09-19T05:12:33.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ad8a0 con 0x56061152cf90 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.033+0000 7febd7bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ab6f0 con 0x56061152c670 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 0x56061154ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 0x56061154ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41072/0 (socket says 172.21.15.64:41072) 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 -- 172.21.15.64:0/2511257051 learned_addr learned my addr 172.21.15.64:0/2511257051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febceffd640 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152cf90 0x560611534840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 -- 172.21.15.64:0/2511257051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 msgr2=0x560611533360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x560611533360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 -- 172.21.15.64:0/2511257051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152cf90 msgr2=0x560611534840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152cf90 0x560611534840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 -- 172.21.15.64:0/2511257051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560611549bf0 con 0x560611537850 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febceffd640 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152cf90 0x560611534840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febce7fc640 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 0x56061154ee40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7febc4031440 tx=0x7febc4009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa18f4eb10af710 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:33.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febb57fa640 1 -- 172.21.15.64:0/2511257051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4035040 con 0x560611537850 2024-09-19T05:12:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febb57fa640 1 -- 172.21.15.64:0/2511257051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc4009520 con 0x560611537850 2024-09-19T05:12:33.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.034+0000 7febb57fa640 1 -- 172.21.15.64:0/2511257051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4000e60 con 0x560611537850 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 -- 172.21.15.64:0/2511257051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 msgr2=0x56061154ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 0x56061154ee40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7febc4031440 tx=0x7febc4009270 comp rx=0 tx=0).stop 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 -- 172.21.15.64:0/2511257051 shutdown_connections 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560611537850 0x56061154ee40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061152cf90 0x560611534840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 --2- 172.21.15.64:0/2511257051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x560611533360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 -- 172.21.15.64:0/2511257051 >> 172.21.15.64:0/2511257051 conn(0x560611528c00 msgr2=0x560611527d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:33.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 -- 172.21.15.64:0/2511257051 shutdown_connections 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 -- 172.21.15.64:0/2511257051 wait complete. 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.035+0000 7febd7bdd740 1 -- start start 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 Processor -- start 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x56061153d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x56061153daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x56061153e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febd01abd60 con 0x56061152cf90 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ad1d0 con 0x560611537850 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd01ac430 con 0x56061152c670 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febd7bdd740 10 client.0 _ll_register_callbacks cb 0x7febc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x56061153daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x56061153daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41084/0 (socket says 172.21.15.64:41084) 2024-09-19T05:12:33.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 -- 172.21.15.64:0/3439779304 learned_addr learned my addr 172.21.15.64:0/3439779304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febce7fc640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x56061153e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febce7fc640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x56061153d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 -- 172.21.15.64:0/3439779304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 msgr2=0x56061153d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x56061153d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 -- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 msgr2=0x56061153e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x56061153e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 -- 172.21.15.64:0/3439779304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606115b0e70 con 0x56061152cf90 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febce7fc640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x56061153d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:33.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febceffd640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x56061153daf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x560611529a60 tx=0x7febc0013e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4615d89d70bc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.036+0000 7febaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febaeffd640 1 -- 172.21.15.64:0/3439779304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0017040 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febaeffd640 1 -- 172.21.15.64:0/3439779304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7febc0005bc0 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc4009520 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febaeffd640 1 -- 172.21.15.64:0/3439779304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febc0013ca0 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 10 client.17085 fetch_fsmap learned FSMap version 25 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606115b2160 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febaeffd640 1 -- 172.21.15.64:0/3439779304 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc4009520 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febaeffd640 1 -- 172.21.15.64:0/3439779304 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febc0006170 con 0x56061152cf90 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 10 client.17085 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 20 client.17085 populate_metadata read hostname 'smithi064' 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5606115b2690 0x5606115b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 4 client.17085 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.037+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560611499d60 con 0x5606115b2690 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febce7fc640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5606115b2690 0x5606115b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febaeffd640 10 client.17085 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febce7fc640 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5606115b2690 0x5606115b4ab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c62daec6a9c0549 server_cookie=64b1c35292575ce1 in_seq=0 out_seq=0 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febaeffd640 1 -- 172.21.15.64:0/3439779304 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560611499d60 con 0x5606115b2690 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febaeffd640 10 client.17085 handle_command_reply: tid=0 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 client.17085 shutdown 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 msgr2=0x56061153daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x56061153daf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x560611529a60 tx=0x7febc0013e70 comp rx=0 tx=0).stop 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 shutdown_connections 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560611537850 0x56061153e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5606115b2690 0x5606115b4ab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febaeffd640 0 client.17085 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56061152cf90 0x56061153daf0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 --2- 172.21.15.64:0/3439779304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56061152c670 0x56061153d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.038+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 >> 172.21.15.64:0/3439779304 conn(0x560611528c00 msgr2=0x5606115359b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 shutdown_connections 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febd7bdd740 1 -- 172.21.15.64:0/3439779304 wait complete. 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febd7bdd740 20 client.17085 trim_cache size 0 max 16384 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:12:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "9ccc1e20-ce0e-45d4-81b0-d12a748bc317": { 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "9ccc1e20-ce0e-45d4-81b0-d12a748bc317", 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:12:33.048 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:12:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 msgr2=0x7febb80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febc809bf70 tx=0x7febc8005750 comp rx=0 tx=0).stop 2024-09-19T05:12:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 msgr2=0x7febd019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 0x7febd019cc30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7febc800eaa0 tx=0x7febc800eea0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 shutdown_connections 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd0107af0 0x7febd019cc30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0102380 0x7febd01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 --2- 172.21.15.64:0/372815903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febd01019e0 0x7febd0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 >> 172.21.15.64:0/372815903 conn(0x7febd00fd700 msgr2=0x7febd00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 shutdown_connections 2024-09-19T05:12:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:12:33.039+0000 7febcffff640 1 -- 172.21.15.64:0/372815903 wait complete. 2024-09-19T05:12:33.352 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:33 smithi064 ceph-mon[30960]: pgmap v202: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:33.370 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:33 smithi103 ceph-mon[34446]: pgmap v202: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:33 smithi174 ceph-mon[34488]: pgmap v202: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:33.764 INFO:tasks.cephfs.filesystem:scrub status for tag:9ccc1e20-ce0e-45d4-81b0-d12a748bc317 - {'path': '/', 'tag': '9ccc1e20-ce0e-45d4-81b0-d12a748bc317', 'options': 'recursive,force'} 2024-09-19T05:12:34.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:34 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:12:34.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:34 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:12:34.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:34 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:12:35.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:35 smithi064 ceph-mon[30960]: pgmap v203: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:35.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:35 smithi103 ceph-mon[34446]: pgmap v203: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:35.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:35 smithi174 ceph-mon[34488]: pgmap v203: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:37.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:37 smithi064 ceph-mon[30960]: pgmap v204: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-09-19T05:12:37.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:37 smithi103 ceph-mon[34446]: pgmap v204: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-09-19T05:12:37.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:37 smithi174 ceph-mon[34488]: pgmap v204: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-09-19T05:12:39.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:39 smithi064 ceph-mon[30960]: pgmap v205: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:39 smithi103 ceph-mon[34446]: pgmap v205: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:39.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:39 smithi174 ceph-mon[34488]: pgmap v205: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:41.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:41 smithi064 ceph-mon[30960]: pgmap v206: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:12:41.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:41 smithi103 ceph-mon[34446]: pgmap v206: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:12:41.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:41 smithi174 ceph-mon[34488]: pgmap v206: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:12:43.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:43 smithi064 ceph-mon[30960]: pgmap v207: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:43.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:43 smithi103 ceph-mon[34446]: pgmap v207: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:43 smithi174 ceph-mon[34488]: pgmap v207: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:45.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:45 smithi064 ceph-mon[30960]: pgmap v208: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:45.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:45 smithi103 ceph-mon[34446]: pgmap v208: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:45.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:45 smithi174 ceph-mon[34488]: pgmap v208: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:12:47.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:47 smithi064 ceph-mon[30960]: pgmap v209: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-09-19T05:12:47.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:47 smithi103 ceph-mon[34446]: pgmap v209: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-09-19T05:12:47.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:47 smithi174 ceph-mon[34488]: pgmap v209: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-09-19T05:12:49.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:49 smithi064 ceph-mon[30960]: pgmap v210: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:49.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:49 smithi103 ceph-mon[34446]: pgmap v210: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:49 smithi174 ceph-mon[34488]: pgmap v210: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:51.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:51 smithi064 ceph-mon[30960]: pgmap v211: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:51.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:51 smithi103 ceph-mon[34446]: pgmap v211: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:51.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:51 smithi174 ceph-mon[34488]: pgmap v211: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:53.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:53 smithi064 ceph-mon[30960]: pgmap v212: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:53.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:53 smithi103 ceph-mon[34446]: pgmap v212: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:53.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:53 smithi174 ceph-mon[34488]: pgmap v212: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:55.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:55 smithi064 ceph-mon[30960]: pgmap v213: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:55.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:55 smithi103 ceph-mon[34446]: pgmap v213: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:55 smithi174 ceph-mon[34488]: pgmap v213: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:12:57.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:57 smithi064 ceph-mon[30960]: pgmap v214: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:57.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:57 smithi103 ceph-mon[34446]: pgmap v214: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:57.538 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:57 smithi174 ceph-mon[34488]: pgmap v214: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:12:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:12:59 smithi064 ceph-mon[30960]: pgmap v215: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:12:59 smithi103 ceph-mon[34446]: pgmap v215: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:12:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:12:59 smithi174 ceph-mon[34488]: pgmap v215: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:01.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:01 smithi064 ceph-mon[30960]: pgmap v216: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:01.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:01 smithi103 ceph-mon[34446]: pgmap v216: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:01 smithi174 ceph-mon[34488]: pgmap v216: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:03.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:03 smithi064 ceph-mon[30960]: pgmap v217: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:03.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:03 smithi103 ceph-mon[34446]: pgmap v217: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:03.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:03 smithi174 ceph-mon[34488]: pgmap v217: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:03.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:04.044 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 -- 172.21.15.64:0/3484266858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 msgr2=0x7f23201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 --2- 172.21.15.64:0/3484266858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f23201067e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2308009900 tx=0x7f230802a410 comp rx=0 tx=0).stop 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 -- 172.21.15.64:0/3484266858 shutdown_connections 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 --2- 172.21.15.64:0/3484266858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320106f10 0x7f232010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 --2- 172.21.15.64:0/3484266858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f23201067e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 --2- 172.21.15.64:0/3484266858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320101de0 0x7f23201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 -- 172.21.15.64:0/3484266858 >> 172.21.15.64:0/3484266858 conn(0x7f23200fd8c0 msgr2=0x7f23200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 -- 172.21.15.64:0/3484266858 shutdown_connections 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.648+0000 7f232510e640 1 -- 172.21.15.64:0/3484266858 wait complete. 2024-09-19T05:13:04.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 Processor -- start 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 -- start start 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320101de0 0x7f23200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f2320074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320106f10 0x7f2320074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2320111e20 con 0x7f2320102790 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2320111ca0 con 0x7f2320101de0 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f232510e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2320111fa0 con 0x7f2320106f10 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f2317fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f2320074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f2317fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f2320074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32782/0 (socket says 172.21.15.64:32782) 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.649+0000 7f2317fff640 1 -- 172.21.15.64:0/1209739454 learned_addr learned my addr 172.21.15.64:0/1209739454 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f231ed9d640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320101de0 0x7f23200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f231f59e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320106f10 0x7f2320074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f2317fff640 1 -- 172.21.15.64:0/1209739454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320106f10 msgr2=0x7f2320074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f2317fff640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320106f10 0x7f2320074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f2317fff640 1 -- 172.21.15.64:0/1209739454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320101de0 msgr2=0x7f23200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f2317fff640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320101de0 0x7f23200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f2317fff640 1 -- 172.21.15.64:0/1209739454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23201a6860 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f231ed9d640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320101de0 0x7f23200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f2317fff640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f2320074a30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2308005940 tx=0x7f23080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f231cd99640 1 -- 172.21.15.64:0/1209739454 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230803f030 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23201a6af0 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f231cd99640 1 -- 172.21.15.64:0/1209739454 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2308003cb0 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f231cd99640 1 -- 172.21.15.64:0/1209739454 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308035070 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.650+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23201a7020 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.652+0000 7f231cd99640 1 -- 172.21.15.64:0/1209739454 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2308042030 con 0x7f2320102790 2024-09-19T05:13:04.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 Processor -- start 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 -- start start 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d72880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561ee5cd82e0 con 0x561ee5d7d680 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ee5c7fce0 con 0x561ee5d72dc0 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f23274e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ee5cd42a0 con 0x561ee5d724a0 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f2317fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f2317fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32790/0 (socket says 172.21.15.64:32790) 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f2317fff640 1 -- 172.21.15.64:0/3392428818 learned_addr learned my addr 172.21.15.64:0/3392428818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f231ed9d640 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.653+0000 7f231f59e640 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d72880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f231cd99640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f230c0a33b0 0x7f230c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f2317fff640 1 -- 172.21.15.64:0/3392428818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 msgr2=0x561ee5d72880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f2317fff640 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d72880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f2317fff640 1 -- 172.21.15.64:0/3392428818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 msgr2=0x561ee5d7d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f231f59e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f230c0a33b0 0x7f230c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f2317fff640 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f2317fff640 1 -- 172.21.15.64:0/3392428818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee5c64440 con 0x561ee5d7d680 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f231ed9d640 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f231cd99640 1 -- 172.21.15.64:0/1209739454 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f23080f0580 con 0x7f2320102790 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f2317fff640 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7fa70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f23080edc40 tx=0x7f230804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e1a0627c4b702e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f231f59e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f230c0a33b0 0x7f230c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2318005c40 tx=0x7f2318004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f23157fa640 1 -- 172.21.15.64:0/3392428818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080bc1c0 con 0x561ee5d7d680 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f23157fa640 1 -- 172.21.15.64:0/3392428818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f230803b460 con 0x561ee5d7d680 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.654+0000 7f23157fa640 1 -- 172.21.15.64:0/3392428818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080044e0 con 0x561ee5d7d680 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 -- 172.21.15.64:0/3392428818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 msgr2=0x561ee5d7fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7fa70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f23080edc40 tx=0x7f230804c000 comp rx=0 tx=0).stop 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 -- 172.21.15.64:0/3392428818 shutdown_connections 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7fa70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 --2- 172.21.15.64:0/3392428818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d72880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 -- 172.21.15.64:0/3392428818 >> 172.21.15.64:0/3392428818 conn(0x561ee5d6ea30 msgr2=0x561ee5d6ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 -- 172.21.15.64:0/3392428818 shutdown_connections 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 -- 172.21.15.64:0/3392428818 wait complete. 2024-09-19T05:13:04.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.655+0000 7f23274e7740 1 -- start start 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 Processor -- start 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561ee5d80010 con 0x561ee5d7d680 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ee5c93050 con 0x561ee5d72dc0 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ee5d80190 con 0x561ee5d724a0 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f23274e7740 10 client.0 _ll_register_callbacks cb 0x561ee5cd3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32800/0 (socket says 172.21.15.64:32800) 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 -- 172.21.15.64:0/8081328 learned_addr learned my addr 172.21.15.64:0/8081328 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f2317fff640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231ed9d640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 -- 172.21.15.64:0/8081328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 msgr2=0x561ee5d7cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d7cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 -- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 msgr2=0x561ee5d7a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.656+0000 7f231f59e640 1 -- 172.21.15.64:0/8081328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee5a493c0 con 0x561ee5d7d680 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f231f59e640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7a740 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f231800f960 tx=0x7f2318004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2008eae2d2c67121 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23067fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23067fc640 1 -- 172.21.15.64:0/8081328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318023070 con 0x561ee5d7d680 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23067fc640 1 -- 172.21.15.64:0/8081328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2318011da0 con 0x561ee5d7d680 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ee5ce5850 con 0x561ee5d7d680 2024-09-19T05:13:04.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23067fc640 1 -- 172.21.15.64:0/8081328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f231802f050 con 0x561ee5d7d680 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 10 client.17097 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ee5d8b930 con 0x561ee5d7d680 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23067fc640 1 -- 172.21.15.64:0/8081328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561ee5ce5850 con 0x561ee5d7d680 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23067fc640 1 -- 172.21.15.64:0/8081328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2318027440 con 0x561ee5d7d680 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 10 client.17097 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 20 client.17097 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5d8cd40 0x561ee5d8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 4 client.17097 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.657+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ee5cdfd60 con 0x561ee5d8cd40 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.658+0000 7f231ed9d640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5d8cd40 0x561ee5d8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.658+0000 7f23067fc640 10 client.17097 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:04.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.658+0000 7f231ed9d640 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5d8cd40 0x561ee5d8f130 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f21888ba898ac4 server_cookie=a1d180537b3c85d0 in_seq=0 out_seq=0 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23067fc640 1 -- 172.21.15.64:0/8081328 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ee5cdfd60 con 0x561ee5d8cd40 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23067fc640 10 client.17097 handle_command_reply: tid=0 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 client.17097 shutdown 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 msgr2=0x561ee5d7a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7a740 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f231800f960 tx=0x7f2318004a40 comp rx=0 tx=0).stop 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 shutdown_connections 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5d8cd40 0x561ee5d8f130 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d7a740 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d72dc0 0x561ee5d7a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23067fc640 0 client.17097 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 --2- 172.21.15.64:0/8081328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d724a0 0x561ee5d7cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 >> 172.21.15.64:0/8081328 conn(0x561ee5d6ea30 msgr2=0x561ee5d74760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 shutdown_connections 2024-09-19T05:13:04.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 1 -- 172.21.15.64:0/8081328 wait complete. 2024-09-19T05:13:04.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.659+0000 7f23274e7740 20 client.17097 trim_cache size 0 max 16384 2024-09-19T05:13:04.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 Processor -- start 2024-09-19T05:13:04.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 -- start start 2024-09-19T05:13:04.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d724a0 0x561ee5d89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23201b2390 con 0x561ee5d7d680 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23201afc60 con 0x561ee5d724a0 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f23274e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23201adab0 con 0x561ee5d72dc0 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f231f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f231f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d94f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32802/0 (socket says 172.21.15.64:32802) 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f231f59e640 1 -- 172.21.15.64:0/4109364173 learned_addr learned my addr 172.21.15.64:0/4109364173 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.677+0000 7f231ed9d640 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d724a0 0x561ee5d89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f2317fff640 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f231f59e640 1 -- 172.21.15.64:0/4109364173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 msgr2=0x561ee5d949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f231f59e640 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f231f59e640 1 -- 172.21.15.64:0/4109364173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d724a0 msgr2=0x561ee5d89180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f231f59e640 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d724a0 0x561ee5d89180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f231f59e640 1 -- 172.21.15.64:0/4109364173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee5d6e860 con 0x561ee5d7d680 2024-09-19T05:13:04.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f2317fff640 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:04.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f231f59e640 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d94f10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2318014da0 tx=0x7f231800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4955e747a82fc2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f2314ff9640 1 -- 172.21.15.64:0/4109364173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318037030 con 0x561ee5d7d680 2024-09-19T05:13:04.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f2314ff9640 1 -- 172.21.15.64:0/4109364173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2318018040 con 0x561ee5d7d680 2024-09-19T05:13:04.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.678+0000 7f2314ff9640 1 -- 172.21.15.64:0/4109364173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318023070 con 0x561ee5d7d680 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 -- 172.21.15.64:0/4109364173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 msgr2=0x561ee5d94f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d94f10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2318014da0 tx=0x7f231800f640 comp rx=0 tx=0).stop 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 -- 172.21.15.64:0/4109364173 shutdown_connections 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d7d680 0x561ee5d94f10 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 --2- 172.21.15.64:0/4109364173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d724a0 0x561ee5d89180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 -- 172.21.15.64:0/4109364173 >> 172.21.15.64:0/4109364173 conn(0x561ee5d6ea30 msgr2=0x561ee5d78ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 -- 172.21.15.64:0/4109364173 shutdown_connections 2024-09-19T05:13:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 -- 172.21.15.64:0/4109364173 wait complete. 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.679+0000 7f23274e7740 1 -- start start 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 Processor -- start 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 0x561ee5d852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d85830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d7d680 0x561ee5d822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23201ae120 con 0x561ee5d724a0 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23201af590 con 0x561ee5d7d680 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23201ae7f0 con 0x561ee5d72dc0 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f23274e7740 10 client.0 _ll_register_callbacks cb 0x7f2318014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 0x561ee5d852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 0x561ee5d852f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32808/0 (socket says 172.21.15.64:32808) 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 -- 172.21.15.64:0/1480851684 learned_addr learned my addr 172.21.15.64:0/1480851684 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231f59e640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d7d680 0x561ee5d822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 -- 172.21.15.64:0/1480851684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 msgr2=0x561ee5d85830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d85830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 -- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d7d680 msgr2=0x561ee5d822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d7d680 0x561ee5d822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231ed9d640 1 -- 172.21.15.64:0/1480851684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee5d82800 con 0x561ee5d724a0 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.680+0000 7f231f59e640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d7d680 0x561ee5d822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f231ed9d640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 0x561ee5d852f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x561ee5d8f940 tx=0x7f23100129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0bc2cae015f72a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23067fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23067fc640 1 -- 172.21.15.64:0/1480851684 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310011a30 con 0x561ee5d724a0 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23067fc640 1 -- 172.21.15.64:0/1480851684 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f2310011bd0 con 0x561ee5d724a0 2024-09-19T05:13:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2318018040 con 0x561ee5d724a0 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23067fc640 1 -- 172.21.15.64:0/1480851684 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2310011dd0 con 0x561ee5d724a0 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 10 client.17103 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ee5df8070 con 0x561ee5d724a0 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23067fc640 1 -- 172.21.15.64:0/1480851684 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2318018040 con 0x561ee5d724a0 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23067fc640 1 -- 172.21.15.64:0/1480851684 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2310025030 con 0x561ee5d724a0 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 10 client.17103 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 20 client.17103 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5df8870 0x561ee5dfacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 4 client.17103 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.681+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ee5d7aad0 con 0x561ee5df8870 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f2317fff640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5df8870 0x561ee5dfacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23067fc640 10 client.17103 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f2317fff640 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5df8870 0x561ee5dfacd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19516c1bfbddbb5 server_cookie=b5a268721c674e74 in_seq=0 out_seq=0 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23067fc640 1 -- 172.21.15.64:0/1480851684 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ee5d7aad0 con 0x561ee5df8870 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23067fc640 10 client.17103 handle_command_reply: tid=0 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 client.17103 shutdown 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 msgr2=0x561ee5d852f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 0x561ee5d852f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x561ee5d8f940 tx=0x7f23100129d0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 shutdown_connections 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ee5d7d680 0x561ee5d822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ee5d72dc0 0x561ee5d85830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561ee5df8870 0x561ee5dfacd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 --2- 172.21.15.64:0/1480851684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ee5d724a0 0x561ee5d852f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 >> 172.21.15.64:0/1480851684 conn(0x561ee5d6ea30 msgr2=0x561ee5d7ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23067fc640 0 client.17103 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.682+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 shutdown_connections 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f23274e7740 1 -- 172.21.15.64:0/1480851684 wait complete. 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f23274e7740 20 client.17103 trim_cache size 0 max 16384 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:13:04.692 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:13:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f230c0a33b0 msgr2=0x7f230c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f230c0a33b0 0x7f230c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2318005c40 tx=0x7f2318004ab0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 msgr2=0x7f2320074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f2320074a30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2308005940 tx=0x7f23080042f0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 shutdown_connections 2024-09-19T05:13:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f230c0a33b0 0x7f230c0a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2320106f10 0x7f2320074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2320102790 0x7f2320074a30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 --2- 172.21.15.64:0/1209739454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2320101de0 0x7f23200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 >> 172.21.15.64:0/1209739454 conn(0x7f23200fd8c0 msgr2=0x7f23200fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 shutdown_connections 2024-09-19T05:13:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:04.683+0000 7f232510e640 1 -- 172.21.15.64:0/1209739454 wait complete. 2024-09-19T05:13:05.323 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:05 smithi064 ceph-mon[30960]: pgmap v218: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:05.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:05 smithi103 ceph-mon[34446]: pgmap v218: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:05 smithi174 ceph-mon[34488]: pgmap v218: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:05.584 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:13:05.584 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:13:05.845 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:06.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 -- 172.21.15.64:0/2030720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c100e20 msgr2=0x7f705c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 --2- 172.21.15.64:0/2030720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c100e20 0x7f705c101200 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02a480 comp rx=0 tx=0).stop 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 -- 172.21.15.64:0/2030720805 shutdown_connections 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 --2- 172.21.15.64:0/2030720805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f705c102ef0 0x7f705c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 --2- 172.21.15.64:0/2030720805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f705c102570 0x7f705c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 --2- 172.21.15.64:0/2030720805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c100e20 0x7f705c101200 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 -- 172.21.15.64:0/2030720805 >> 172.21.15.64:0/2030720805 conn(0x7f705c0fc7d0 msgr2=0x7f705c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 -- 172.21.15.64:0/2030720805 shutdown_connections 2024-09-19T05:13:06.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 -- 172.21.15.64:0/2030720805 wait complete. 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.325+0000 7f7062180640 1 Processor -- start 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 -- start start 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f705c100e20 0x7f705c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 0x7f705c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f705c102ef0 0x7f705c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f705c110e50 con 0x7f705c102570 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f705c110cd0 con 0x7f705c102ef0 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f7062180640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f705c110fd0 con 0x7f705c100e20 2024-09-19T05:13:06.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 0x7f705c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 0x7f705c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32836/0 (socket says 172.21.15.64:32836) 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 -- 172.21.15.64:0/4044498285 learned_addr learned my addr 172.21.15.64:0/4044498285 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705b7fe640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f705c100e20 0x7f705c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705bfff640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f705c102ef0 0x7f705c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 -- 172.21.15.64:0/4044498285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f705c100e20 msgr2=0x7f705c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f705c100e20 0x7f705c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 -- 172.21.15.64:0/4044498285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f705c102ef0 msgr2=0x7f705c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.326+0000 7f705affd640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f705c102ef0 0x7f705c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f705affd640 1 -- 172.21.15.64:0/4044498285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c1a6960 con 0x7f705c102570 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f705affd640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 0x7f705c0fd1c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f705000b4a0 tx=0x7f705000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050014070 con 0x7f705c102570 2024-09-19T05:13:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a6c50 con 0x7f705c102570 2024-09-19T05:13:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7050010ac0 con 0x7f705c102570 2024-09-19T05:13:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050017c30 con 0x7f705c102570 2024-09-19T05:13:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.327+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a7140 con 0x7f705c102570 2024-09-19T05:13:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.328+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c101200 con 0x7f705c102570 2024-09-19T05:13:06.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.330+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7050017070 con 0x7f705c102570 2024-09-19T05:13:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.331+0000 7f7058ff9640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f70440a33b0 0x7f70440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.331+0000 7f705b7fe640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f70440a33b0 0x7f70440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.331+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f70500cf3f0 con 0x7f705c102570 2024-09-19T05:13:06.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.331+0000 7f705b7fe640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f70440a33b0 0x7f70440a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f704c009230 tx=0x7f704c0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.334+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f705009bee0 con 0x7f705c102570 2024-09-19T05:13:06.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.549+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f705c0fe1b0 con 0x7f705c102570 2024-09-19T05:13:06.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.550+0000 7f7058ff9640 1 -- 172.21.15.64:0/4044498285 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f705c0fe1b0 con 0x7f705c102570 2024-09-19T05:13:06.552 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:13:06.552 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:13:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:13:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f70440a33b0 msgr2=0x7f70440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f70440a33b0 0x7f70440a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f704c009230 tx=0x7f704c0058d0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 msgr2=0x7f705c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 0x7f705c0fd1c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f705000b4a0 tx=0x7f705000b960 comp rx=0 tx=0).stop 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 shutdown_connections 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f70440a33b0 0x7f70440a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f705c102ef0 0x7f705c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f705c102570 0x7f705c0fd1c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 --2- 172.21.15.64:0/4044498285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f705c100e20 0x7f705c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 >> 172.21.15.64:0/4044498285 conn(0x7f705c0fc7d0 msgr2=0x7f705c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 shutdown_connections 2024-09-19T05:13:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:06.553+0000 7f7062180640 1 -- 172.21.15.64:0/4044498285 wait complete. 2024-09-19T05:13:07.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:13:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:07 smithi064 ceph-mon[30960]: pgmap v219: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T05:13:07.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:07 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4044498285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:07.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:07 smithi103 ceph-mon[34446]: pgmap v219: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T05:13:07.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:07 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4044498285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:07 smithi174 ceph-mon[34488]: pgmap v219: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-19T05:13:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:07 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4044498285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/4010221918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74106f10 msgr2=0x7f5f7410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/4010221918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74106f10 0x7f5f7410ea90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009ca0 tx=0x7f5f6c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/4010221918 shutdown_connections 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/4010221918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74106f10 0x7f5f7410ea90 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/4010221918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f74102790 0x7f5f741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/4010221918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 0x7f5f741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/4010221918 >> 172.21.15.64:0/4010221918 conn(0x7f5f740fd8c0 msgr2=0x7f5f740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/4010221918 shutdown_connections 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.976+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/4010221918 wait complete. 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 Processor -- start 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 -- start start 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 0x7f5f74198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 0x7f5f74198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f74106f10 0x7f5f7419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111e20 con 0x7f5f74102790 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111ca0 con 0x7f5f74106f10 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7c2d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111fa0 con 0x7f5f74101de0 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 0x7f5f74198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 0x7f5f74198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32858/0 (socket says 172.21.15.64:32858) 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 -- 172.21.15.64:0/814178790 learned_addr learned my addr 172.21.15.64:0/814178790 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7a875640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f74106f10 0x7f5f7419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7a074640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 0x7f5f74198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 -- 172.21.15.64:0/814178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 msgr2=0x7f5f74198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 0x7f5f74198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 -- 172.21.15.64:0/814178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f74106f10 msgr2=0x7f5f7419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f74106f10 0x7f5f7419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f79873640 1 -- 172.21.15.64:0/814178790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7419d2f0 con 0x7f5f74102790 2024-09-19T05:13:07.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.977+0000 7f5f7a074640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 0x7f5f74198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.978+0000 7f5f79873640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 0x7f5f74198880 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e9f0 tx=0x7f5f6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.978+0000 7f5f677fe640 1 -- 172.21.15.64:0/814178790 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800cd80 con 0x7f5f74102790 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.978+0000 7f5f677fe640 1 -- 172.21.15.64:0/814178790 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f680108e0 con 0x7f5f74102790 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.978+0000 7f5f677fe640 1 -- 172.21.15.64:0/814178790 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68018920 con 0x7f5f74102790 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.978+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7419d5e0 con 0x7f5f74102790 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.978+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741a4ec0 con 0x7f5f74102790 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.980+0000 7f5f677fe640 1 -- 172.21.15.64:0/814178790 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f5f68010a90 con 0x7f5f74102790 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.980+0000 7f5f7e6b1740 1 Processor -- start 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 -- start start 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6c232f0 con 0x55d1b6cbeb40 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6bcacf0 con 0x55d1b6ccc780 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6c1f2b0 con 0x55d1b6cbd5d0 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f677fe640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5f600a33b0 0x7f5f600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7a875640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7a875640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32868/0 (socket says 172.21.15.64:32868) 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7a875640 1 -- 172.21.15.64:0/1262797373 learned_addr learned my addr 172.21.15.64:0/1262797373 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f79873640 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7a875640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5f600a33b0 0x7f5f600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f7a074640 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.981+0000 7f5f677fe640 1 -- 172.21.15.64:0/814178790 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5f680d0060 con 0x7f5f74102790 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f79873640 1 -- 172.21.15.64:0/1262797373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 msgr2=0x55d1b6cbda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:07.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f79873640 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cbda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f79873640 1 -- 172.21.15.64:0/1262797373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 msgr2=0x55d1b6cbd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f79873640 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f79873640 1 -- 172.21.15.64:0/1262797373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b6baf3a0 con 0x55d1b6ccc780 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f79873640 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cceb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f680cd7b0 tx=0x7f5f6801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77494411f904a5ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f7a875640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5f600a33b0 0x7f5f600a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c005fe0 tx=0x7f5f6c004a60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f64dea640 1 -- 172.21.15.64:0/1262797373 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6809ca40 con 0x55d1b6ccc780 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f64dea640 1 -- 172.21.15.64:0/1262797373 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f6809c340 con 0x55d1b6ccc780 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.982+0000 7f5f64dea640 1 -- 172.21.15.64:0/1262797373 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680cdbc0 con 0x55d1b6ccc780 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1262797373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 msgr2=0x55d1b6cceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cceb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f680cd7b0 tx=0x7f5f6801c830 comp rx=0 tx=0).stop 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1262797373 shutdown_connections 2024-09-19T05:13:07.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cceb70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cbda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1262797373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1262797373 >> 172.21.15.64:0/1262797373 conn(0x55d1b6c86310 msgr2=0x55d1b6cb9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1262797373 shutdown_connections 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1262797373 wait complete. 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 -- start start 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.983+0000 7f5f7e6b1740 1 Processor -- start 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6ccc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6ccf480 con 0x55d1b6ccc780 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6bde060 con 0x55d1b6cbeb40 2024-09-19T05:13:07.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6ccf600 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7e6b1740 10 client.0 _ll_register_callbacks cb 0x55d1b69318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f79873640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f79873640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:55734/0 (socket says 172.21.15.64:55734) 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32880/0 (socket says 172.21.15.64:32880) 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f79873640 1 -- 172.21.15.64:0/2918590524 learned_addr learned my addr 172.21.15.64:0/2918590524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a074640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6ccc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 -- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 msgr2=0x55d1b6ccc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6ccc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 -- 172.21.15.64:0/2918590524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 msgr2=0x55d1b6cc9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 -- 172.21.15.64:0/2918590524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b69943c0 con 0x55d1b6ccc780 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f79873640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a074640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6ccc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f7a875640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc9c10 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c006010 tx=0x7f5f6c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c6c6aa232228f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.984+0000 7f5f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f5dffb640 1 -- 172.21.15.64:0/2918590524 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004450 con 0x55d1b6ccc780 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f5dffb640 1 -- 172.21.15.64:0/2918590524 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5f6c0045f0 con 0x55d1b6ccc780 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6809c340 con 0x55d1b6ccc780 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f5dffb640 1 -- 172.21.15.64:0/2918590524 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f6c0047f0 con 0x55d1b6ccc780 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 10 client.17121 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1b6cdaf90 con 0x55d1b6ccc780 2024-09-19T05:13:08.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f5dffb640 1 -- 172.21.15.64:0/2918590524 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f6809c340 con 0x55d1b6ccc780 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f5dffb640 1 -- 172.21.15.64:0/2918590524 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f6c02f5e0 con 0x55d1b6ccc780 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 10 client.17121 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 20 client.17121 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6cdc3a0 0x55d1b6cde7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 4 client.17121 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.985+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b6c30860 con 0x55d1b6cdc3a0 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.986+0000 7f5f7a074640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6cdc3a0 0x55d1b6cde7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.986+0000 7f5f5dffb640 10 client.17121 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.986+0000 7f5f7a074640 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6cdc3a0 0x55d1b6cde7d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b1f14d7bea8f919 server_cookie=13d3a45a0847e601 in_seq=0 out_seq=0 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f5dffb640 1 -- 172.21.15.64:0/2918590524 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1b6c30860 con 0x55d1b6cdc3a0 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f5dffb640 10 client.17121 handle_command_reply: tid=0 2024-09-19T05:13:08.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 client.17121 shutdown 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 msgr2=0x55d1b6cc9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc9c10 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c006010 tx=0x7f5f6c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 shutdown_connections 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6cdc3a0 0x55d1b6cde7d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc9c10 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f5dffb640 0 client.17121 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2918590524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6ccc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 >> 172.21.15.64:0/2918590524 conn(0x55d1b6c86310 msgr2=0x55d1b6cbf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 shutdown_connections 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2918590524 wait complete. 2024-09-19T05:13:08.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:07.987+0000 7f5f7e6b1740 20 client.17121 trim_cache size 0 max 16384 2024-09-19T05:13:08.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 Processor -- start 2024-09-19T05:13:08.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 -- start start 2024-09-19T05:13:08.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cb9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741adf50 con 0x55d1b6cbeb40 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741ab820 con 0x55d1b6ccc780 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741a9670 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f79873640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.000+0000 7f5f79873640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32890/0 (socket says 172.21.15.64:32890) 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 -- 172.21.15.64:0/2170061570 learned_addr learned my addr 172.21.15.64:0/2170061570 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f7a074640 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cb9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f7a875640 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 -- 172.21.15.64:0/2170061570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 msgr2=0x55d1b6cb9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cb9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 -- 172.21.15.64:0/2170061570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 msgr2=0x55d1b6cc8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 -- 172.21.15.64:0/2170061570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b6ccc540 con 0x55d1b6cbeb40 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f79873640 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd9b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800cfd0 tx=0x7f5f6809bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f29efbcb7cdc7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f5ffff640 1 -- 172.21.15.64:0/2170061570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680a4fb0 con 0x55d1b6cbeb40 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f5ffff640 1 -- 172.21.15.64:0/2170061570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f680a06f0 con 0x55d1b6cbeb40 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.001+0000 7f5f5ffff640 1 -- 172.21.15.64:0/2170061570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6809b790 con 0x55d1b6cbeb40 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2170061570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 msgr2=0x55d1b6cbd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd9b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800cfd0 tx=0x7f5f6809bdc0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2170061570 shutdown_connections 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cbd9b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/2170061570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cb9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2170061570 >> 172.21.15.64:0/2170061570 conn(0x55d1b6c86310 msgr2=0x55d1b6cb9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2170061570 shutdown_connections 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/2170061570 wait complete. 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 -- start start 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.002+0000 7f5f7e6b1740 1 Processor -- start 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cd4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741a9ce0 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741ab150 con 0x55d1b6cbeb40 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741aa3b0 con 0x55d1b6ccc780 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 10 client.0 _ll_register_callbacks cb 0x7f5f6c032f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cd4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a875640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cd4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32902/0 (socket says 172.21.15.64:32902) 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 -- 172.21.15.64:0/1488688283 learned_addr learned my addr 172.21.15.64:0/1488688283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f79873640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 -- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6ccc780 msgr2=0x55d1b6cc6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 -- 172.21.15.64:0/1488688283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 msgr2=0x55d1b6cc5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 -- 172.21.15.64:0/1488688283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b6cc6940 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a875640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f79873640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:08.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7a074640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cd4d10 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x55d1b6cdf010 tx=0x7f5f7001d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b0da3cb2cd53f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f5dffb640 1 -- 172.21.15.64:0/1488688283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70011b30 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f5dffb640 1 -- 172.21.15.64:0/1488688283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8527+0+0 (secure 0 0 0) 0x7f5f70011cd0 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.003+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f680a06f0 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f5dffb640 1 -- 172.21.15.64:0/1488688283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f70026450 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 10 client.17127 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1b6d46de0 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f5dffb640 1 -- 172.21.15.64:0/1488688283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f680a06f0 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f5dffb640 1 -- 172.21.15.64:0/1488688283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f7002f770 con 0x55d1b6cbd5d0 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 10 client.17127 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 10 client.17127 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 20 client.17127 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:08.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6d474f0 0x55d1b6d49910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 4 client.17127 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1b6c2ad70 con 0x55d1b6d474f0 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.004+0000 7f5f79873640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6d474f0 0x55d1b6d49910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f79873640 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6d474f0 0x55d1b6d49910 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82bf71cdc294dc5 server_cookie=fc7fcd4de30417af in_seq=0 out_seq=0 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f5dffb640 10 client.17127 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f5dffb640 1 -- 172.21.15.64:0/1488688283 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1b6c2ad70 con 0x55d1b6d474f0 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f5dffb640 10 client.17127 handle_command_reply: tid=0 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 client.17127 shutdown 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 msgr2=0x55d1b6cd4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cd4d10 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x55d1b6cdf010 tx=0x7f5f7001d970 comp rx=0 tx=0).stop 2024-09-19T05:13:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 shutdown_connections 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x55d1b6d474f0 0x55d1b6d49910 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1b6ccc780 0x55d1b6cc6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b6cbeb40 0x55d1b6cc5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 --2- 172.21.15.64:0/1488688283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1b6cbd5d0 0x55d1b6cd4d10 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 >> 172.21.15.64:0/1488688283 conn(0x55d1b6c86310 msgr2=0x55d1b6ccace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f5dffb640 0 client.17127 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 shutdown_connections 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 1 -- 172.21.15.64:0/1488688283 wait complete. 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.005+0000 7f5f7e6b1740 20 client.17127 trim_cache size 0 max 16384 2024-09-19T05:13:08.015 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:13:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5f600a33b0 msgr2=0x7f5f600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5f600a33b0 0x7f5f600a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c005fe0 tx=0x7f5f6c004a60 comp rx=0 tx=0).stop 2024-09-19T05:13:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 msgr2=0x7f5f74198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 0x7f5f74198880 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e9f0 tx=0x7f5f6800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 shutdown_connections 2024-09-19T05:13:08.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f5f600a33b0 0x7f5f600a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f74106f10 0x7f5f7419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f74102790 0x7f5f74198880 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 --2- 172.21.15.64:0/814178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f74101de0 0x7f5f74198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 >> 172.21.15.64:0/814178790 conn(0x7f5f740fd8c0 msgr2=0x7f5f740fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 shutdown_connections 2024-09-19T05:13:08.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:08.006+0000 7f5f7c2d8640 1 -- 172.21.15.64:0/814178790 wait complete. 2024-09-19T05:13:08.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:13:08.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:13:08.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:13:08.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:13:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:13:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:13:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:13:08.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:13:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:13:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:13:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:13:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:13:08.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:13:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:09.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:09 smithi064 ceph-mon[30960]: pgmap v220: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T05:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.378+0000 7fc3f85e3640 1 -- 172.21.15.64:0/3637862683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 msgr2=0x7fc3f010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.378+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/3637862683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f010ec30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009cd0 tx=0x7fc3e802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 -- 172.21.15.64:0/3637862683 shutdown_connections 2024-09-19T05:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/3637862683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f010ec30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/3637862683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3f0103740 0x7fc3f0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/3637862683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0102d90 0x7fc3f0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 -- 172.21.15.64:0/3637862683 >> 172.21.15.64:0/3637862683 conn(0x7fc3f00fc7d0 msgr2=0x7fc3f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 -- 172.21.15.64:0/3637862683 shutdown_connections 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 -- 172.21.15.64:0/3637862683 wait complete. 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 Processor -- start 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.379+0000 7fc3f85e3640 1 -- start start 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f85e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3f0102d90 0x7fc3f0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f85e3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0103740 0x7fc3f0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f85e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f85e3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0111fd0 con 0x7fc3f0108290 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f85e3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0111e50 con 0x7fc3f0103740 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f85e3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0112150 con 0x7fc3f0102d90 2024-09-19T05:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32928/0 (socket says 172.21.15.64:32928) 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 -- 172.21.15.64:0/763265494 learned_addr learned my addr 172.21.15.64:0/763265494 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f637f640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3f0102d90 0x7fc3f0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0103740 0x7fc3f0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 -- 172.21.15.64:0/763265494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3f0102d90 msgr2=0x7fc3f0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3f0102d90 0x7fc3f0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 -- 172.21.15.64:0/763265494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0103740 msgr2=0x7fc3f0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0103740 0x7fc3f0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 -- 172.21.15.64:0/763265494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f019d4b0 con 0x7fc3f0108290 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0103740 0x7fc3f0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.380+0000 7fc3f6b80640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f019cdd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005e30 tx=0x7fc3e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.381+0000 7fc3e77fe640 1 -- 172.21.15.64:0/763265494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8004240 con 0x7fc3f0108290 2024-09-19T05:13:09.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.381+0000 7fc3e77fe640 1 -- 172.21.15.64:0/763265494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e8031ce0 con 0x7fc3f0108290 2024-09-19T05:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.381+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f019d740 con 0x7fc3f0108290 2024-09-19T05:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.381+0000 7fc3e77fe640 1 -- 172.21.15.64:0/763265494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8046480 con 0x7fc3f0108290 2024-09-19T05:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.381+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f01a5080 con 0x7fc3f0108290 2024-09-19T05:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.384+0000 7fc3fa9bc740 1 Processor -- start 2024-09-19T05:13:09.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.384+0000 7fc3e77fe640 1 -- 172.21.15.64:0/763265494 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc3e80043e0 con 0x7fc3f0108290 2024-09-19T05:13:09.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.384+0000 7fc3fa9bc740 1 -- start start 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561170e862f0 con 0x561170f2b370 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561170e2dcf0 con 0x561170f21b40 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561170e822b0 con 0x561170f205d0 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32940/0 (socket says 172.21.15.64:32940) 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/188570858 learned_addr learned my addr 172.21.15.64:0/188570858 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f637f640 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3e77fe640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc3dc0a3430 0x7fc3dc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f6b80640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc3dc0a3430 0x7fc3dc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/188570858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 msgr2=0x561170f20a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.385+0000 7fc3e77fe640 1 -- 172.21.15.64:0/763265494 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fc3e802e030 con 0x7fc3f0108290 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f20a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/188570858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 msgr2=0x561170f2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/188570858 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561170e123a0 con 0x561170f21b40 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f637f640 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f20090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00188b0 tx=0x7fc3e0006f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cc7be4da8268a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3f6b80640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc3dc0a3430 0x7fc3dc0a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80b9da0 tx=0x7fc3e80095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3e4ff9640 1 -- 172.21.15.64:0/188570858 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0012670 con 0x561170f21b40 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3e4ff9640 1 -- 172.21.15.64:0/188570858 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e001e070 con 0x561170f21b40 2024-09-19T05:13:09.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.386+0000 7fc3e4ff9640 1 -- 172.21.15.64:0/188570858 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000f620 con 0x561170f21b40 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/188570858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 msgr2=0x561170f20090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f20090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3e00188b0 tx=0x7fc3e0006f90 comp rx=0 tx=0).stop 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/188570858 shutdown_connections 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/188570858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f20090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/188570858 >> 172.21.15.64:0/188570858 conn(0x561170ee9310 msgr2=0x561170f1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/188570858 shutdown_connections 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/188570858 wait complete. 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.387+0000 7fc3fa9bc740 1 -- start start 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 Processor -- start 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f2ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f27fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f28530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561170f2e010 con 0x561170f2b370 2024-09-19T05:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561170e41060 con 0x561170f205d0 2024-09-19T05:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561170f2e190 con 0x561170f21b40 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3fa9bc740 10 client.0 _ll_register_callbacks cb 0x561170b948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f28530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f28530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32944/0 (socket says 172.21.15.64:32944) 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3466814122 learned_addr learned my addr 172.21.15.64:0/3466814122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f2ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f27fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 msgr2=0x561170f27fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f27fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3466814122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 msgr2=0x561170f2ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f2ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.388+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3466814122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561170bf73c0 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f28530 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000a5b0 tx=0x7fc3e0006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5513f5154e303695 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3466814122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000fc50 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3466814122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fc3e000fdf0 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e001e070 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3466814122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3e0036050 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3fa9bc740 10 client.17139 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561170f39a40 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.389+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3466814122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e0012b20 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3466814122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc3e0022660 con 0x561170f2b370 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3fa9bc740 10 client.17139 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:09.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3fa9bc740 10 client.17139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3fa9bc740 20 client.17139 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170f3adf0 0x561170f3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3fa9bc740 4 client.17139 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc3e001e070 con 0x561170f3adf0 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170f3adf0 0x561170f3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3d9ffb640 10 client.17139 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.390+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170f3adf0 0x561170f3d1e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=676a5b96519dedae server_cookie=ad640b19509c00eb in_seq=0 out_seq=0 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3466814122 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc3e001e070 con 0x561170f3adf0 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3d9ffb640 10 client.17139 handle_command_reply: tid=0 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 client.17139 shutdown 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 msgr2=0x561170f28530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f28530 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000a5b0 tx=0x7fc3e0006a30 comp rx=0 tx=0).stop 2024-09-19T05:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 shutdown_connections 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170f3adf0 0x561170f3d1e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f2b370 0x561170f28530 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f27fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3466814122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f2ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 >> 172.21.15.64:0/3466814122 conn(0x561170ee9310 msgr2=0x561170f22960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.391+0000 7fc3d9ffb640 0 client.17139 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.392+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 shutdown_connections 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.392+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3466814122 wait complete. 2024-09-19T05:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.392+0000 7fc3fa9bc740 20 client.17139 trim_cache size 0 max 16384 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 Processor -- start 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 -- start start 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f23120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01ae010 con 0x561170f23660 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01ab8e0 con 0x561170f205d0 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01a9770 con 0x561170f21b40 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32958/0 (socket says 172.21.15.64:32958) 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3582587317 learned_addr learned my addr 172.21.15.64:0/3582587317 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.405+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f23120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3582587317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 msgr2=0x561170f23120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f23120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3582587317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 msgr2=0x561170f372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3582587317 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561170f2e180 con 0x561170f23660 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f427e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000ad20 tx=0x7fc3e0006aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597b6951049246cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3dbfff640 1 -- 172.21.15.64:0/3582587317 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000f0b0 con 0x561170f23660 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3dbfff640 1 -- 172.21.15.64:0/3582587317 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e001e050 con 0x561170f23660 2024-09-19T05:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.406+0000 7fc3dbfff640 1 -- 172.21.15.64:0/3582587317 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0012640 con 0x561170f23660 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3582587317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 msgr2=0x561170f427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f427e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000ad20 tx=0x7fc3e0006aa0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3582587317 shutdown_connections 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f427e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f21b40 0x561170f23120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3582587317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f205d0 0x561170f372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3582587317 >> 172.21.15.64:0/3582587317 conn(0x561170ee9310 msgr2=0x561170f270a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3582587317 shutdown_connections 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3582587317 wait complete. 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- start start 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 Processor -- start 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f2f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f2f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f2ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01a9dc0 con 0x561170f23660 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01ab210 con 0x561170f21b40 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.407+0000 7fc3fa9bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01aa470 con 0x561170f205d0 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3fa9bc740 10 client.0 _ll_register_callbacks cb 0x7fc3e000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f2ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f2ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32972/0 (socket says 172.21.15.64:32972) 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3526095342 learned_addr learned my addr 172.21.15.64:0/3526095342 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f2f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f2f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 msgr2=0x561170f2f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f2f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3526095342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 msgr2=0x561170f2f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f2f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 -- 172.21.15.64:0/3526095342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561170fa4a80 con 0x561170f23660 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f2f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3f5b7e640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f2ff00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000f7e0 tx=0x7fc3e0031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70010bf1e96c0516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3526095342 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000a980 con 0x561170f23660 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3526095342 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fc3e0022640 con 0x561170f23660 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e001e050 con 0x561170f23660 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.408+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3526095342 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3e004c050 con 0x561170f23660 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 10 client.17145 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:09.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561170fa5d80 con 0x561170f23660 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3526095342 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e001e050 con 0x561170f23660 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3526095342 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc3e0012b00 con 0x561170f23660 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 10 client.17145 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 10 client.17145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 20 client.17145 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170fa6430 0x561170fa8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 4 client.17145 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561170e93860 con 0x561170fa6430 2024-09-19T05:13:09.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.409+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170fa6430 0x561170fa8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3f637f640 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170fa6430 0x561170fa8850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74f39ec278acb5e server_cookie=b65f957fa1243d9d in_seq=0 out_seq=0 2024-09-19T05:13:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3d9ffb640 10 client.17145 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:09.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3d9ffb640 1 -- 172.21.15.64:0/3526095342 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561170e93860 con 0x561170fa6430 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3d9ffb640 10 client.17145 handle_command_reply: tid=0 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 client.17145 shutdown 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 msgr2=0x561170f2ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f2ff00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000f7e0 tx=0x7fc3e0031040 comp rx=0 tx=0).stop 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 shutdown_connections 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x561170fa6430 0x561170fa8850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561170f23660 0x561170f2ff00 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561170f21b40 0x561170f2f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 --2- 172.21.15.64:0/3526095342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561170f205d0 0x561170f2f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 >> 172.21.15.64:0/3526095342 conn(0x561170ee9310 msgr2=0x561170f29810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3d9ffb640 0 client.17145 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:09.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 shutdown_connections 2024-09-19T05:13:09.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 1 -- 172.21.15.64:0/3526095342 wait complete. 2024-09-19T05:13:09.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.410+0000 7fc3fa9bc740 20 client.17145 trim_cache size 0 max 16384 2024-09-19T05:13:09.419 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:13:09.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc3dc0a3430 msgr2=0x7fc3dc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc3dc0a3430 0x7fc3dc0a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80b9da0 tx=0x7fc3e80095c0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 msgr2=0x7fc3f019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f019cdd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005e30 tx=0x7fc3e8003ba0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 shutdown_connections 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc3dc0a3430 0x7fc3dc0a58f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3f0108290 0x7fc3f019cdd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3f0103740 0x7fc3f0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 --2- 172.21.15.64:0/763265494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3f0102d90 0x7fc3f0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 >> 172.21.15.64:0/763265494 conn(0x7fc3f00fc7d0 msgr2=0x7fc3f0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:09.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 shutdown_connections 2024-09-19T05:13:09.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:09.411+0000 7fc3f85e3640 1 -- 172.21.15.64:0/763265494 wait complete. 2024-09-19T05:13:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:09 smithi103 ceph-mon[34446]: pgmap v220: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T05:13:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:09 smithi174 ceph-mon[34488]: pgmap v220: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-19T05:13:10.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:10 smithi064 ceph-mon[30960]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:13:10.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:13:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:10 smithi103 ceph-mon[34446]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:13:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:10 smithi174 ceph-mon[34488]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:13:10.558 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 -- 172.21.15.64:0/1546201300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c10c180 msgr2=0x7f403c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 --2- 172.21.15.64:0/1546201300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c10c180 0x7f403c10e600 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f402c009900 tx=0x7f402c02a480 comp rx=0 tx=0).stop 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 -- 172.21.15.64:0/1546201300 shutdown_connections 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 --2- 172.21.15.64:0/1546201300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c10c180 0x7f403c10e600 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 --2- 172.21.15.64:0/1546201300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c1036c0 0x7f403c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 --2- 172.21.15.64:0/1546201300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c102da0 0x7f403c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 -- 172.21.15.64:0/1546201300 >> 172.21.15.64:0/1546201300 conn(0x7f403c0fc7a0 msgr2=0x7f403c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 -- 172.21.15.64:0/1546201300 shutdown_connections 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.006+0000 7f404087a640 1 -- 172.21.15.64:0/1546201300 wait complete. 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 Processor -- start 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 -- start start 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c102da0 0x7f403c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 0x7f403c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c10c180 0x7f403c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f403c111e10 con 0x7f403c1036c0 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f403c111c90 con 0x7f403c102da0 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f404087a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f403c111f90 con 0x7f403c10c180 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f4039d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 0x7f403c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f403a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c102da0 0x7f403c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f4039d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 0x7f403c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32986/0 (socket says 172.21.15.64:32986) 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f4039d9b640 1 -- 172.21.15.64:0/4051023288 learned_addr learned my addr 172.21.15.64:0/4051023288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.007+0000 7f403a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c102da0 0x7f403c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:55814/0 (socket says 172.21.15.64:55814) 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f4039d9b640 1 -- 172.21.15.64:0/4051023288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c10c180 msgr2=0x7f403c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f4039d9b640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c10c180 0x7f403c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f4039d9b640 1 -- 172.21.15.64:0/4051023288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c102da0 msgr2=0x7f403c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f4039d9b640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c102da0 0x7f403c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f4039d9b640 1 -- 172.21.15.64:0/4051023288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c19d2c0 con 0x7f403c1036c0 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f4039d9b640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 0x7f403c198850 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f402400c890 tx=0x7f402400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f40337fe640 1 -- 172.21.15.64:0/4051023288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024004220 con 0x7f403c1036c0 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c19d5b0 con 0x7f403c1036c0 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f40337fe640 1 -- 172.21.15.64:0/4051023288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40240043c0 con 0x7f403c1036c0 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f40337fe640 1 -- 172.21.15.64:0/4051023288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024004a00 con 0x7f403c1036c0 2024-09-19T05:13:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.008+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a4e90 con 0x7f403c1036c0 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f40337fe640 1 -- 172.21.15.64:0/4051023288 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4024004ba0 con 0x7f403c1036c0 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 Processor -- start 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 -- start start 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582676e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x55582676ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582677d780 0x55582677fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5558266d42f0 con 0x55582676e5d0 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55582667bcf0 con 0x55582677d780 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f4042c53740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5558266d02b0 con 0x55582676fb40 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f403a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x55582676ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.011+0000 7f403a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x55582676ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32996/0 (socket says 172.21.15.64:32996) 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 -- 172.21.15.64:0/2666821466 learned_addr learned my addr 172.21.15.64:0/2666821466 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f4039d9b640 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582677d780 0x55582677fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403ad9d640 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582676e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f40337fe640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f40280a33b0 0x7f40280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 -- 172.21.15.64:0/2666821466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 msgr2=0x55582676e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582676e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 -- 172.21.15.64:0/2666821466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582677d780 msgr2=0x55582677fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582677d780 0x55582677fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 -- 172.21.15.64:0/2666821466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558266603a0 con 0x55582676e5d0 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f40337fe640 1 -- 172.21.15.64:0/4051023288 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f40240cfa10 con 0x7f403c1036c0 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403ad9d640 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582676e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403ad9d640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f40280a33b0 0x7f40280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f403a59c640 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x55582676ea10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f40340103f0 tx=0x7f40340108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f4c706a69d9757 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f4030ff9640 1 -- 172.21.15.64:0/2666821466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034016070 con 0x55582676e5d0 2024-09-19T05:13:11.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f4030ff9640 1 -- 172.21.15.64:0/2666821466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4034004d20 con 0x55582676e5d0 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.012+0000 7f4030ff9640 1 -- 172.21.15.64:0/2666821466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034005440 con 0x55582676e5d0 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 -- 172.21.15.64:0/2666821466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 msgr2=0x55582676ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f403ad9d640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f40280a33b0 0x7f40280a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f402c009230 tx=0x7f402c009260 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x55582676ea10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f40340103f0 tx=0x7f40340108b0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 -- 172.21.15.64:0/2666821466 shutdown_connections 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582677d780 0x55582677fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x55582676ea10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 --2- 172.21.15.64:0/2666821466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582676e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 -- 172.21.15.64:0/2666821466 >> 172.21.15.64:0/2666821466 conn(0x555826737310 msgr2=0x55582676ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 -- 172.21.15.64:0/2666821466 shutdown_connections 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.013+0000 7f4042c53740 1 -- 172.21.15.64:0/2666821466 wait complete. 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 -- start start 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 Processor -- start 2024-09-19T05:13:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x55582677d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582677a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 0x55582677aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555826780480 con 0x55582677d780 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55582668f060 con 0x55582676e5d0 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555826780600 con 0x55582676fb40 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f4042c53740 10 client.0 _ll_register_callbacks cb 0x5558263e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f403ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 0x55582677aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f403ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 0x55582677aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33000/0 (socket says 172.21.15.64:33000) 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f403ad9d640 1 -- 172.21.15.64:0/1544221817 learned_addr learned my addr 172.21.15.64:0/1544221817 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.014+0000 7f403a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x55582677d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4039d9b640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582677a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403ad9d640 1 -- 172.21.15.64:0/1544221817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 msgr2=0x55582677a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403ad9d640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582677a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403ad9d640 1 -- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 msgr2=0x55582677d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403ad9d640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x55582677d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403ad9d640 1 -- 172.21.15.64:0/1544221817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558264453c0 con 0x55582677d780 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403a59c640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x55582677d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4039d9b640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582677a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f403ad9d640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 0x55582677aa30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f402c0042f0 tx=0x7f402c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a892b3524c2d85ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4021ffb640 1 -- 172.21.15.64:0/1544221817 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c035040 con 0x55582677d780 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4021ffb640 1 -- 172.21.15.64:0/1544221817 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f402c0046c0 con 0x55582677d780 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4034004d20 con 0x55582677d780 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4021ffb640 1 -- 172.21.15.64:0/1544221817 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f402c056050 con 0x55582677d780 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4042c53740 10 client.17157 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55582678bf10 con 0x55582677d780 2024-09-19T05:13:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.015+0000 7f4021ffb640 1 -- 172.21.15.64:0/1544221817 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4034004d20 con 0x55582677d780 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4021ffb640 1 -- 172.21.15.64:0/1544221817 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f402c003cd0 con 0x55582677d780 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4042c53740 10 client.17157 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4042c53740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4042c53740 20 client.17157 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4042c53740 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x55582678f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4042c53740 4 client.17157 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558266e1860 con 0x55582678d440 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f403a59c640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x55582678f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f403a59c640 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x55582678f8e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78594559784c9c10 server_cookie=4a7245d47ec53fe3 in_seq=0 out_seq=0 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.016+0000 7f4021ffb640 10 client.17157 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4021ffb640 1 -- 172.21.15.64:0/1544221817 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558266e1860 con 0x55582678d440 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4021ffb640 10 client.17157 handle_command_reply: tid=0 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 client.17157 shutdown 2024-09-19T05:13:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 msgr2=0x55582677aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 0x55582677aa30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f402c0042f0 tx=0x7f402c031400 comp rx=0 tx=0).stop 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 shutdown_connections 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x55582678f8e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4021ffb640 0 client.17157 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582677d780 0x55582677aa30 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582676fb40 0x55582677a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 --2- 172.21.15.64:0/1544221817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x55582677d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 >> 172.21.15.64:0/1544221817 conn(0x555826737310 msgr2=0x555826770960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 shutdown_connections 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.017+0000 7f4042c53740 1 -- 172.21.15.64:0/1544221817 wait complete. 2024-09-19T05:13:11.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.018+0000 7f4042c53740 20 client.17157 trim_cache size 0 max 16384 2024-09-19T05:13:11.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 Processor -- start 2024-09-19T05:13:11.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 -- start start 2024-09-19T05:13:11.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x555826789720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 0x5558267792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826795200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1adfa0 con 0x55582676fb40 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1ab870 con 0x55582676e5d0 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4042c53740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1a96e0 con 0x55582677d780 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4039d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 0x5558267792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4039d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 0x5558267792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33008/0 (socket says 172.21.15.64:33008) 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f4039d9b640 1 -- 172.21.15.64:0/661572445 learned_addr learned my addr 172.21.15.64:0/661572445 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f403a59c640 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x555826789720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.037+0000 7f403ad9d640 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826795200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4039d9b640 1 -- 172.21.15.64:0/661572445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 msgr2=0x555826795200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4039d9b640 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826795200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4039d9b640 1 -- 172.21.15.64:0/661572445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 msgr2=0x555826789720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4039d9b640 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x555826789720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4039d9b640 1 -- 172.21.15.64:0/661572445 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555826771120 con 0x55582676fb40 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f403ad9d640 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826795200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4039d9b640 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 0x5558267792d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f40240cd160 tx=0x7f4024086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01fab197db7f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4023fff640 1 -- 172.21.15.64:0/661572445 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402401d040 con 0x55582676fb40 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4023fff640 1 -- 172.21.15.64:0/661572445 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402409c390 con 0x55582676fb40 2024-09-19T05:13:11.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.038+0000 7f4023fff640 1 -- 172.21.15.64:0/661572445 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40240050b0 con 0x55582676fb40 2024-09-19T05:13:11.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 -- 172.21.15.64:0/661572445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 msgr2=0x5558267792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 0x5558267792d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f40240cd160 tx=0x7f4024086040 comp rx=0 tx=0).stop 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 -- 172.21.15.64:0/661572445 shutdown_connections 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826795200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676fb40 0x5558267792d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 --2- 172.21.15.64:0/661572445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676e5d0 0x555826789720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 -- 172.21.15.64:0/661572445 >> 172.21.15.64:0/661572445 conn(0x555826737310 msgr2=0x555826778bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 -- 172.21.15.64:0/661572445 shutdown_connections 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 -- 172.21.15.64:0/661572445 wait complete. 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 -- start start 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 Processor -- start 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.039+0000 7f4042c53740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x555826786110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4042c53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676fb40 0x555826781440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4042c53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826781a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4042c53740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1a9d30 con 0x55582676e5d0 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4042c53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1ab1a0 con 0x55582676fb40 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4042c53740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1aa400 con 0x55582677d780 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4042c53740 10 client.0 _ll_register_callbacks cb 0x7f402c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x555826786110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x555826786110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33022/0 (socket says 172.21.15.64:33022) 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 -- 172.21.15.64:0/1166066206 learned_addr learned my addr 172.21.15.64:0/1166066206 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4039d9b640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676fb40 0x555826781440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403ad9d640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826781a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 -- 172.21.15.64:0/1166066206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 msgr2=0x555826781a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826781a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 -- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676fb40 msgr2=0x555826781440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676fb40 0x555826781440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 -- 172.21.15.64:0/1166066206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555826782280 con 0x55582676e5d0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f403a59c640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x555826786110 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4034009930 tx=0x7f4034003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732dab94aa0254d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4021ffb640 1 -- 172.21.15.64:0/1166066206 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034016070 con 0x55582676e5d0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.040+0000 7f4021ffb640 1 -- 172.21.15.64:0/1166066206 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f4034003cf0 con 0x55582676e5d0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402409c390 con 0x55582676e5d0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4021ffb640 1 -- 172.21.15.64:0/1166066206 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4034034050 con 0x55582676e5d0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 10 client.17163 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5558267f8270 con 0x55582676e5d0 2024-09-19T05:13:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4021ffb640 1 -- 172.21.15.64:0/1166066206 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402409c390 con 0x55582676e5d0 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4021ffb640 1 -- 172.21.15.64:0/1166066206 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f403400f040 con 0x55582676e5d0 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 10 client.17163 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 20 client.17163 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x5558267fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 4 client.17163 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558266dbd70 con 0x55582678d440 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.041+0000 7f4039d9b640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x5558267fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4039d9b640 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x5558267fa750 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57718ac0e0d43f15 server_cookie=365f591a0228ca96 in_seq=0 out_seq=0 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4021ffb640 10 client.17163 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4021ffb640 1 -- 172.21.15.64:0/1166066206 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558266dbd70 con 0x55582678d440 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4021ffb640 10 client.17163 handle_command_reply: tid=0 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 client.17163 shutdown 2024-09-19T05:13:11.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 msgr2=0x555826786110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x555826786110 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4034009930 tx=0x7f4034003970 comp rx=0 tx=0).stop 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 shutdown_connections 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582677d780 0x555826781a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55582676fb40 0x555826781440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55582678d440 0x5558267fa750 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 --2- 172.21.15.64:0/1166066206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55582676e5d0 0x555826786110 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 >> 172.21.15.64:0/1166066206 conn(0x555826737310 msgr2=0x555826778bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4021ffb640 0 client.17163 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 shutdown_connections 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.042+0000 7f4042c53740 1 -- 172.21.15.64:0/1166066206 wait complete. 2024-09-19T05:13:11.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f4042c53740 20 client.17163 trim_cache size 0 max 16384 2024-09-19T05:13:11.051 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:13:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f40280a33b0 msgr2=0x7f40280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f40280a33b0 0x7f40280a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f402c009230 tx=0x7f402c009260 comp rx=0 tx=0).stop 2024-09-19T05:13:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 msgr2=0x7f403c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 0x7f403c198850 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f402400c890 tx=0x7f402400cd50 comp rx=0 tx=0).stop 2024-09-19T05:13:11.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 shutdown_connections 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f40280a33b0 0x7f40280a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c10c180 0x7f403c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403c1036c0 0x7f403c198850 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 --2- 172.21.15.64:0/4051023288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c102da0 0x7f403c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 >> 172.21.15.64:0/4051023288 conn(0x7f403c0fc7a0 msgr2=0x7f403c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 shutdown_connections 2024-09-19T05:13:11.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:11.043+0000 7f404087a640 1 -- 172.21.15.64:0/4051023288 wait complete. 2024-09-19T05:13:11.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:11 smithi064 ceph-mon[30960]: pgmap v222: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:11.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:11 smithi103 ceph-mon[34446]: pgmap v222: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:11.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:11 smithi174 ceph-mon[34488]: pgmap v222: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:12.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:13:12.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:13:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:13.383 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:13 smithi064 ceph-mon[30960]: pgmap v223: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:13.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:13 smithi103 ceph-mon[34446]: pgmap v223: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 -- 172.21.15.64:0/1706554481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00100a80 msgr2=0x7f2f00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 --2- 172.21.15.64:0/1706554481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00100a80 0x7f2f00100ee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009930 tx=0x7f2ef402a420 comp rx=0 tx=0).stop 2024-09-19T05:13:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 -- 172.21.15.64:0/1706554481 shutdown_connections 2024-09-19T05:13:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 --2- 172.21.15.64:0/1706554481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f00101420 0x7f2f0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 --2- 172.21.15.64:0/1706554481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00100a80 0x7f2f00100ee0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 --2- 172.21.15.64:0/1706554481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00106a80 0x7f2f00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.467+0000 7f2f06a44640 1 -- 172.21.15.64:0/1706554481 >> 172.21.15.64:0/1706554481 conn(0x7f2f000fc7a0 msgr2=0x7f2f000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.468+0000 7f2f06a44640 1 -- 172.21.15.64:0/1706554481 shutdown_connections 2024-09-19T05:13:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.468+0000 7f2f06a44640 1 -- 172.21.15.64:0/1706554481 wait complete. 2024-09-19T05:13:13.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.468+0000 7f2f06a44640 1 Processor -- start 2024-09-19T05:13:13.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.468+0000 7f2f06a44640 1 -- start start 2024-09-19T05:13:13.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f06a44640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00100a80 0x7f2f0019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f06a44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f00101420 0x7f2f0019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f06a44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 0x7f2f0019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f06a44640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00110d40 con 0x7f2f00106a80 2024-09-19T05:13:13.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f06a44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00110bc0 con 0x7f2f00101420 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f06a44640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00110ec0 con 0x7f2f00100a80 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 0x7f2f0019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 0x7f2f0019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33034/0 (socket says 172.21.15.64:33034) 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 -- 172.21.15.64:0/317235239 learned_addr learned my addr 172.21.15.64:0/317235239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f00101420 0x7f2f0019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2efffff640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00100a80 0x7f2f0019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 -- 172.21.15.64:0/317235239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00100a80 msgr2=0x7f2f0019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00100a80 0x7f2f0019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 -- 172.21.15.64:0/317235239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f00101420 msgr2=0x7f2f0019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f00101420 0x7f2f0019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 -- 172.21.15.64:0/317235239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0019f510 con 0x7f2f00106a80 2024-09-19T05:13:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2efffff640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00100a80 0x7f2f0019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 0x7f2f0019ee30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800eaa0 tx=0x7f2ef800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:13.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.469+0000 7f2efd7fa640 1 -- 172.21.15.64:0/317235239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800cdb0 con 0x7f2f00106a80 2024-09-19T05:13:13.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.470+0000 7f2efd7fa640 1 -- 172.21.15.64:0/317235239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef8005020 con 0x7f2f00106a80 2024-09-19T05:13:13.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.470+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0019f800 con 0x7f2f00106a80 2024-09-19T05:13:13.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.470+0000 7f2efd7fa640 1 -- 172.21.15.64:0/317235239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80116d0 con 0x7f2f00106a80 2024-09-19T05:13:13.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.470+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f001a70e0 con 0x7f2f00106a80 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 Processor -- start 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2efd7fa640 1 -- 172.21.15.64:0/317235239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2ef80051d0 con 0x7f2f00106a80 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 -- start start 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 0x559a10643f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a1064ed40 0x559a10651130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a105a9280 con 0x559a10643b60 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a10550b10 con 0x559a1064ed40 2024-09-19T05:13:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a105a5240 con 0x559a10644480 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.472+0000 7f2eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 0x559a10643f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 0x559a10643f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33046/0 (socket says 172.21.15.64:33046) 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 -- 172.21.15.64:0/1592364458 learned_addr learned my addr 172.21.15.64:0/1592364458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2efffff640 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a1064ed40 0x559a10651130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 -- 172.21.15.64:0/1592364458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 msgr2=0x559a1064e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 -- 172.21.15.64:0/1592364458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a1064ed40 msgr2=0x559a10651130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a1064ed40 0x559a10651130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 -- 172.21.15.64:0/1592364458 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1055d1f0 con 0x559a10643b60 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2efd7fa640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2ee80a33b0 0x7f2ee80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2efffff640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2ee80a33b0 0x7f2ee80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 0x559a10643f40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4005ab0 tx=0x7f2ef4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bf682ee9a5946c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2efd7fa640 1 -- 172.21.15.64:0/317235239 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f2ef80cfc00 con 0x7f2f00106a80 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2ee67fc640 1 -- 172.21.15.64:0/1592364458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4044020 con 0x559a10643b60 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.473+0000 7f2ee67fc640 1 -- 172.21.15.64:0/1592364458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef4004750 con 0x559a10643b60 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2ee67fc640 1 -- 172.21.15.64:0/1592364458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035040 con 0x559a10643b60 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1592364458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 msgr2=0x559a10643f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 0x559a10643f40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4005ab0 tx=0x7f2ef4000bc0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2efffff640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2ee80a33b0 0x7f2ee80a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x559a10640f50 tx=0x7f2ef0008440 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1592364458 shutdown_connections 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a1064ed40 0x559a10651130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1592364458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10643b60 0x559a10643f40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1592364458 >> 172.21.15.64:0/1592364458 conn(0x559a106400f0 msgr2=0x559a106404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1592364458 shutdown_connections 2024-09-19T05:13:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.474+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1592364458 wait complete. 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 -- start start 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 Processor -- start 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a1064bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a1064c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a10651910 con 0x559a1064ed40 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a10563f10 con 0x559a10643b60 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a10651a90 con 0x559a10644480 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f04fe1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a1064c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f04fe1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a1064c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33062/0 (socket says 172.21.15.64:33062) 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f04fe1640 1 -- 172.21.15.64:0/1729535060 learned_addr learned my addr 172.21.15.64:0/1729535060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2f08e1d740 10 client.0 _ll_register_callbacks cb 0x559a1052ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.475+0000 7f2efffff640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a1064bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f04fe1640 1 -- 172.21.15.64:0/1729535060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 msgr2=0x559a1064c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f04fe1640 1 -- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 msgr2=0x559a1064bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a1064bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f04fe1640 1 -- 172.21.15.64:0/1729535060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1031a6a0 con 0x559a1064ed40 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a1064c710 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8098850 tx=0x7f2ef809bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96261f8a692b6318 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2edf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2edf7fe640 1 -- 172.21.15.64:0/1729535060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef801f030 con 0x559a1064ed40 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2edf7fe640 1 -- 172.21.15.64:0/1729535060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f2ef809b2d0 con 0x559a1064ed40 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef4004750 con 0x559a1064ed40 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2edf7fe640 1 -- 172.21.15.64:0/1729535060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ef809b4d0 con 0x559a1064ed40 2024-09-19T05:13:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f08e1d740 10 client.17175 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a1065d5f0 con 0x559a1064ed40 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.476+0000 7f2edf7fe640 1 -- 172.21.15.64:0/1729535060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef4004750 con 0x559a1064ed40 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2edf7fe640 1 -- 172.21.15.64:0/1729535060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ef8022060 con 0x559a1064ed40 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2f08e1d740 10 client.17175 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2f08e1d740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2f08e1d740 20 client.17175 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a1065eab0 0x559a10660ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2f08e1d740 4 client.17175 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a105b67f0 con 0x559a1065eab0 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2efffff640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a1065eab0 0x559a10660ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2edf7fe640 10 client.17175 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.477+0000 7f2efffff640 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a1065eab0 0x559a10660ea0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1513dec6c2be91cc server_cookie=9c40527b14cf7605 in_seq=0 out_seq=0 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2edf7fe640 1 -- 172.21.15.64:0/1729535060 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a105b67f0 con 0x559a1065eab0 2024-09-19T05:13:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2edf7fe640 10 client.17175 handle_command_reply: tid=0 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 client.17175 shutdown 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 msgr2=0x559a1064c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a1064c710 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8098850 tx=0x7f2ef809bf40 comp rx=0 tx=0).stop 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 shutdown_connections 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a1065eab0 0x559a10660ea0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a1064c710 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10644480 0x559a1064c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/1729535060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a1064bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2edf7fe640 0 client.17175 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 >> 172.21.15.64:0/1729535060 conn(0x559a106400f0 msgr2=0x559a10645e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 shutdown_connections 2024-09-19T05:13:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.478+0000 7f2f08e1d740 1 -- 172.21.15.64:0/1729535060 wait complete. 2024-09-19T05:13:13.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.479+0000 7f2f08e1d740 20 client.17175 trim_cache size 0 max 16384 2024-09-19T05:13:13.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 Processor -- start 2024-09-19T05:13:13.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 -- start start 2024-09-19T05:13:13.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a106465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 0x559a106666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a1064ed40 0x559a10666c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001b22a0 con 0x559a10644480 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001afb70 con 0x559a10643b60 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001ad9e0 con 0x559a1064ed40 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 0x559a106666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2eff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 0x559a106666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33070/0 (socket says 172.21.15.64:33070) 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.496+0000 7f2eff7fe640 1 -- 172.21.15.64:0/3753147475 learned_addr learned my addr 172.21.15.64:0/3753147475 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2efffff640 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a106465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2eff7fe640 1 -- 172.21.15.64:0/3753147475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a1064ed40 msgr2=0x559a10666c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a1064ed40 0x559a10666c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2eff7fe640 1 -- 172.21.15.64:0/3753147475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 msgr2=0x559a106465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a106465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2eff7fe640 1 -- 172.21.15.64:0/3753147475 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1064abd0 con 0x559a10644480 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2efffff640 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a106465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 0x559a106666d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009930 tx=0x7f2ef4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6982d632739d1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2ee5ffb640 1 -- 172.21.15.64:0/3753147475 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef404e070 con 0x559a10644480 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2ee5ffb640 1 -- 172.21.15.64:0/3753147475 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef4005400 con 0x559a10644480 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.497+0000 7f2ee5ffb640 1 -- 172.21.15.64:0/3753147475 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035070 con 0x559a10644480 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3753147475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 msgr2=0x559a106666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 0x559a106666d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009930 tx=0x7f2ef4009560 comp rx=0 tx=0).stop 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3753147475 shutdown_connections 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a1064ed40 0x559a10666c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a10644480 0x559a106666d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3753147475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10643b60 0x559a106465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3753147475 >> 172.21.15.64:0/3753147475 conn(0x559a106400f0 msgr2=0x559a1064dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3753147475 shutdown_connections 2024-09-19T05:13:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3753147475 wait complete. 2024-09-19T05:13:13.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 -- start start 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.498+0000 7f2f08e1d740 1 Processor -- start 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10643b60 0x559a10652e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10644480 0x559a10653390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a106c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001ae030 con 0x559a1064ed40 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001af4a0 con 0x559a10644480 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001ae700 con 0x559a10643b60 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f08e1d740 10 client.0 _ll_register_callbacks cb 0x7f2ef8098120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:13.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a106c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a106c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33086/0 (socket says 172.21.15.64:33086) 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 -- 172.21.15.64:0/3141772009 learned_addr learned my addr 172.21.15.64:0/3141772009 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10644480 0x559a10653390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2efffff640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10643b60 0x559a10652e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 -- 172.21.15.64:0/3141772009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10643b60 msgr2=0x559a10652e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10643b60 0x559a10652e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 -- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10644480 msgr2=0x559a10653390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10644480 0x559a10653390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 -- 172.21.15.64:0/3141772009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a106c8be0 con 0x559a1064ed40 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2eff7fe640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10644480 0x559a10653390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2f04fe1640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a106c86a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80982e0 tx=0x7f2ef809b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adbf440f19393a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2edf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:13.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.499+0000 7f2edf7fe640 1 -- 172.21.15.64:0/3141772009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef801f040 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2edf7fe640 1 -- 172.21.15.64:0/3141772009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f2ef8097bd0 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef4005400 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2edf7fe640 1 -- 172.21.15.64:0/3141772009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ef809bc50 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 10 client.17181 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a106c9de0 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2edf7fe640 1 -- 172.21.15.64:0/3141772009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef4005400 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2edf7fe640 1 -- 172.21.15.64:0/3141772009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ef8081020 con 0x559a1064ed40 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 10 client.17181 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 20 client.17181 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a106ca2c0 0x559a106cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 4 client.17181 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a105b0d00 con 0x559a106ca2c0 2024-09-19T05:13:13.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.500+0000 7f2efffff640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a106ca2c0 0x559a106cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.501+0000 7f2efffff640 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a106ca2c0 0x559a106cc6b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580f870d669a5354 server_cookie=b8ccb676ac8e6e26 in_seq=0 out_seq=0 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.501+0000 7f2edf7fe640 10 client.17181 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2edf7fe640 1 -- 172.21.15.64:0/3141772009 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a105b0d00 con 0x559a106ca2c0 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2edf7fe640 10 client.17181 handle_command_reply: tid=0 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 client.17181 shutdown 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 msgr2=0x559a106c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a106c86a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80982e0 tx=0x7f2ef809b8b0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 shutdown_connections 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x559a106ca2c0 0x559a106cc6b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a1064ed40 0x559a106c86a0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a10644480 0x559a10653390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2edf7fe640 0 client.17181 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:13.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 --2- 172.21.15.64:0/3141772009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a10643b60 0x559a10652e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 >> 172.21.15.64:0/3141772009 conn(0x559a106400f0 msgr2=0x559a1064bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 shutdown_connections 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 1 -- 172.21.15.64:0/3141772009 wait complete. 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.502+0000 7f2f08e1d740 20 client.17181 trim_cache size 0 max 16384 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "389220f8-bec6-416c-b471-9976000d86c3", 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:13:13.510 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2ee80a33b0 msgr2=0x7f2ee80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2ee80a33b0 0x7f2ee80a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x559a10640f50 tx=0x7f2ef0008440 comp rx=0 tx=0).stop 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 msgr2=0x7f2f0019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 0x7f2f0019ee30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800eaa0 tx=0x7f2ef800ef60 comp rx=0 tx=0).stop 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 shutdown_connections 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2ee80a33b0 0x7f2ee80a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f00106a80 0x7f2f0019ee30 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f00101420 0x7f2f0019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 --2- 172.21.15.64:0/317235239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f00100a80 0x7f2f0019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 >> 172.21.15.64:0/317235239 conn(0x7f2f000fc7a0 msgr2=0x7f2f0010c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 shutdown_connections 2024-09-19T05:13:13.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:13.503+0000 7f2f06a44640 1 -- 172.21.15.64:0/317235239 wait complete. 2024-09-19T05:13:13.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:13 smithi174 ceph-mon[34488]: pgmap v223: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:14.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:14.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:14 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:13:14.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:14 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:14.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:14 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:13:14.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:14 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:14.416 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:14.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:14 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:13:14.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:14 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:14.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:14 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:13:14.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:14 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:14 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:13:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:14 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:14 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:13:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:14 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:15.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 -- 172.21.15.64:0/789417013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102ef0 msgr2=0x7f939410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 --2- 172.21.15.64:0/789417013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102ef0 0x7f939410da40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9384009ca0 tx=0x7f938402a950 comp rx=0 tx=0).stop 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 -- 172.21.15.64:0/789417013 shutdown_connections 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 --2- 172.21.15.64:0/789417013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102ef0 0x7f939410da40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 --2- 172.21.15.64:0/789417013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9394102570 0x7f93941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 --2- 172.21.15.64:0/789417013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9394100e20 0x7f9394101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 -- 172.21.15.64:0/789417013 >> 172.21.15.64:0/789417013 conn(0x7f93940fc7d0 msgr2=0x7f93940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 -- 172.21.15.64:0/789417013 shutdown_connections 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 -- 172.21.15.64:0/789417013 wait complete. 2024-09-19T05:13:15.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 Processor -- start 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 -- start start 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.007+0000 7f939bd39640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9394100e20 0x7f93941991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 0x7f9394199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9394102ef0 0x7f93941a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9394110e30 con 0x7f9394102570 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9394110cb0 con 0x7f9394102ef0 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9394110fb0 con 0x7f9394100e20 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 0x7f9394199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 0x7f9394199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42040/0 (socket says 172.21.15.64:42040) 2024-09-19T05:13:15.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 -- 172.21.15.64:0/3701048820 learned_addr learned my addr 172.21.15.64:0/3701048820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939a2d6640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9394102ef0 0x7f93941a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f9399ad5640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9394100e20 0x7f93941991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 -- 172.21.15.64:0/3701048820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9394100e20 msgr2=0x7f93941991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9394100e20 0x7f93941991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 -- 172.21.15.64:0/3701048820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9394102ef0 msgr2=0x7f93941a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9394102ef0 0x7f93941a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 -- 172.21.15.64:0/3701048820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93941a47b0 con 0x7f9394102570 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f93992d4640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 0x7f9394199730 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f938800d950 tx=0x7f938800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f9392ffd640 1 -- 172.21.15.64:0/3701048820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388017070 con 0x7f9394102570 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f9392ffd640 1 -- 172.21.15.64:0/3701048820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9388009c30 con 0x7f9394102570 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941a4980 con 0x7f9394102570 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f9392ffd640 1 -- 172.21.15.64:0/3701048820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800f620 con 0x7f9394102570 2024-09-19T05:13:15.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.008+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a4ec0 con 0x7f9394102570 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f9392ffd640 1 -- 172.21.15.64:0/3701048820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f93880109a0 con 0x7f9394102570 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 Processor -- start 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 -- start start 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c044760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c053550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c055e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d09bfaa2e0 con 0x55d09c053a90 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d09bf51ce0 con 0x55d09c044380 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.011+0000 7f939e112740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d09bfa62a0 con 0x55d09c044ca0 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f9392ffd640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f937c0a3430 0x7f937c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c055e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c055e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42046/0 (socket says 172.21.15.64:42046) 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 -- 172.21.15.64:0/4121116226 learned_addr learned my addr 172.21.15.64:0/4121116226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f939a2d6640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f937c0a3430 0x7f937c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f9399ad5640 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c053550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f939a2d6640 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c044760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f9392ffd640 1 -- 172.21.15.64:0/3701048820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f93880cf9c0 con 0x7f9394102570 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 -- 172.21.15.64:0/4121116226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 msgr2=0x55d09c053550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c053550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 -- 172.21.15.64:0/4121116226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 msgr2=0x55d09c044760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c044760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 -- 172.21.15.64:0/4121116226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09bf36440 con 0x55d09c053a90 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f939a2d6640 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c044760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:15.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f93992d4640 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c055e80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f93880ccff0 tx=0x7f938800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b61f68a6b93927 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f939a2d6640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f937c0a3430 0x7f937c0a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9384009a60 tx=0x7f9384005d80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.012+0000 7f937bfff640 1 -- 172.21.15.64:0/4121116226 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388020070 con 0x55d09c053a90 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f937bfff640 1 -- 172.21.15.64:0/4121116226 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938809bc40 con 0x55d09c053a90 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f937bfff640 1 -- 172.21.15.64:0/4121116226 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388010460 con 0x55d09c053a90 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 -- 172.21.15.64:0/4121116226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 msgr2=0x55d09c055e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c055e80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f93880ccff0 tx=0x7f938800bf60 comp rx=0 tx=0).stop 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 -- 172.21.15.64:0/4121116226 shutdown_connections 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c055e80 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c053550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 --2- 172.21.15.64:0/4121116226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c044760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 -- 172.21.15.64:0/4121116226 >> 172.21.15.64:0/4121116226 conn(0x55d09c040a30 msgr2=0x55d09c040e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:15.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 -- 172.21.15.64:0/4121116226 shutdown_connections 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.013+0000 7f939e112740 1 -- 172.21.15.64:0/4121116226 wait complete. 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 -- start start 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 Processor -- start 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044380 0x55d09c053410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 0x55d09c0506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c053a90 0x55d09c050c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d09c056660 con 0x55d09c044ca0 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d09bf65050 con 0x55d09c053a90 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d09c0567e0 con 0x55d09c044380 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.014+0000 7f939e112740 10 client.0 _ll_register_callbacks cb 0x55d09bfa5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 0x55d09c0506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 0x55d09c0506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42058/0 (socket says 172.21.15.64:42058) 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 -- 172.21.15.64:0/1417820704 learned_addr learned my addr 172.21.15.64:0/1417820704 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f9399ad5640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044380 0x55d09c053410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f939a2d6640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c053a90 0x55d09c050c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 -- 172.21.15.64:0/1417820704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044380 msgr2=0x55d09c053410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044380 0x55d09c053410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 -- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c053a90 msgr2=0x55d09c050c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c053a90 0x55d09c050c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 -- 172.21.15.64:0/1417820704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09bd1b3c0 con 0x55d09c044ca0 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f939a2d6640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c053a90 0x55d09c050c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93992d4640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 0x55d09c0506c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9388098610 tx=0x7f9388098530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab8ae46da94786d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93797fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:15.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93797fa640 1 -- 172.21.15.64:0/1417820704 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388020050 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93797fa640 1 -- 172.21.15.64:0/1417820704 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f938809b090 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938809bc40 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.015+0000 7f93797fa640 1 -- 172.21.15.64:0/1417820704 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f938809b2d0 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 10 client.17193 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d09c062110 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f93797fa640 1 -- 172.21.15.64:0/1417820704 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938809bc40 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f93797fa640 1 -- 172.21.15.64:0/1417820704 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9388097990 con 0x55d09c044ca0 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 10 client.17193 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 20 client.17193 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c063460 0x55d09c065850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 4 client.17193 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:15.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09bfb7850 con 0x55d09c063460 2024-09-19T05:13:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.016+0000 7f9399ad5640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c063460 0x55d09c065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.017+0000 7f9399ad5640 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c063460 0x55d09c065850 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6649379108c43d8 server_cookie=cdb109530cf58e78 in_seq=0 out_seq=0 2024-09-19T05:13:15.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.017+0000 7f93797fa640 10 client.17193 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f93797fa640 1 -- 172.21.15.64:0/1417820704 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d09bfb7850 con 0x55d09c063460 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f93797fa640 10 client.17193 handle_command_reply: tid=0 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 client.17193 shutdown 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 msgr2=0x55d09c0506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 0x55d09c0506c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9388098610 tx=0x7f9388098530 comp rx=0 tx=0).stop 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 shutdown_connections 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c053a90 0x55d09c050c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c063460 0x55d09c065850 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044ca0 0x55d09c0506c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 --2- 172.21.15.64:0/1417820704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044380 0x55d09c053410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 >> 172.21.15.64:0/1417820704 conn(0x55d09c040a30 msgr2=0x55d09c046790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f93797fa640 0 client.17193 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 shutdown_connections 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 1 -- 172.21.15.64:0/1417820704 wait complete. 2024-09-19T05:13:15.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.018+0000 7f939e112740 20 client.17193 trim_cache size 0 max 16384 2024-09-19T05:13:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 Processor -- start 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 -- start start 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 0x55d09c068870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044ca0 0x55d09c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c053a90 0x55d09c04d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93941ae0b0 con 0x55d09c044380 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93941ab980 con 0x55d09c044ca0 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.035+0000 7f939e112740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93941a97f0 con 0x55d09c053a90 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 0x55d09c068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 0x55d09c068870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42066/0 (socket says 172.21.15.64:42066) 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 -- 172.21.15.64:0/225708082 learned_addr learned my addr 172.21.15.64:0/225708082 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f939a2d6640 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c053a90 0x55d09c04d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f93992d4640 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044ca0 0x55d09c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 -- 172.21.15.64:0/225708082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c053a90 msgr2=0x55d09c04d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c053a90 0x55d09c04d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 -- 172.21.15.64:0/225708082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044ca0 msgr2=0x55d09c068db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044ca0 0x55d09c068db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 -- 172.21.15.64:0/225708082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09c050fb0 con 0x55d09c044380 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f9399ad5640 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 0x55d09c068870 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55d09c056820 tx=0x7f938c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e988950265bf1be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f937b7fe640 1 -- 172.21.15.64:0/225708082 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c027610 con 0x55d09c044380 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f937b7fe640 1 -- 172.21.15.64:0/225708082 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938c027c60 con 0x55d09c044380 2024-09-19T05:13:15.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.036+0000 7f937b7fe640 1 -- 172.21.15.64:0/225708082 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c011d10 con 0x55d09c044380 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 -- 172.21.15.64:0/225708082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 msgr2=0x55d09c068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 0x55d09c068870 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55d09c056820 tx=0x7f938c005f90 comp rx=0 tx=0).stop 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 -- 172.21.15.64:0/225708082 shutdown_connections 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c053a90 0x55d09c04d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044ca0 0x55d09c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 --2- 172.21.15.64:0/225708082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c044380 0x55d09c068870 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 -- 172.21.15.64:0/225708082 >> 172.21.15.64:0/225708082 conn(0x55d09c040a30 msgr2=0x55d09c052740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 -- 172.21.15.64:0/225708082 shutdown_connections 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 -- 172.21.15.64:0/225708082 wait complete. 2024-09-19T05:13:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 -- start start 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.037+0000 7f939e112740 1 Processor -- start 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c0663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c066910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93941a9e40 con 0x55d09c053a90 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93941ab2b0 con 0x55d09c044380 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93941aa510 con 0x55d09c044ca0 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939e112740 10 client.0 _ll_register_callbacks cb 0x7f938c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c066910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42080/0 (socket says 172.21.15.64:42080) 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 -- 172.21.15.64:0/3195457917 learned_addr learned my addr 172.21.15.64:0/3195457917 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:15.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f93992d4640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c0663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f9399ad5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 -- 172.21.15.64:0/3195457917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 msgr2=0x55d09c0663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c0663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 -- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 msgr2=0x55d09c069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f939a2d6640 1 -- 172.21.15.64:0/3195457917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09c067140 con 0x55d09c053a90 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.038+0000 7f93992d4640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c0663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f939a2d6640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c066910 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9384035000 tx=0x7f93840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c26522e5f3809b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f93797fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f93797fa640 1 -- 172.21.15.64:0/3195457917 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384034040 con 0x55d09c053a90 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f93797fa640 1 -- 172.21.15.64:0/3195457917 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f93840045c0 con 0x55d09c053a90 2024-09-19T05:13:15.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c027c60 con 0x55d09c053a90 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f93797fa640 1 -- 172.21.15.64:0/3195457917 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f938402fb50 con 0x55d09c053a90 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f939e112740 10 client.17199 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d09c0cde40 con 0x55d09c053a90 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.039+0000 7f93797fa640 1 -- 172.21.15.64:0/3195457917 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938c027c60 con 0x55d09c053a90 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f93797fa640 1 -- 172.21.15.64:0/3195457917 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9384051070 con 0x55d09c053a90 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f939e112740 10 client.17199 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f939e112740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f939e112740 20 client.17199 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f939e112740 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c0ce5c0 0x55d09c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f939e112740 4 client.17199 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d09bfb1d60 con 0x55d09c0ce5c0 2024-09-19T05:13:15.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f9399ad5640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c0ce5c0 0x55d09c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f9399ad5640 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c0ce5c0 0x55d09c0d09e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e172caa168e2aeed server_cookie=d6abfb728f65b349 in_seq=0 out_seq=0 2024-09-19T05:13:15.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.040+0000 7f93797fa640 10 client.17199 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f93797fa640 1 -- 172.21.15.64:0/3195457917 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d09bfb1d60 con 0x55d09c0ce5c0 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f93797fa640 10 client.17199 handle_command_reply: tid=0 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 client.17199 shutdown 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 msgr2=0x55d09c066910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c066910 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9384035000 tx=0x7f93840095c0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 shutdown_connections 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d09c0ce5c0 0x55d09c0d09e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d09c053a90 0x55d09c066910 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f93797fa640 0 client.17199 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d09c044ca0 0x55d09c0663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 --2- 172.21.15.64:0/3195457917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d09c044380 0x55d09c069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 >> 172.21.15.64:0/3195457917 conn(0x55d09c040a30 msgr2=0x55d09c04ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 shutdown_connections 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 1 -- 172.21.15.64:0/3195457917 wait complete. 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.041+0000 7f939e112740 20 client.17199 trim_cache size 0 max 16384 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "389220f8-bec6-416c-b471-9976000d86c3": { 2024-09-19T05:13:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "389220f8-bec6-416c-b471-9976000d86c3", 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f937c0a3430 msgr2=0x7f937c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f937c0a3430 0x7f937c0a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9384009a60 tx=0x7f9384005d80 comp rx=0 tx=0).stop 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 msgr2=0x7f9394199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 0x7f9394199730 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f938800d950 tx=0x7f938800de10 comp rx=0 tx=0).stop 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 shutdown_connections 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f937c0a3430 0x7f937c0a58f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9394102ef0 0x7f93941a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394102570 0x7f9394199730 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 --2- 172.21.15.64:0/3701048820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9394100e20 0x7f93941991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 >> 172.21.15.64:0/3701048820 conn(0x7f93940fc7d0 msgr2=0x7f93940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 shutdown_connections 2024-09-19T05:13:15.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:15.042+0000 7f939bd39640 1 -- 172.21.15.64:0/3701048820 wait complete. 2024-09-19T05:13:15.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:15 smithi064 ceph-mon[30960]: pgmap v224: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:15 smithi103 ceph-mon[34446]: pgmap v224: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:15 smithi174 ceph-mon[34488]: pgmap v224: 97 pgs: 97 active+clean; 25 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T05:13:15.716 INFO:tasks.cephfs.filesystem:scrub status for tag:389220f8-bec6-416c-b471-9976000d86c3 - {'path': '/', 'tag': '389220f8-bec6-416c-b471-9976000d86c3', 'options': 'recursive,force'} 2024-09-19T05:13:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:17 smithi103 ceph-mon[34446]: pgmap v225: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-19T05:13:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:17 smithi103 ceph-mon[34446]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:13:17.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:17 smithi174 ceph-mon[34488]: pgmap v225: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-19T05:13:17.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:17 smithi174 ceph-mon[34488]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:13:17.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:17 smithi064 ceph-mon[30960]: pgmap v225: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-19T05:13:17.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:17 smithi064 ceph-mon[30960]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:13:19.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:19 smithi103 ceph-mon[34446]: pgmap v227: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 236 B/s wr, 8 op/s 2024-09-19T05:13:19.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:19 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:13:19.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:19 smithi174 ceph-mon[34488]: pgmap v227: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 236 B/s wr, 8 op/s 2024-09-19T05:13:19.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:19 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:13:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:19 smithi064 ceph-mon[30960]: pgmap v227: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 236 B/s wr, 8 op/s 2024-09-19T05:13:19.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:19 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:13:21.453 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:21 smithi064 ceph-mon[30960]: pgmap v228: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:21 smithi103 ceph-mon[34446]: pgmap v228: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:21 smithi174 ceph-mon[34488]: pgmap v228: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:23.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:23 smithi103 ceph-mon[34446]: pgmap v229: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:23.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:23 smithi174 ceph-mon[34488]: pgmap v229: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:23.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:23 smithi064 ceph-mon[30960]: pgmap v229: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:25.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:25 smithi103 ceph-mon[34446]: pgmap v230: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:25.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:25 smithi174 ceph-mon[34488]: pgmap v230: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:25 smithi064 ceph-mon[30960]: pgmap v230: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-19T05:13:27.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:27 smithi103 ceph-mon[34446]: pgmap v231: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T05:13:27.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:27 smithi174 ceph-mon[34488]: pgmap v231: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T05:13:27.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:27 smithi064 ceph-mon[30960]: pgmap v231: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T05:13:29.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:29 smithi103 ceph-mon[34446]: pgmap v232: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T05:13:29.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:29 smithi174 ceph-mon[34488]: pgmap v232: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T05:13:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:29 smithi064 ceph-mon[30960]: pgmap v232: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T05:13:31.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:31 smithi103 ceph-mon[34446]: pgmap v233: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:31.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:31 smithi174 ceph-mon[34488]: pgmap v233: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:31 smithi064 ceph-mon[30960]: pgmap v233: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:33.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:33 smithi103 ceph-mon[34446]: pgmap v234: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:33.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:33 smithi174 ceph-mon[34488]: pgmap v234: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:33 smithi064 ceph-mon[30960]: pgmap v234: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:35.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:35 smithi103 ceph-mon[34446]: pgmap v235: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:35 smithi174 ceph-mon[34488]: pgmap v235: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:35 smithi064 ceph-mon[30960]: pgmap v235: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:37.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:37 smithi103 ceph-mon[34446]: pgmap v236: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:37.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:37 smithi174 ceph-mon[34488]: pgmap v236: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:37.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:37 smithi064 ceph-mon[30960]: pgmap v236: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:39.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:39 smithi103 ceph-mon[34446]: pgmap v237: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:39 smithi174 ceph-mon[34488]: pgmap v237: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:39 smithi064 ceph-mon[30960]: pgmap v237: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:41.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:41 smithi103 ceph-mon[34446]: pgmap v238: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:41.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:41 smithi174 ceph-mon[34488]: pgmap v238: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:41.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:41 smithi064 ceph-mon[30960]: pgmap v238: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:43.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:43 smithi103 ceph-mon[34446]: pgmap v239: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:43.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:43 smithi174 ceph-mon[34488]: pgmap v239: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:43 smithi064 ceph-mon[30960]: pgmap v239: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:45.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:45 smithi103 ceph-mon[34446]: pgmap v240: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:45.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:45 smithi174 ceph-mon[34488]: pgmap v240: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:45.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:45 smithi064 ceph-mon[30960]: pgmap v240: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:45.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:45.995 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 -- 172.21.15.64:0/793908266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102570 msgr2=0x7ffb541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/793908266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102570 0x7ffb541029b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009900 tx=0x7ffb4802a480 comp rx=0 tx=0).stop 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 -- 172.21.15.64:0/793908266 shutdown_connections 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/793908266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54102ef0 0x7ffb5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/793908266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102570 0x7ffb541029b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/793908266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54100e20 0x7ffb54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 -- 172.21.15.64:0/793908266 >> 172.21.15.64:0/793908266 conn(0x7ffb540fc7d0 msgr2=0x7ffb540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 -- 172.21.15.64:0/793908266 shutdown_connections 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 -- 172.21.15.64:0/793908266 wait complete. 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.460+0000 7ffb59ae1640 1 Processor -- start 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 -- start start 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54100e20 0x7ffb540fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54102570 0x7ffb540fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 0x7ffb540fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54110e30 con 0x7ffb54102ef0 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54110cb0 con 0x7ffb54102570 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb59ae1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54110fb0 con 0x7ffb54100e20 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 0x7ffb540fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54100e20 0x7ffb540fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 0x7ffb540fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60898/0 (socket says 172.21.15.64:60898) 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb53fff640 1 -- 172.21.15.64:0/791615781 learned_addr learned my addr 172.21.15.64:0/791615781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54102570 0x7ffb540fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb537fe640 1 -- 172.21.15.64:0/791615781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54102570 msgr2=0x7ffb540fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb537fe640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54102570 0x7ffb540fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb537fe640 1 -- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 msgr2=0x7ffb540fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb537fe640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 0x7ffb540fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb537fe640 1 -- 172.21.15.64:0/791615781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb541a68e0 con 0x7ffb54100e20 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54102570 0x7ffb540fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.461+0000 7ffb53fff640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 0x7ffb540fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.462+0000 7ffb537fe640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54100e20 0x7ffb540fcc80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400e9f0 tx=0x7ffb4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.462+0000 7ffb50ff9640 1 -- 172.21.15.64:0/791615781 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400ccc0 con 0x7ffb54100e20 2024-09-19T05:13:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.462+0000 7ffb50ff9640 1 -- 172.21.15.64:0/791615781 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb440108e0 con 0x7ffb54100e20 2024-09-19T05:13:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.462+0000 7ffb50ff9640 1 -- 172.21.15.64:0/791615781 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44018880 con 0x7ffb54100e20 2024-09-19T05:13:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.462+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb541a6bd0 con 0x7ffb54100e20 2024-09-19T05:13:46.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.462+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb541a70c0 con 0x7ffb54100e20 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 Processor -- start 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 -- start start 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 0x55fbc56b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fbc560a2e0 con 0x55fbc56af680 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fbc55b1ce0 con 0x55fbc56a4dc0 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb5beba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fbc56062a0 con 0x55fbc56a44a0 2024-09-19T05:13:46.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb50ff9640 1 -- 172.21.15.64:0/791615781 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ffb44010a90 con 0x7ffb54100e20 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 0x55fbc56b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 0x55fbc56b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60910/0 (socket says 172.21.15.64:60910) 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb52ffd640 1 -- 172.21.15.64:0/99553312 learned_addr learned my addr 172.21.15.64:0/99553312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.465+0000 7ffb53fff640 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb537fe640 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb53fff640 1 -- 172.21.15.64:0/99553312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 msgr2=0x55fbc56af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb53fff640 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb53fff640 1 -- 172.21.15.64:0/99553312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 msgr2=0x55fbc56b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb53fff640 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 0x55fbc56b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb53fff640 1 -- 172.21.15.64:0/99553312 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc5596440 con 0x55fbc56a44a0 2024-09-19T05:13:46.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 0x55fbc56b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb53fff640 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56a4880 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c011760 tx=0x7ffb4c011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b99dc74567c8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb39ffb640 1 -- 172.21.15.64:0/99553312 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c004d20 con 0x55fbc56a44a0 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb39ffb640 1 -- 172.21.15.64:0/99553312 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4c005020 con 0x55fbc56a44a0 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb39ffb640 1 -- 172.21.15.64:0/99553312 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c019690 con 0x55fbc56a44a0 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.466+0000 7ffb50ff9640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb3c0a3430 0x7ffb3c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb537fe640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb3c0a3430 0x7ffb3c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb50ff9640 1 -- 172.21.15.64:0/791615781 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7ffb440d0090 con 0x7ffb54100e20 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 -- 172.21.15.64:0/99553312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 msgr2=0x55fbc56a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56a4880 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c011760 tx=0x7ffb4c011c20 comp rx=0 tx=0).stop 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb537fe640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb3c0a3430 0x7ffb3c0a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb440cd860 tx=0x7ffb44009700 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 -- 172.21.15.64:0/99553312 shutdown_connections 2024-09-19T05:13:46.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56af680 0x55fbc56b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 --2- 172.21.15.64:0/99553312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56a4880 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 -- 172.21.15.64:0/99553312 >> 172.21.15.64:0/99553312 conn(0x55fbc56a0a30 msgr2=0x55fbc56a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 -- 172.21.15.64:0/99553312 shutdown_connections 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.467+0000 7ffb5beba740 1 -- 172.21.15.64:0/99553312 wait complete. 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 -- start start 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 Processor -- start 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fbc56b2250 con 0x55fbc56a44a0 2024-09-19T05:13:46.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fbc55c5050 con 0x55fbc56a4dc0 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.468+0000 7ffb5beba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fbc56b23d0 con 0x55fbc56af680 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb5beba740 10 client.0 _ll_register_callbacks cb 0x55fbc5605160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60912/0 (socket says 172.21.15.64:60912) 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 -- 172.21.15.64:0/1885774175 learned_addr learned my addr 172.21.15.64:0/1885774175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 -- 172.21.15.64:0/1885774175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 msgr2=0x55fbc56ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 -- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 msgr2=0x55fbc56ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 -- 172.21.15.64:0/1885774175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc537b3c0 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56aeed0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009aa0 tx=0x7ffb4c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ece0767d55aca80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1885774175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c004c50 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1885774175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7ffb4c004df0 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.469+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4c005020 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1885774175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb4c036050 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 10 client.17211 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fbc56bdb20 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1885774175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4c005020 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1885774175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffb4c025040 con 0x55fbc56a44a0 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 10 client.17211 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 20 client.17211 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:46.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc56bef80 0x55fbc56c1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 4 client.17211 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:46.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc5617850 con 0x55fbc56bef80 2024-09-19T05:13:46.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.470+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc56bef80 0x55fbc56c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.471+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc56bef80 0x55fbc56c1370 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8087f992bc066b3b server_cookie=795dfaa72c6fefa8 in_seq=0 out_seq=0 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.471+0000 7ffb32ffd640 10 client.17211 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1885774175 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbc5617850 con 0x55fbc56bef80 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb32ffd640 10 client.17211 handle_command_reply: tid=0 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 client.17211 shutdown 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 msgr2=0x55fbc56aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56aeed0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009aa0 tx=0x7ffb4c005bd0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 shutdown_connections 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc56bef80 0x55fbc56c1370 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1885774175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56aeed0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 >> 172.21.15.64:0/1885774175 conn(0x55fbc56a0a30 msgr2=0x55fbc56a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb32ffd640 0 client.17211 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 shutdown_connections 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 1 -- 172.21.15.64:0/1885774175 wait complete. 2024-09-19T05:13:46.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.472+0000 7ffb5beba740 20 client.17211 trim_cache size 0 max 16384 2024-09-19T05:13:46.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.489+0000 7ffb5beba740 1 Processor -- start 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.489+0000 7ffb5beba740 1 -- start start 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 0x55fbc56c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56af680 0x55fbc56c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb5beba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541b23c0 con 0x55fbc56a4dc0 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb5beba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541afc90 con 0x55fbc56af680 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb5beba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541adb00 con 0x55fbc56a44a0 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 0x55fbc56c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 0x55fbc56c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60920/0 (socket says 172.21.15.64:60920) 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 -- 172.21.15.64:0/1618919505 learned_addr learned my addr 172.21.15.64:0/1618919505 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:46.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56af680 0x55fbc56c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 -- 172.21.15.64:0/1618919505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 msgr2=0x55fbc56bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 -- 172.21.15.64:0/1618919505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56af680 msgr2=0x55fbc56c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56af680 0x55fbc56c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 -- 172.21.15.64:0/1618919505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc56a4bf0 con 0x55fbc56a4dc0 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56af680 0x55fbc56c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.490+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 0x55fbc56c6a60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ffb48006010 tx=0x7ffb48005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926d59d4489616ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb397fa640 1 -- 172.21.15.64:0/1618919505 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4804a070 con 0x55fbc56a4dc0 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb397fa640 1 -- 172.21.15.64:0/1618919505 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb480054f0 con 0x55fbc56a4dc0 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb397fa640 1 -- 172.21.15.64:0/1618919505 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48035040 con 0x55fbc56a4dc0 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 -- 172.21.15.64:0/1618919505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 msgr2=0x55fbc56c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 0x55fbc56c6a60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ffb48006010 tx=0x7ffb48005b50 comp rx=0 tx=0).stop 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 -- 172.21.15.64:0/1618919505 shutdown_connections 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56af680 0x55fbc56c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a4dc0 0x55fbc56c6a60 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1618919505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56a44a0 0x55fbc56bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.491+0000 7ffb5beba740 1 -- 172.21.15.64:0/1618919505 >> 172.21.15.64:0/1618919505 conn(0x55fbc56a0a30 msgr2=0x55fbc56ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 -- 172.21.15.64:0/1618919505 shutdown_connections 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 -- 172.21.15.64:0/1618919505 wait complete. 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 -- start start 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 Processor -- start 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ae150 con 0x55fbc56a44a0 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541af5c0 con 0x55fbc56a4dc0 2024-09-19T05:13:46.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb5beba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ae820 con 0x55fbc56af680 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.492+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb5beba740 10 client.0 _ll_register_callbacks cb 0x7ffb4c011040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60932/0 (socket says 172.21.15.64:60932) 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 -- 172.21.15.64:0/1443405501 learned_addr learned my addr 172.21.15.64:0/1443405501 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 -- 172.21.15.64:0/1443405501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 msgr2=0x55fbc56b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 -- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 msgr2=0x55fbc56b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 -- 172.21.15.64:0/1443405501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc56b3df0 con 0x55fbc56a44a0 2024-09-19T05:13:46.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb53fff640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56b7d50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005c40 tx=0x7ffb4c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fa247628f7a1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1443405501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c041060 con 0x55fbc56a44a0 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1443405501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7ffb4c010430 con 0x55fbc56a44a0 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb480054f0 con 0x55fbc56a44a0 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1443405501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb4c00f790 con 0x55fbc56a44a0 2024-09-19T05:13:46.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb5beba740 10 client.17217 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.493+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fbc57299a0 con 0x55fbc56a44a0 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1443405501 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb480054f0 con 0x55fbc56a44a0 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1443405501 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffb4c00fd20 con 0x55fbc56a44a0 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb5beba740 10 client.17217 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb5beba740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb5beba740 20 client.17217 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:46.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc572a060 0x55fbc572c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb5beba740 4 client.17217 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbc56aca50 con 0x55fbc572a060 2024-09-19T05:13:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.494+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc572a060 0x55fbc572c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb52ffd640 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc572a060 0x55fbc572c4a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1a6988c1a56739 server_cookie=104e5659ccd55665 in_seq=0 out_seq=0 2024-09-19T05:13:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb32ffd640 10 client.17217 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb32ffd640 1 -- 172.21.15.64:0/1443405501 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fbc56aca50 con 0x55fbc572a060 2024-09-19T05:13:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb32ffd640 10 client.17217 handle_command_reply: tid=0 2024-09-19T05:13:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 client.17217 shutdown 2024-09-19T05:13:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 msgr2=0x55fbc56b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56b7d50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005c40 tx=0x7ffb4c005cf0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 shutdown_connections 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbc56af680 0x55fbc56b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fbc56a4dc0 0x55fbc56b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fbc572a060 0x55fbc572c4a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 --2- 172.21.15.64:0/1443405501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fbc56a44a0 0x55fbc56b7d50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb32ffd640 0 client.17217 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 >> 172.21.15.64:0/1443405501 conn(0x55fbc56a0a30 msgr2=0x55fbc56a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 shutdown_connections 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 1 -- 172.21.15.64:0/1443405501 wait complete. 2024-09-19T05:13:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.495+0000 7ffb5beba740 20 client.17217 trim_cache size 0 max 16384 2024-09-19T05:13:46.504 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:13:46.504 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:13:46.504 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:13:46.504 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb3c0a3430 msgr2=0x7ffb3c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb3c0a3430 0x7ffb3c0a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb440cd860 tx=0x7ffb44009700 comp rx=0 tx=0).stop 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54100e20 msgr2=0x7ffb540fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54100e20 0x7ffb540fcc80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400e9f0 tx=0x7ffb4400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 shutdown_connections 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ffb3c0a3430 0x7ffb3c0a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb54102ef0 0x7ffb540fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb54102570 0x7ffb540fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 --2- 172.21.15.64:0/791615781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb54100e20 0x7ffb540fcc80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 >> 172.21.15.64:0/791615781 conn(0x7ffb540fc7d0 msgr2=0x7ffb5406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 shutdown_connections 2024-09-19T05:13:46.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:46.496+0000 7ffb59ae1640 1 -- 172.21.15.64:0/791615781 wait complete. 2024-09-19T05:13:47.273 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:13:47.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:13:47.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:47 smithi064 ceph-mon[30960]: pgmap v241: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:47.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:47 smithi103 ceph-mon[34446]: pgmap v241: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:47.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:47 smithi174 ceph-mon[34488]: pgmap v241: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:47.543 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 -- 172.21.15.64:0/3983716970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0101de0 msgr2=0x7f96e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 --2- 172.21.15.64:0/3983716970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0101de0 0x7f96e01021c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009900 tx=0x7f96d002a410 comp rx=0 tx=0).stop 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 -- 172.21.15.64:0/3983716970 shutdown_connections 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 --2- 172.21.15.64:0/3983716970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96e0106f10 0x7f96e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 --2- 172.21.15.64:0/3983716970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 --2- 172.21.15.64:0/3983716970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0101de0 0x7f96e01021c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 -- 172.21.15.64:0/3983716970 >> 172.21.15.64:0/3983716970 conn(0x7f96e00fd8c0 msgr2=0x7f96e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 -- 172.21.15.64:0/3983716970 shutdown_connections 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.987+0000 7f96e5d30640 1 -- 172.21.15.64:0/3983716970 wait complete. 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.988+0000 7f96e5d30640 1 Processor -- start 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.988+0000 7f96e5d30640 1 -- start start 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96e5d30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96e0101de0 0x7f96e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96e5d30640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96e5d30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0106f10 0x7f96e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96e5d30640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96e0111e40 con 0x7f96e0106f10 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96e5d30640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96e0111cc0 con 0x7f96e0101de0 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96e5d30640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96e0111fc0 con 0x7f96e0102790 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:57052/0 (socket says 172.21.15.64:57052) 2024-09-19T05:13:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 -- 172.21.15.64:0/1252858396 learned_addr learned my addr 172.21.15.64:0/1252858396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96dffff640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0106f10 0x7f96e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96df7fe640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96e0101de0 0x7f96e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 -- 172.21.15.64:0/1252858396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96e0101de0 msgr2=0x7f96e00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96e0101de0 0x7f96e00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 -- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0106f10 msgr2=0x7f96e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0106f10 0x7f96e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96deffd640 1 -- 172.21.15.64:0/1252858396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e01aadb0 con 0x7f96e0102790 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.989+0000 7f96dffff640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0106f10 0x7f96e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.990+0000 7f96deffd640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e0074a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b4a0 tx=0x7f96d400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.990+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4014070 con 0x7f96e0102790 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.990+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d4010ac0 con 0x7f96e0102790 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.990+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e01aafe0 con 0x7f96e0102790 2024-09-19T05:13:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.990+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4017c30 con 0x7f96e0102790 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.990+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e01ab3e0 con 0x7f96e0102790 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.991+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e01032c0 con 0x7f96e0102790 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.993+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f96d4017070 con 0x7f96e0102790 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.993+0000 7f96dcff9640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96c80a33b0 0x7f96c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.993+0000 7f96df7fe640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96c80a33b0 0x7f96c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.993+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f96d40cf320 con 0x7f96e0102790 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.994+0000 7f96df7fe640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96c80a33b0 0x7f96c80a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96d0031440 tx=0x7f96d0005e80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:48.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:47.997+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96d409b730 con 0x7f96e0102790 2024-09-19T05:13:48.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.208+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96e0075a20 con 0x7f96e0102790 2024-09-19T05:13:48.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.209+0000 7f96dcff9640 1 -- 172.21.15.64:0/1252858396 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f96d409fa30 con 0x7f96e0102790 2024-09-19T05:13:48.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:13:48.211 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:13:48.211 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:13:48.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96c80a33b0 msgr2=0x7f96c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96c80a33b0 0x7f96c80a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96d0031440 tx=0x7f96d0005e80 comp rx=0 tx=0).stop 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 msgr2=0x7f96e0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e0074a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b4a0 tx=0x7f96d400b960 comp rx=0 tx=0).stop 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 shutdown_connections 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f96c80a33b0 0x7f96c80a5870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e0106f10 0x7f96e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102790 0x7f96e0074a30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 --2- 172.21.15.64:0/1252858396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96e0101de0 0x7f96e00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 >> 172.21.15.64:0/1252858396 conn(0x7f96e00fd8c0 msgr2=0x7f96e00fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 shutdown_connections 2024-09-19T05:13:48.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:48.212+0000 7f96e5d30640 1 -- 172.21.15.64:0/1252858396 wait complete. 2024-09-19T05:13:48.798 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:13:49.056 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:49 smithi103 ceph-mon[34446]: pgmap v242: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:49 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1252858396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:49.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:49 smithi064 ceph-mon[30960]: pgmap v242: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:49.507 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:49 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1252858396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:49 smithi174 ceph-mon[34488]: pgmap v242: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:49 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1252858396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:49.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.504+0000 7f6e163c3640 1 -- 172.21.15.64:0/3098662016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 msgr2=0x7f6e1010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.504+0000 7f6e163c3640 1 --2- 172.21.15.64:0/3098662016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1010ea90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009ca0 tx=0x7f6e0802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 -- 172.21.15.64:0/3098662016 shutdown_connections 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 --2- 172.21.15.64:0/3098662016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1010ea90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 --2- 172.21.15.64:0/3098662016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 0x7f6e101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 --2- 172.21.15.64:0/3098662016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e10101de0 0x7f6e101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 -- 172.21.15.64:0/3098662016 >> 172.21.15.64:0/3098662016 conn(0x7f6e100fd8c0 msgr2=0x7f6e100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 -- 172.21.15.64:0/3098662016 shutdown_connections 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 -- 172.21.15.64:0/3098662016 wait complete. 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 Processor -- start 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.505+0000 7f6e163c3640 1 -- start start 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e163c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e10101de0 0x7f6e1019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e163c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 0x7f6e1019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e163c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e163c3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111e20 con 0x7f6e10106f10 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e163c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111ca0 con 0x7f6e10101de0 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e163c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111fa0 con 0x7f6e10102790 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60952/0 (socket says 172.21.15.64:60952) 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 -- 172.21.15.64:0/4141097147 learned_addr learned my addr 172.21.15.64:0/4141097147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e10101de0 0x7f6e1019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e07fff640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 0x7f6e1019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 -- 172.21.15.64:0/4141097147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 msgr2=0x7f6e1019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 0x7f6e1019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 -- 172.21.15.64:0/4141097147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e10101de0 msgr2=0x7f6e1019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e10101de0 0x7f6e1019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 -- 172.21.15.64:0/4141097147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1019f4c0 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e07fff640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 0x7f6e1019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.506+0000 7f6e14960640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1019ede0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009dc0 tx=0x7f6e08032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.507+0000 7f6e0dffb640 1 -- 172.21.15.64:0/4141097147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08003cf0 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.507+0000 7f6e0dffb640 1 -- 172.21.15.64:0/4141097147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e080044f0 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.507+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1019f750 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.507+0000 7f6e0dffb640 1 -- 172.21.15.64:0/4141097147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08030bf0 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.507+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e101a7090 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.509+0000 7f6e0dffb640 1 -- 172.21.15.64:0/4141097147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6e08004040 con 0x7f6e10106f10 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.509+0000 7f6e1879c740 1 Processor -- start 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.509+0000 7f6e1879c740 1 -- start start 2024-09-19T05:13:49.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa253090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa253a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa260760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e1879c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644aa1b92f0 con 0x5644aa254b40 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e1879c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644aa160cf0 con 0x5644aa2535d0 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e1879c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644aa1b52b0 con 0x5644aa25e370 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa253090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa253090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60958/0 (socket says 172.21.15.64:60958) 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 -- 172.21.15.64:0/2863095071 learned_addr learned my addr 172.21.15.64:0/2863095071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa253a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e0dffb640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6dfc0a3430 0x7f6dfc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa260760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e14960640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6dfc0a3430 0x7f6dfc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 -- 172.21.15.64:0/2863095071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 msgr2=0x5644aa260760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa260760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 -- 172.21.15.64:0/2863095071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 msgr2=0x5644aa253a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa253a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 -- 172.21.15.64:0/2863095071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644aa1453a0 con 0x5644aa254b40 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e0dffb640 1 -- 172.21.15.64:0/4141097147 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f6e080f1280 con 0x7f6e10106f10 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.510+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa253090 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6df8013520 tx=0x7f6df800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b09ae07a5269c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e067fc640 1 -- 172.21.15.64:0/2863095071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8019030 con 0x5644aa254b40 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e14960640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6dfc0a3430 0x7f6dfc0a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e080b9a40 tx=0x7f6e0802a8c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e067fc640 1 -- 172.21.15.64:0/2863095071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df80054d0 con 0x5644aa254b40 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e067fc640 1 -- 172.21.15.64:0/2863095071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80114c0 con 0x5644aa254b40 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 -- 172.21.15.64:0/2863095071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 msgr2=0x5644aa253090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa253090 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6df8013520 tx=0x7f6df800b6b0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 -- 172.21.15.64:0/2863095071 shutdown_connections 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa260760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa253a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2863095071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa253090 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 -- 172.21.15.64:0/2863095071 >> 172.21.15.64:0/2863095071 conn(0x5644aa21c310 msgr2=0x5644aa24fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.511+0000 7f6e1879c740 1 -- 172.21.15.64:0/2863095071 shutdown_connections 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.512+0000 7f6e1879c740 1 -- 172.21.15.64:0/2863095071 wait complete. 2024-09-19T05:13:49.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.512+0000 7f6e1879c740 1 -- start start 2024-09-19T05:13:49.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.512+0000 7f6e1879c740 1 Processor -- start 2024-09-19T05:13:49.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.512+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa25dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa25afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa25b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644aa261070 con 0x5644aa254b40 2024-09-19T05:13:49.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5644aa174060 con 0x5644aa2535d0 2024-09-19T05:13:49.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644aa2611f0 con 0x5644aa25e370 2024-09-19T05:13:49.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 10 client.0 _ll_register_callbacks cb 0x5644a9ec78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:49.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa25afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa25afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60964/0 (socket says 172.21.15.64:60964) 2024-09-19T05:13:49.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 -- 172.21.15.64:0/4222655452 learned_addr learned my addr 172.21.15.64:0/4222655452 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:49.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e07fff640 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa25dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 -- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 msgr2=0x5644aa25b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa25b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 -- 172.21.15.64:0/4222655452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 msgr2=0x5644aa25dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa25dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 -- 172.21.15.64:0/4222655452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a9f2a3c0 con 0x5644aa254b40 2024-09-19T05:13:49.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa25afb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000bad0 tx=0x7f6e0000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b816c732de9d859 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6def7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6def7fe640 1 -- 172.21.15.64:0/4222655452 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00005020 con 0x5644aa254b40 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df80054d0 con 0x5644aa254b40 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.513+0000 7f6def7fe640 1 -- 172.21.15.64:0/4222655452 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f6e000051c0 con 0x5644aa254b40 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6def7fe640 1 -- 172.21.15.64:0/4222655452 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e00005400 con 0x5644aa254b40 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 10 client.17235 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:49.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644aa26cb00 con 0x5644aa254b40 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6def7fe640 1 -- 172.21.15.64:0/4222655452 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df80054d0 con 0x5644aa254b40 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6def7fe640 1 -- 172.21.15.64:0/4222655452 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6e0001b590 con 0x5644aa254b40 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 20 client.17235 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa26e010 0x5644aa270430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 4 client.17235 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:49.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.514+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644aa1c6860 con 0x5644aa26e010 2024-09-19T05:13:49.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.515+0000 7f6e07fff640 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa26e010 0x5644aa270430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.515+0000 7f6def7fe640 10 client.17235 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:49.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.515+0000 7f6e07fff640 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa26e010 0x5644aa270430 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=220a9278400264e7 server_cookie=bbc3d3717bd10119 in_seq=0 out_seq=0 2024-09-19T05:13:49.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6def7fe640 1 -- 172.21.15.64:0/4222655452 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644aa1c6860 con 0x5644aa26e010 2024-09-19T05:13:49.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6def7fe640 10 client.17235 handle_command_reply: tid=0 2024-09-19T05:13:49.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 client.17235 shutdown 2024-09-19T05:13:49.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 msgr2=0x5644aa25afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa25afb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000bad0 tx=0x7f6e0000bed0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 shutdown_connections 2024-09-19T05:13:49.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa26e010 0x5644aa270430 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa25e370 0x5644aa25b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa254b40 0x5644aa25afb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 --2- 172.21.15.64:0/4222655452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa2535d0 0x5644aa25dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6def7fe640 0 client.17235 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 >> 172.21.15.64:0/4222655452 conn(0x5644aa21c310 msgr2=0x5644aa255930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 shutdown_connections 2024-09-19T05:13:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 1 -- 172.21.15.64:0/4222655452 wait complete. 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.516+0000 7f6e1879c740 20 client.17235 trim_cache size 0 max 16384 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 Processor -- start 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 -- start start 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa2535d0 0x5644aa26a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa254b40 0x5644aa256120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 0x5644aa256660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101b23c0 con 0x5644aa25e370 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101afc90 con 0x5644aa254b40 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e1879c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101adb00 con 0x5644aa2535d0 2024-09-19T05:13:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 0x5644aa256660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 0x5644aa256660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60980/0 (socket says 172.21.15.64:60980) 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.529+0000 7f6e07fff640 1 -- 172.21.15.64:0/2536605394 learned_addr learned my addr 172.21.15.64:0/2536605394 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa2535d0 0x5644aa26a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa254b40 0x5644aa256120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 -- 172.21.15.64:0/2536605394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa2535d0 msgr2=0x5644aa26a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa2535d0 0x5644aa26a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 -- 172.21.15.64:0/2536605394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa254b40 msgr2=0x5644aa256120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa254b40 0x5644aa256120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 -- 172.21.15.64:0/2536605394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644aa250960 con 0x5644aa25e370 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa254b40 0x5644aa256120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 0x5644aa256660 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6df80057c0 tx=0x7f6df80134b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fc99089b54434c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e05ffb640 1 -- 172.21.15.64:0/2536605394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8019040 con 0x5644aa25e370 2024-09-19T05:13:49.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e05ffb640 1 -- 172.21.15.64:0/2536605394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df801a040 con 0x5644aa25e370 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.530+0000 7f6e05ffb640 1 -- 172.21.15.64:0/2536605394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8011720 con 0x5644aa25e370 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 -- 172.21.15.64:0/2536605394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 msgr2=0x5644aa256660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 0x5644aa256660 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6df80057c0 tx=0x7f6df80134b0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 -- 172.21.15.64:0/2536605394 shutdown_connections 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa25e370 0x5644aa256660 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa254b40 0x5644aa256120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2536605394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa2535d0 0x5644aa26a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 -- 172.21.15.64:0/2536605394 >> 172.21.15.64:0/2536605394 conn(0x5644aa21c310 msgr2=0x5644aa25d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 -- 172.21.15.64:0/2536605394 shutdown_connections 2024-09-19T05:13:49.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 -- 172.21.15.64:0/2536605394 wait complete. 2024-09-19T05:13:49.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.531+0000 7f6e1879c740 1 -- start start 2024-09-19T05:13:49.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 Processor -- start 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 0x5644aa266a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa254b40 0x5644aa261d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa25e370 0x5644aa2622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101ae150 con 0x5644aa2535d0 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101af5c0 con 0x5644aa25e370 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101ae820 con 0x5644aa254b40 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e1879c740 10 client.0 _ll_register_callbacks cb 0x7f6e000040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 0x5644aa266a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 0x5644aa266a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60990/0 (socket says 172.21.15.64:60990) 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 -- 172.21.15.64:0/2131609281 learned_addr learned my addr 172.21.15.64:0/2131609281 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa254b40 0x5644aa261d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa25e370 0x5644aa2622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 -- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa254b40 msgr2=0x5644aa261d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa254b40 0x5644aa261d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 -- 172.21.15.64:0/2131609281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa25e370 msgr2=0x5644aa2622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa25e370 0x5644aa2622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.532+0000 7f6e07fff640 1 -- 172.21.15.64:0/2131609281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644aa262960 con 0x5644aa2535d0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6e07fff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 0x5644aa266a50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000c00 tx=0x7f6df8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc19cb30d2a06b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6def7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6def7fe640 1 -- 172.21.15.64:0/2131609281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8040060 con 0x5644aa2535d0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6def7fe640 1 -- 172.21.15.64:0/2131609281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f6df8004cb0 con 0x5644aa2535d0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df801a040 con 0x5644aa2535d0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6def7fe640 1 -- 172.21.15.64:0/2131609281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6df8019050 con 0x5644aa2535d0 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6e1879c740 10 client.17241 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:49.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644aa2d8a90 con 0x5644aa2535d0 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6def7fe640 1 -- 172.21.15.64:0/2131609281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df801a040 con 0x5644aa2535d0 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.533+0000 7f6def7fe640 1 -- 172.21.15.64:0/2131609281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6df8011a50 con 0x5644aa2535d0 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 10 client.17241 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 10 client.17241 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 20 client.17241 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa2d8f60 0x5644aa2db350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 4 client.17241 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644aa1c0d70 con 0x5644aa2d8f60 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa2d8f60 0x5644aa2db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e0ffff640 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa2d8f60 0x5644aa2db350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22b6c18b0ddeba16 server_cookie=41effeb65eb80cd7 in_seq=0 out_seq=0 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6def7fe640 10 client.17241 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6def7fe640 1 -- 172.21.15.64:0/2131609281 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644aa1c0d70 con 0x5644aa2d8f60 2024-09-19T05:13:49.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6def7fe640 10 client.17241 handle_command_reply: tid=0 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.534+0000 7f6e1879c740 1 client.17241 shutdown 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 msgr2=0x5644aa266a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 0x5644aa266a50 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6df8000c00 tx=0x7f6df8005970 comp rx=0 tx=0).stop 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 shutdown_connections 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x5644aa2d8f60 0x5644aa2db350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5644aa25e370 0x5644aa2622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644aa254b40 0x5644aa261d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 --2- 172.21.15.64:0/2131609281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644aa2535d0 0x5644aa266a50 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6def7fe640 0 client.17241 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:13:49.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 >> 172.21.15.64:0/2131609281 conn(0x5644aa21c310 msgr2=0x5644aa25a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:49.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 shutdown_connections 2024-09-19T05:13:49.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 1 -- 172.21.15.64:0/2131609281 wait complete. 2024-09-19T05:13:49.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e1879c740 20 client.17241 trim_cache size 0 max 16384 2024-09-19T05:13:49.557 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6dfc0a3430 msgr2=0x7f6dfc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e163c3640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6dfc0a3430 0x7f6dfc0a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e080b9a40 tx=0x7f6e0802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 msgr2=0x7f6e1019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.535+0000 7f6e163c3640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1019ede0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009dc0 tx=0x7f6e08032ff0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 shutdown_connections 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6dfc0a3430 0x7f6dfc0a58f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e10106f10 0x7f6e1019ede0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e10102790 0x7f6e1019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 --2- 172.21.15.64:0/4141097147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e10101de0 0x7f6e1019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:49.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 >> 172.21.15.64:0/4141097147 conn(0x7f6e100fd8c0 msgr2=0x7f6e100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:49.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 shutdown_connections 2024-09-19T05:13:49.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:49.536+0000 7f6e163c3640 1 -- 172.21.15.64:0/4141097147 wait complete. 2024-09-19T05:13:50.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:13:50.405 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 -- 172.21.15.64:0/1739290343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488102ef0 msgr2=0x7f448810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 --2- 172.21.15.64:0/1739290343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488102ef0 0x7f448810da40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4480009ca0 tx=0x7f448002a950 comp rx=0 tx=0).stop 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 -- 172.21.15.64:0/1739290343 shutdown_connections 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 --2- 172.21.15.64:0/1739290343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488102ef0 0x7f448810da40 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 --2- 172.21.15.64:0/1739290343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4488102570 0x7f44881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 --2- 172.21.15.64:0/1739290343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488100e20 0x7f4488101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 -- 172.21.15.64:0/1739290343 >> 172.21.15.64:0/1739290343 conn(0x7f44880fc7d0 msgr2=0x7f44880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 -- 172.21.15.64:0/1739290343 shutdown_connections 2024-09-19T05:13:50.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.888+0000 7f4490778640 1 -- 172.21.15.64:0/1739290343 wait complete. 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 Processor -- start 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 -- start start 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 0x7f448819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4488102570 0x7f448819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488102ef0 0x7f448819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4488110e30 con 0x7f4488100e20 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4488110cb0 con 0x7f4488102570 2024-09-19T05:13:50.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f4490778640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4488110fb0 con 0x7f4488102ef0 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 0x7f448819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 0x7f448819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32786/0 (socket says 172.21.15.64:32786) 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 -- 172.21.15.64:0/981768607 learned_addr learned my addr 172.21.15.64:0/981768607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448dd13640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4488102570 0x7f448819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448ed15640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488102ef0 0x7f448819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 -- 172.21.15.64:0/981768607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488102ef0 msgr2=0x7f448819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488102ef0 0x7f448819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 -- 172.21.15.64:0/981768607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4488102570 msgr2=0x7f448819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4488102570 0x7f448819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448e514640 1 -- 172.21.15.64:0/981768607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448819f4e0 con 0x7f4488100e20 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.889+0000 7f448ed15640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488102ef0 0x7f448819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:50.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.890+0000 7f448e514640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 0x7f448819a530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f448400e9f0 tx=0x7f448400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.890+0000 7f447f7fe640 1 -- 172.21.15.64:0/981768607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448400cd80 con 0x7f4488100e20 2024-09-19T05:13:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.890+0000 7f447f7fe640 1 -- 172.21.15.64:0/981768607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44840054d0 con 0x7f4488100e20 2024-09-19T05:13:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.890+0000 7f447f7fe640 1 -- 172.21.15.64:0/981768607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484010490 con 0x7f4488100e20 2024-09-19T05:13:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.890+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448819f7d0 con 0x7f4488100e20 2024-09-19T05:13:50.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.890+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44881a70b0 con 0x7f4488100e20 2024-09-19T05:13:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.892+0000 7f4492b51740 1 Processor -- start 2024-09-19T05:13:50.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.892+0000 7f447f7fe640 1 -- 172.21.15.64:0/981768607 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f4484005020 con 0x7f4488100e20 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.892+0000 7f4492b51740 1 -- start start 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f4492b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a87052090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f4492b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a870525d0 0x563a87052a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f4492b51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 0x563a8705f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f4492b51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a86fb82f0 con 0x563a8705d370 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f4492b51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563a86f5fcf0 con 0x563a870525d0 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f4492b51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a86fb42b0 con 0x563a87053b40 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 0x563a8705f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 0x563a8705f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32804/0 (socket says 172.21.15.64:32804) 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 -- 172.21.15.64:0/496928931 learned_addr learned my addr 172.21.15.64:0/496928931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448e514640 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a870525d0 0x563a87052a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448ed15640 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a87052090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 -- 172.21.15.64:0/496928931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 msgr2=0x563a87052090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a87052090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 -- 172.21.15.64:0/496928931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a870525d0 msgr2=0x563a87052a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a870525d0 0x563a87052a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448dd13640 1 -- 172.21.15.64:0/496928931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a86f443a0 con 0x563a8705d370 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f448e514640 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a870525d0 0x563a87052a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.893+0000 7f447f7fe640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f44740a33b0 0x7f44740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.894+0000 7f448dd13640 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 0x563a8705f760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f447800bcb0 tx=0x7f44780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36e56a954419555 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.894+0000 7f448ed15640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f44740a33b0 0x7f44740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.894+0000 7f447f7fe640 1 -- 172.21.15.64:0/981768607 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f44840d0180 con 0x7f4488100e20 2024-09-19T05:13:50.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.894+0000 7f447cff9640 1 -- 172.21.15.64:0/496928931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478004030 con 0x563a8705d370 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.894+0000 7f447cff9640 1 -- 172.21.15.64:0/496928931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44780041d0 con 0x563a8705d370 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.894+0000 7f447cff9640 1 -- 172.21.15.64:0/496928931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44780116c0 con 0x563a8705d370 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 -- 172.21.15.64:0/496928931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 msgr2=0x563a8705f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 0x563a8705f760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f447800bcb0 tx=0x7f44780107a0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 -- 172.21.15.64:0/496928931 shutdown_connections 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a8705d370 0x563a8705f760 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a870525d0 0x563a87052a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 --2- 172.21.15.64:0/496928931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a87052090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 -- 172.21.15.64:0/496928931 >> 172.21.15.64:0/496928931 conn(0x563a8701b310 msgr2=0x563a8704ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f448ed15640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f44740a33b0 0x7f44740a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4480009a60 tx=0x7f4480005c50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 -- 172.21.15.64:0/496928931 shutdown_connections 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 -- 172.21.15.64:0/496928931 wait complete. 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 -- start start 2024-09-19T05:13:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 Processor -- start 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a8705cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.895+0000 7f4492b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a8705a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4492b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a8705a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4492b51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a87060070 con 0x563a870525d0 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4492b51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563a86f73060 con 0x563a87053b40 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4492b51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a870601f0 con 0x563a8705d370 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4492b51740 10 client.0 _ll_register_callbacks cb 0x563a86cc68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a8705cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a8705cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32806/0 (socket says 172.21.15.64:32806) 2024-09-19T05:13:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 -- 172.21.15.64:0/3087681896 learned_addr learned my addr 172.21.15.64:0/3087681896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448dd13640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a8705a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 -- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 msgr2=0x563a8705a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448ed15640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a8705a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a8705a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 -- 172.21.15.64:0/3087681896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 msgr2=0x563a8705a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a8705a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 -- 172.21.15.64:0/3087681896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a86d293c0 con 0x563a870525d0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448ed15640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a8705a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448dd13640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a8705a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f448e514640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a8705cd70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f448400c6a0 tx=0x7f448409c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55dccb18cdf5abf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4471ffb640 1 -- 172.21.15.64:0/3087681896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484005680 con 0x563a870525d0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4471ffb640 1 -- 172.21.15.64:0/3087681896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f44840a4c10 con 0x563a870525d0 2024-09-19T05:13:50.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.896+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44780041d0 con 0x563a870525d0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4471ffb640 1 -- 172.21.15.64:0/3087681896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f448409cb30 con 0x563a870525d0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 10 client.17253 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a8706bb10 con 0x563a870525d0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4471ffb640 1 -- 172.21.15.64:0/3087681896 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44780041d0 con 0x563a870525d0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4471ffb640 1 -- 172.21.15.64:0/3087681896 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44840b51f0 con 0x563a870525d0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 10 client.17253 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 20 client.17253 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a8706cfd0 0x563a8706f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 4 client.17253 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a86fc5860 con 0x563a8706cfd0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.897+0000 7f448dd13640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a8706cfd0 0x563a8706f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.898+0000 7f4471ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.898+0000 7f448dd13640 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a8706cfd0 0x563a8706f440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9ce1fce4fdefb09 server_cookie=8a47f9844d7f058a in_seq=0 out_seq=0 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.898+0000 7f4471ffb640 1 -- 172.21.15.64:0/3087681896 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a86fc5860 con 0x563a8706cfd0 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4471ffb640 10 client.17253 handle_command_reply: tid=0 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 client.17253 shutdown 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 msgr2=0x563a8705cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a8705cd70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f448400c6a0 tx=0x7f448409c2c0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 shutdown_connections 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a8705a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a8705a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a8706cfd0 0x563a8706f440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 --2- 172.21.15.64:0/3087681896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a8705cd70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4471ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 >> 172.21.15.64:0/3087681896 conn(0x563a8701b310 msgr2=0x563a87054960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:50.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 shutdown_connections 2024-09-19T05:13:50.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 1 -- 172.21.15.64:0/3087681896 wait complete. 2024-09-19T05:13:50.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.899+0000 7f4492b51740 20 client.17253 trim_cache size 0 max 16384 2024-09-19T05:13:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 Processor -- start 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 -- start start 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a870598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a8705bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8705d370 0x563a8705e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44881b22c0 con 0x563a870525d0 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44881afb90 con 0x563a8705d370 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f4492b51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44881ada00 con 0x563a87053b40 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.912+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a870598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a870598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32812/0 (socket says 172.21.15.64:32812) 2024-09-19T05:13:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 -- 172.21.15.64:0/706992736 learned_addr learned my addr 172.21.15.64:0/706992736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448dd13640 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a8705bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448ed15640 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8705d370 0x563a8705e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 -- 172.21.15.64:0/706992736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 msgr2=0x563a8705bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a8705bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 -- 172.21.15.64:0/706992736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8705d370 msgr2=0x563a8705e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8705d370 0x563a8705e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 -- 172.21.15.64:0/706992736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8705c990 con 0x563a870525d0 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f448e514640 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a870598e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f448400cfd0 tx=0x7f448409c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f196e1640b1209f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f4473fff640 1 -- 172.21.15.64:0/706992736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484005680 con 0x563a870525d0 2024-09-19T05:13:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f4473fff640 1 -- 172.21.15.64:0/706992736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44840a50a0 con 0x563a870525d0 2024-09-19T05:13:50.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.913+0000 7f4473fff640 1 -- 172.21.15.64:0/706992736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44840b51f0 con 0x563a870525d0 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 -- 172.21.15.64:0/706992736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 msgr2=0x563a870598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a870598e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f448400cfd0 tx=0x7f448409c330 comp rx=0 tx=0).stop 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 -- 172.21.15.64:0/706992736 shutdown_connections 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8705d370 0x563a8705e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87053b40 0x563a8705bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 --2- 172.21.15.64:0/706992736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a870598e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 -- 172.21.15.64:0/706992736 >> 172.21.15.64:0/706992736 conn(0x563a8701b310 msgr2=0x563a8705b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:50.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 -- 172.21.15.64:0/706992736 shutdown_connections 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 -- 172.21.15.64:0/706992736 wait complete. 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 -- start start 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.914+0000 7f4492b51740 1 Processor -- start 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a87065660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a87065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a87056290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44881ae050 con 0x563a870525d0 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44881af4c0 con 0x563a87053b40 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44881ae720 con 0x563a8705d370 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 10 client.0 _ll_register_callbacks cb 0x7f448400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a87065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a87065660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32818/0 (socket says 172.21.15.64:32818) 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 -- 172.21.15.64:0/2221112146 learned_addr learned my addr 172.21.15.64:0/2221112146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448dd13640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a87065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448ed15640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a87056290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 -- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 msgr2=0x563a87056290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a87056290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 -- 172.21.15.64:0/2221112146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 msgr2=0x563a87065ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a87065ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 -- 172.21.15.64:0/2221112146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a87056ac0 con 0x563a870525d0 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448ed15640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a87056290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f448e514640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a87065660 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f448409c3c0 tx=0x7f44840980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e009ac05b5d64208 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4471ffb640 1 -- 172.21.15.64:0/2221112146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484005680 con 0x563a870525d0 2024-09-19T05:13:50.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4471ffb640 1 -- 172.21.15.64:0/2221112146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7f448409b880 con 0x563a870525d0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.915+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44840a50a0 con 0x563a870525d0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4471ffb640 1 -- 172.21.15.64:0/2221112146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f448409c430 con 0x563a870525d0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 10 client.17259 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a87056ac0 con 0x563a870525d0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4471ffb640 1 -- 172.21.15.64:0/2221112146 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44840a50a0 con 0x563a870525d0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4471ffb640 1 -- 172.21.15.64:0/2221112146 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44840a6020 con 0x563a870525d0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 10 client.17259 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 10 client.17259 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 20 client.17259 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a870d8cc0 0x563a870db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 4 client.17259 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a8705faf0 con 0x563a870d8cc0 2024-09-19T05:13:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.916+0000 7f448dd13640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a870d8cc0 0x563a870db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4471ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f448dd13640 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a870d8cc0 0x563a870db0b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af4470612f4ce002 server_cookie=ab02769eb16c21eb in_seq=0 out_seq=0 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4471ffb640 1 -- 172.21.15.64:0/2221112146 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a8705faf0 con 0x563a870d8cc0 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4471ffb640 10 client.17259 handle_command_reply: tid=0 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 client.17259 shutdown 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 msgr2=0x563a87065660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a87065660 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f448409c3c0 tx=0x7f44840980d0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 shutdown_connections 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8705d370 0x563a87056290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87053b40 0x563a87065ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x563a870d8cc0 0x563a870db0b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 --2- 172.21.15.64:0/2221112146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a870525d0 0x563a87065660 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 >> 172.21.15.64:0/2221112146 conn(0x563a8701b310 msgr2=0x563a8705c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4471ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:13:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 shutdown_connections 2024-09-19T05:13:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 1 -- 172.21.15.64:0/2221112146 wait complete. 2024-09-19T05:13:50.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.917+0000 7f4492b51740 20 client.17259 trim_cache size 0 max 16384 2024-09-19T05:13:50.925 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f44740a33b0 msgr2=0x7f44740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f44740a33b0 0x7f44740a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4480009a60 tx=0x7f4480005c50 comp rx=0 tx=0).stop 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 msgr2=0x7f448819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 0x7f448819a530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f448400e9f0 tx=0x7f448400edf0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 shutdown_connections 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f44740a33b0 0x7f44740a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4488102ef0 0x7f448819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4488102570 0x7f448819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 --2- 172.21.15.64:0/981768607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4488100e20 0x7f448819a530 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 >> 172.21.15.64:0/981768607 conn(0x7f44880fc7d0 msgr2=0x7f448806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 shutdown_connections 2024-09-19T05:13:50.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:50.918+0000 7f4490778640 1 -- 172.21.15.64:0/981768607 wait complete. 2024-09-19T05:13:51.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:51 smithi103 ceph-mon[34446]: pgmap v243: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:51.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:51 smithi064 ceph-mon[30960]: pgmap v243: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:51.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:51 smithi174 ceph-mon[34488]: pgmap v243: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:51.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:13:52.082 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 -- 172.21.15.64:0/1660550199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 msgr2=0x7fcaac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 --2- 172.21.15.64:0/1660550199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac106e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fca94009930 tx=0x7fca9402a420 comp rx=0 tx=0).stop 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 -- 172.21.15.64:0/1660550199 shutdown_connections 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 --2- 172.21.15.64:0/1660550199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac101420 0x7fcaac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 --2- 172.21.15.64:0/1660550199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac100a80 0x7fcaac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 --2- 172.21.15.64:0/1660550199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac106e60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 -- 172.21.15.64:0/1660550199 >> 172.21.15.64:0/1660550199 conn(0x7fcaac0fc7a0 msgr2=0x7fcaac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 -- 172.21.15.64:0/1660550199 shutdown_connections 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 -- 172.21.15.64:0/1660550199 wait complete. 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 Processor -- start 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.539+0000 7fcab30e5640 1 -- start start 2024-09-19T05:13:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab30e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac100a80 0x7fcaac199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab30e5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac101420 0x7fcaac1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab30e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab30e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac110d50 con 0x7fcaac106a80 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab30e5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac110bd0 con 0x7fcaac100a80 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab30e5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac110ed0 con 0x7fcaac101420 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac1a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32826/0 (socket says 172.21.15.64:32826) 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 -- 172.21.15.64:0/3441964486 learned_addr learned my addr 172.21.15.64:0/3441964486 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac100a80 0x7fcaac199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac101420 0x7fcaac1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 -- 172.21.15.64:0/3441964486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac101420 msgr2=0x7fcaac1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac101420 0x7fcaac1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 -- 172.21.15.64:0/3441964486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac100a80 msgr2=0x7fcaac199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac100a80 0x7fcaac199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 -- 172.21.15.64:0/3441964486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac1a4820 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac101420 0x7fcaac1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac100a80 0x7fcaac199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcab1682640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac1a42e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400c970 tx=0x7fcaa400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcaa9ffb640 1 -- 172.21.15.64:0/3441964486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4007bb0 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.540+0000 7fcaa9ffb640 1 -- 172.21.15.64:0/3441964486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcaa4007d50 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.541+0000 7fcaa9ffb640 1 -- 172.21.15.64:0/3441964486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4018700 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.541+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaac1a49f0 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.541+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaac1a4eb0 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcaa9ffb640 1 -- 172.21.15.64:0/3441964486 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fcaa400fa20 con 0x7fcaac106a80 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 Processor -- start 2024-09-19T05:13:52.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 -- start start 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335efb40 0x55e9335ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 0x55e9335eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9335542f0 con 0x55e9335ee5d0 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9334fbcf0 con 0x55e9335efb40 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab54be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9335502b0 con 0x55e9335f9370 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 0x55e9335eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 0x55e9335eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32830/0 (socket says 172.21.15.64:32830) 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcab0e81640 1 -- 172.21.15.64:0/94800447 learned_addr learned my addr 172.21.15.64:0/94800447 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcaa9ffb640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca980a3430 0x7fca980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.544+0000 7fcaabfff640 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335efb40 0x55e9335ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab1682640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca980a3430 0x7fca980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcaabfff640 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab0e81640 1 -- 172.21.15.64:0/94800447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 msgr2=0x55e9335fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab0e81640 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab0e81640 1 -- 172.21.15.64:0/94800447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335efb40 msgr2=0x55e9335ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab0e81640 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335efb40 0x55e9335ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab0e81640 1 -- 172.21.15.64:0/94800447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9334e03a0 con 0x55e9335ee5d0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcaabfff640 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335efb40 0x55e9335ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcaa9ffb640 1 -- 172.21.15.64:0/3441964486 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fcaa40cfe70 con 0x7fcaac106a80 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab0e81640 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 0x55e9335eea10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fca94009930 tx=0x7fca94031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3ec27453a84d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcab1682640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca980a3430 0x7fca980a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaa407dfd0 tx=0x7fcaa4085230 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcaa2ffd640 1 -- 172.21.15.64:0/94800447 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9403f070 con 0x55e9335ee5d0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcaa2ffd640 1 -- 172.21.15.64:0/94800447 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca940316a0 con 0x55e9335ee5d0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.545+0000 7fcaa2ffd640 1 -- 172.21.15.64:0/94800447 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94035070 con 0x55e9335ee5d0 2024-09-19T05:13:52.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 -- 172.21.15.64:0/94800447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 msgr2=0x55e9335eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 0x55e9335eea10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fca94009930 tx=0x7fca94031e40 comp rx=0 tx=0).stop 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 -- 172.21.15.64:0/94800447 shutdown_connections 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335ee5d0 0x55e9335eea10 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 --2- 172.21.15.64:0/94800447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335efb40 0x55e9335ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 -- 172.21.15.64:0/94800447 >> 172.21.15.64:0/94800447 conn(0x55e9335b7310 msgr2=0x55e9335eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 -- 172.21.15.64:0/94800447 shutdown_connections 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 -- 172.21.15.64:0/94800447 wait complete. 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 -- start start 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.546+0000 7fcab54be740 1 Processor -- start 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335ee5d0 0x55e9335f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335f9370 0x55e9335f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9335fc010 con 0x55e9335efb40 2024-09-19T05:13:52.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e93350f060 con 0x55e9335f9370 2024-09-19T05:13:52.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9335fc190 con 0x55e9335ee5d0 2024-09-19T05:13:52.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab54be740 10 client.0 _ll_register_callbacks cb 0x55e9332628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:52.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32842/0 (socket says 172.21.15.64:32842) 2024-09-19T05:13:52.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 -- 172.21.15.64:0/3315849721 learned_addr learned my addr 172.21.15.64:0/3315849721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:52.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335ee5d0 0x55e9335f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 -- 172.21.15.64:0/3315849721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335ee5d0 msgr2=0x55e9335f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335f9370 0x55e9335f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335ee5d0 0x55e9335f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 -- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335f9370 msgr2=0x55e9335f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335f9370 0x55e9335f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 -- 172.21.15.64:0/3315849721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9332c53c0 con 0x55e9335efb40 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335f9370 0x55e9335f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335f5fc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fca94000c00 tx=0x7fca94031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8f63817761cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.547+0000 7fca8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fca8bfff640 1 -- 172.21.15.64:0/3315849721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9404a030 con 0x55e9335efb40 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fca8bfff640 1 -- 172.21.15.64:0/3315849721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fca94005020 con 0x55e9335efb40 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca940316a0 con 0x55e9335efb40 2024-09-19T05:13:52.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fca8bfff640 1 -- 172.21.15.64:0/3315849721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca94053050 con 0x55e9335efb40 2024-09-19T05:13:52.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 10 client.17271 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:52.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e93360bea0 con 0x55e9335efb40 2024-09-19T05:13:52.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fca8bfff640 1 -- 172.21.15.64:0/3315849721 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca940316a0 con 0x55e9335efb40 2024-09-19T05:13:52.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fca8bfff640 1 -- 172.21.15.64:0/3315849721 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca94003e00 con 0x55e9335efb40 2024-09-19T05:13:52.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 10 client.17271 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:52.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 20 client.17271 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e93360d1f0 0x55e93360f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 4 client.17271 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.548+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e933561860 con 0x55e93360d1f0 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.549+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e93360d1f0 0x55e93360f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.549+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e93360d1f0 0x55e93360f5e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c271ea6a83ab19 server_cookie=eb6fa777b0fc52ad in_seq=0 out_seq=0 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.549+0000 7fca8bfff640 10 client.17271 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fca8bfff640 1 -- 172.21.15.64:0/3315849721 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e933561860 con 0x55e93360d1f0 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fca8bfff640 10 client.17271 handle_command_reply: tid=0 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 client.17271 shutdown 2024-09-19T05:13:52.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 msgr2=0x55e9335f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335f5fc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fca94000c00 tx=0x7fca94031520 comp rx=0 tx=0).stop 2024-09-19T05:13:52.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 shutdown_connections 2024-09-19T05:13:52.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335f9370 0x55e9335f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.592 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e93360d1f0 0x55e93360f5e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335f5fc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 --2- 172.21.15.64:0/3315849721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335ee5d0 0x55e9335f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 >> 172.21.15.64:0/3315849721 conn(0x55e9335b7310 msgr2=0x55e9335f0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fca8bfff640 0 client.17271 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 shutdown_connections 2024-09-19T05:13:52.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 1 -- 172.21.15.64:0/3315849721 wait complete. 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.550+0000 7fcab54be740 20 client.17271 trim_cache size 0 max 16384 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 Processor -- start 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 -- start start 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933609770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1adf30 con 0x55e9335efb40 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1ab800 con 0x55e9335ee5d0 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab54be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1a9670 con 0x55e9335f9370 2024-09-19T05:13:52.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335ead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32850/0 (socket says 172.21.15.64:32850) 2024-09-19T05:13:52.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcab0e81640 1 -- 172.21.15.64:0/3263531049 learned_addr learned my addr 172.21.15.64:0/3263531049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:52.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.563+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933609770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab0e81640 1 -- 172.21.15.64:0/3263531049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 msgr2=0x55e9335f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab0e81640 1 -- 172.21.15.64:0/3263531049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 msgr2=0x55e933609770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933609770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab0e81640 1 -- 172.21.15.64:0/3263531049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9335e9310 con 0x55e9335efb40 2024-09-19T05:13:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcaabfff640 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933609770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab0e81640 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335ead70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fca94006010 tx=0x7fca94005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f84b27f23bed3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:52.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcaa27fc640 1 -- 172.21.15.64:0/3263531049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94000e60 con 0x55e9335efb40 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcaa27fc640 1 -- 172.21.15.64:0/3263531049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca94031590 con 0x55e9335efb40 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcaa27fc640 1 -- 172.21.15.64:0/3263531049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9404a070 con 0x55e9335efb40 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab54be740 1 -- 172.21.15.64:0/3263531049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 msgr2=0x55e9335ead70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab54be740 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335ead70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fca94006010 tx=0x7fca94005d50 comp rx=0 tx=0).stop 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab54be740 1 -- 172.21.15.64:0/3263531049 shutdown_connections 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab54be740 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab54be740 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9335ead70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.564+0000 7fcab54be740 1 --2- 172.21.15.64:0/3263531049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933609770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- 172.21.15.64:0/3263531049 >> 172.21.15.64:0/3263531049 conn(0x55e9335b7310 msgr2=0x55e9335f4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- 172.21.15.64:0/3263531049 shutdown_connections 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- 172.21.15.64:0/3263531049 wait complete. 2024-09-19T05:13:52.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- start start 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 Processor -- start 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933605b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9336060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1a9cc0 con 0x55e9335efb40 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1ab130 con 0x55e9335ee5d0 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac1aa390 con 0x55e9335f9370 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab54be740 10 client.0 _ll_register_callbacks cb 0x7fca94031d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9336060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab0e81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9336060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:32866/0 (socket says 172.21.15.64:32866) 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.565+0000 7fcab0e81640 1 -- 172.21.15.64:0/2661822813 learned_addr learned my addr 172.21.15.64:0/2661822813 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcaabfff640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933605b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab0e81640 1 -- 172.21.15.64:0/2661822813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 msgr2=0x55e9335f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab0e81640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab0e81640 1 -- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 msgr2=0x55e933605b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab0e81640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933605b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab0e81640 1 -- 172.21.15.64:0/2661822813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9335eb960 con 0x55e9335efb40 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcaabfff640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933605b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab0e81640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9336060b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fca94006010 tx=0x7fca9402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfae40fe33ace10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fca8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fca8bfff640 1 -- 172.21.15.64:0/2661822813 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9404a070 con 0x55e9335efb40 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fca8bfff640 1 -- 172.21.15.64:0/2661822813 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fca9403b680 con 0x55e9335efb40 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca94031590 con 0x55e9335efb40 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fca8bfff640 1 -- 172.21.15.64:0/2661822813 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca9403f070 con 0x55e9335efb40 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab54be740 10 client.17277 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:52.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e933677480 con 0x55e9335efb40 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.566+0000 7fca8bfff640 1 -- 172.21.15.64:0/2661822813 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca94031590 con 0x55e9335efb40 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fca8bfff640 1 -- 172.21.15.64:0/2661822813 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fca9404f940 con 0x55e9335efb40 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcab54be740 10 client.17277 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcab54be740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcab54be740 20 client.17277 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcab54be740 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e933677a30 0x55e933679e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcab54be740 4 client.17277 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e93355bd70 con 0x55e933677a30 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcaabfff640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e933677a30 0x55e933679e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fcaabfff640 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e933677a30 0x55e933679e20 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f588f6360f989b9a server_cookie=a26d8a1d8159b8d4 in_seq=0 out_seq=0 2024-09-19T05:13:52.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.567+0000 7fca8bfff640 10 client.17277 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fca8bfff640 1 -- 172.21.15.64:0/2661822813 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e93355bd70 con 0x55e933677a30 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fca8bfff640 10 client.17277 handle_command_reply: tid=0 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 client.17277 shutdown 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 msgr2=0x55e9336060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9336060b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fca94006010 tx=0x7fca9402a420 comp rx=0 tx=0).stop 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 shutdown_connections 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9335f9370 0x55e9335f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55e933677a30 0x55e933679e20 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9335efb40 0x55e9336060b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 --2- 172.21.15.64:0/2661822813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9335ee5d0 0x55e933605b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fca8bfff640 0 client.17277 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 >> 172.21.15.64:0/2661822813 conn(0x55e9335b7310 msgr2=0x55e9335f0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 shutdown_connections 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 1 -- 172.21.15.64:0/2661822813 wait complete. 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.568+0000 7fcab54be740 20 client.17277 trim_cache size 0 max 16384 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca980a3430 msgr2=0x7fca980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca980a3430 0x7fca980a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaa407dfd0 tx=0x7fcaa4085230 comp rx=0 tx=0).stop 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 msgr2=0x7fcaac1a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac1a42e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400c970 tx=0x7fcaa400ce30 comp rx=0 tx=0).stop 2024-09-19T05:13:52.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 shutdown_connections 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fca980a3430 0x7fca980a58f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcaac106a80 0x7fcaac1a42e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaac101420 0x7fcaac1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 --2- 172.21.15.64:0/3441964486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaac100a80 0x7fcaac199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 >> 172.21.15.64:0/3441964486 conn(0x7fcaac0fc7a0 msgr2=0x7fcaac0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 shutdown_connections 2024-09-19T05:13:52.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:52.569+0000 7fcab30e5640 1 -- 172.21.15.64:0/3441964486 wait complete. 2024-09-19T05:13:53.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:53 smithi103 ceph-mon[34446]: pgmap v244: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:53.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:53 smithi174 ceph-mon[34488]: pgmap v244: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:53.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:53 smithi064 ceph-mon[30960]: pgmap v244: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:54.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:13:54.319 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:13:54.589 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:55.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 -- 172.21.15.64:0/3195167767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288106050 msgr2=0x7fe28810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 --2- 172.21.15.64:0/3195167767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288106050 0x7fe28810dbd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe280009900 tx=0x7fe28002a410 comp rx=0 tx=0).stop 2024-09-19T05:13:55.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 -- 172.21.15.64:0/3195167767 shutdown_connections 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 --2- 172.21.15.64:0/3195167767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288106050 0x7fe28810dbd0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 --2- 172.21.15.64:0/3195167767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 0x7fe288105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 --2- 172.21.15.64:0/3195167767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288101440 0x7fe288101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 -- 172.21.15.64:0/3195167767 >> 172.21.15.64:0/3195167767 conn(0x7fe288077210 msgr2=0x7fe2880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 -- 172.21.15.64:0/3195167767 shutdown_connections 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.048+0000 7fe28c921640 1 -- 172.21.15.64:0/3195167767 wait complete. 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 Processor -- start 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 -- start start 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 0x7fe288199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 0x7fe288199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288106050 0x7fe2881a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe288110f80 con 0x7fe288101440 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe288110e00 con 0x7fe288101df0 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28c921640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe288111100 con 0x7fe288106050 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 0x7fe288199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 0x7fe288199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43194/0 (socket says 172.21.15.64:43194) 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.049+0000 7fe28659c640 1 -- 172.21.15.64:0/2161178753 learned_addr learned my addr 172.21.15.64:0/2161178753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe285d9b640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 0x7fe288199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe286d9d640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288106050 0x7fe2881a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28659c640 1 -- 172.21.15.64:0/2161178753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288106050 msgr2=0x7fe2881a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28659c640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288106050 0x7fe2881a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28659c640 1 -- 172.21.15.64:0/2161178753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 msgr2=0x7fe288199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28659c640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 0x7fe288199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28659c640 1 -- 172.21.15.64:0/2161178753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2881a4a10 con 0x7fe288101440 2024-09-19T05:13:55.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe286d9d640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288106050 0x7fe2881a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe285d9b640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 0x7fe288199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28659c640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 0x7fe288199450 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe27800ba50 tx=0x7fe27800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe27f7fe640 1 -- 172.21.15.64:0/2161178753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278019070 con 0x7fe288101440 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2881a4be0 con 0x7fe288101440 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe27f7fe640 1 -- 172.21.15.64:0/2161178753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe27800cbe0 con 0x7fe288101440 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe27f7fe640 1 -- 172.21.15.64:0/2161178753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780075d0 con 0x7fe288101440 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.050+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2881a50f0 con 0x7fe288101440 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe27f7fe640 1 -- 172.21.15.64:0/2161178753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe27801c070 con 0x7fe288101440 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 Processor -- start 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 -- start start 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a8b60 0x5571ec4a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 0x5571ec4b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4b3d40 0x5571ec4b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5571ec40e280 con 0x5571ec4a9480 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5571ec3b5b10 con 0x5571ec4a8b60 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5571ec40a240 con 0x5571ec4b3d40 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 0x5571ec4b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.053+0000 7fe28659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 0x5571ec4b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43198/0 (socket says 172.21.15.64:43198) 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 -- 172.21.15.64:0/34113422 learned_addr learned my addr 172.21.15.64:0/34113422 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe286d9d640 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a8b60 0x5571ec4a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe27f7fe640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2740a33b0 0x7fe2740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe285d9b640 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4b3d40 0x5571ec4b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe286d9d640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2740a33b0 0x7fe2740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe27f7fe640 1 -- 172.21.15.64:0/2161178753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fe2780cf2f0 con 0x7fe288101440 2024-09-19T05:13:55.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 -- 172.21.15.64:0/34113422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4b3d40 msgr2=0x5571ec4b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4b3d40 0x5571ec4b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 -- 172.21.15.64:0/34113422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a8b60 msgr2=0x5571ec4a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a8b60 0x5571ec4a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 -- 172.21.15.64:0/34113422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec3c21f0 con 0x5571ec4a9480 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe285d9b640 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4b3d40 0x5571ec4b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe28659c640 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 0x5571ec4b3800 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe27809b3b0 tx=0x7fe278004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbc3e720c6228f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe27cff9640 1 -- 172.21.15.64:0/34113422 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2780d2a50 con 0x5571ec4a9480 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe286d9d640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2740a33b0 0x7fe2740a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe280031440 tx=0x7fe28002a410 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe27cff9640 1 -- 172.21.15.64:0/34113422 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe27809b6b0 con 0x5571ec4a9480 2024-09-19T05:13:55.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.054+0000 7fe27cff9640 1 -- 172.21.15.64:0/34113422 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278097d20 con 0x5571ec4a9480 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 -- 172.21.15.64:0/34113422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 msgr2=0x5571ec4b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 0x5571ec4b3800 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe27809b3b0 tx=0x7fe278004b50 comp rx=0 tx=0).stop 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 -- 172.21.15.64:0/34113422 shutdown_connections 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4b3d40 0x5571ec4b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4a9480 0x5571ec4b3800 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/34113422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a8b60 0x5571ec4a8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 -- 172.21.15.64:0/34113422 >> 172.21.15.64:0/34113422 conn(0x5571ec4a50f0 msgr2=0x5571ec4a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 -- 172.21.15.64:0/34113422 shutdown_connections 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.055+0000 7fe28ecfa740 1 -- 172.21.15.64:0/34113422 wait complete. 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 -- start start 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 Processor -- start 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5571ec4bad20 con 0x5571ec4b3d40 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5571ec3c8f10 con 0x5571ec4a9480 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5571ec4baea0 con 0x5571ec4a8b60 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28ecfa740 10 client.0 _ll_register_callbacks cb 0x5571ec391a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:55.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe286d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe285d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe286d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43200/0 (socket says 172.21.15.64:43200) 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe286d9d640 1 -- 172.21.15.64:0/3147952933 learned_addr learned my addr 172.21.15.64:0/3147952933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.056+0000 7fe28659c640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe285d9b640 1 -- 172.21.15.64:0/3147952933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 msgr2=0x5571ec4b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe285d9b640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe285d9b640 1 -- 172.21.15.64:0/3147952933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 msgr2=0x5571ec4b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe285d9b640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe285d9b640 1 -- 172.21.15.64:0/3147952933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ec17f6a0 con 0x5571ec4a9480 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe286d9d640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe285d9b640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4b1120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe288100c20 tx=0x7fe270010b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=456a6a0c05081945 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:55.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe26dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe26dffb640 1 -- 172.21.15.64:0/3147952933 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270012920 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27809b6b0 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe26dffb640 1 -- 172.21.15.64:0/3147952933 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fe270012ac0 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe26dffb640 1 -- 172.21.15.64:0/3147952933 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe270012cc0 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe28ecfa740 10 client.26738 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.057+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571ec4c6910 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe26dffb640 1 -- 172.21.15.64:0/3147952933 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe27809b6b0 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe26dffb640 1 -- 172.21.15.64:0/3147952933 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe27000f040 con 0x5571ec4a9480 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28ecfa740 10 client.26738 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28ecfa740 10 client.26738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28ecfa740 20 client.26738 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:55.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec4c7dc0 0x5571ec4ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28ecfa740 4 client.26738 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ec41b7f0 con 0x5571ec4c7dc0 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.058+0000 7fe28659c640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec4c7dc0 0x5571ec4ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28659c640 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec4c7dc0 0x5571ec4ca200 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66418d5e1a4ba84e server_cookie=905653ece5e8e91b in_seq=0 out_seq=0 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe26dffb640 10 client.26738 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe26dffb640 1 -- 172.21.15.64:0/3147952933 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571ec41b7f0 con 0x5571ec4c7dc0 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe26dffb640 10 client.26738 handle_command_reply: tid=0 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 client.26738 shutdown 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 msgr2=0x5571ec4b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4b1120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe288100c20 tx=0x7fe270010b00 comp rx=0 tx=0).stop 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 shutdown_connections 2024-09-19T05:13:55.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec4c7dc0 0x5571ec4ca200 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4b1120 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe26dffb640 0 client.26738 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/3147952933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.059+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 >> 172.21.15.64:0/3147952933 conn(0x5571ec4a50f0 msgr2=0x5571ec4aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.060+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 shutdown_connections 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.060+0000 7fe28ecfa740 1 -- 172.21.15.64:0/3147952933 wait complete. 2024-09-19T05:13:55.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.060+0000 7fe28ecfa740 20 client.26738 trim_cache size 0 max 16384 2024-09-19T05:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 Processor -- start 2024-09-19T05:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 -- start start 2024-09-19T05:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881adff0 con 0x5571ec4b3d40 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881ab8c0 con 0x5571ec4a9480 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881a9730 con 0x5571ec4a8b60 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe286d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe286d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43216/0 (socket says 172.21.15.64:43216) 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.077+0000 7fe286d9d640 1 -- 172.21.15.64:0/1943531964 learned_addr learned my addr 172.21.15.64:0/1943531964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe28659c640 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe285d9b640 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe286d9d640 1 -- 172.21.15.64:0/1943531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 msgr2=0x5571ec4afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe286d9d640 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe286d9d640 1 -- 172.21.15.64:0/1943531964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 msgr2=0x5571ec4cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe286d9d640 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe286d9d640 1 -- 172.21.15.64:0/1943531964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ec4a4f20 con 0x5571ec4b3d40 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe285d9b640 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe286d9d640 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4d0010 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe280031610 tx=0x7fe280005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d51c3edb0e6810f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe26ffff640 1 -- 172.21.15.64:0/1943531964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280035070 con 0x5571ec4b3d40 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe26ffff640 1 -- 172.21.15.64:0/1943531964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe28004f070 con 0x5571ec4b3d40 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.078+0000 7fe26ffff640 1 -- 172.21.15.64:0/1943531964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280004c40 con 0x5571ec4b3d40 2024-09-19T05:13:55.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 -- 172.21.15.64:0/1943531964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 msgr2=0x5571ec4d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4d0010 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe280031610 tx=0x7fe280005b70 comp rx=0 tx=0).stop 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 -- 172.21.15.64:0/1943531964 shutdown_connections 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4d0010 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/1943531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 -- 172.21.15.64:0/1943531964 >> 172.21.15.64:0/1943531964 conn(0x5571ec4a50f0 msgr2=0x5571ec4af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 -- 172.21.15.64:0/1943531964 shutdown_connections 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 -- 172.21.15.64:0/1943531964 wait complete. 2024-09-19T05:13:55.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 -- start start 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.079+0000 7fe28ecfa740 1 Processor -- start 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881a9d80 con 0x5571ec4b3d40 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881ab1f0 con 0x5571ec4a9480 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881aa450 con 0x5571ec4a8b60 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28ecfa740 10 client.0 _ll_register_callbacks cb 0x7fe27000b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4bcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43230/0 (socket says 172.21.15.64:43230) 2024-09-19T05:13:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe28659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 -- 172.21.15.64:0/592901364 learned_addr learned my addr 172.21.15.64:0/592901364 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe285d9b640 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 -- 172.21.15.64:0/592901364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 msgr2=0x5571ec4bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 -- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 msgr2=0x5571ec4bc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4bc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 -- 172.21.15.64:0/592901364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ec531d40 con 0x5571ec4b3d40 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe286d9d640 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4bcb90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe2800092a0 tx=0x7fe2800045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be9d17b37cb6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.080+0000 7fe26dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe26dffb640 1 -- 172.21.15.64:0/592901364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280057070 con 0x5571ec4b3d40 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe26dffb640 1 -- 172.21.15.64:0/592901364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fe2800039c0 con 0x5571ec4b3d40 2024-09-19T05:13:55.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28004f070 con 0x5571ec4b3d40 2024-09-19T05:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe26dffb640 1 -- 172.21.15.64:0/592901364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe280063050 con 0x5571ec4b3d40 2024-09-19T05:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 10 client.17295 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571ec533080 con 0x5571ec4b3d40 2024-09-19T05:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe26dffb640 1 -- 172.21.15.64:0/592901364 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe28004f070 con 0x5571ec4b3d40 2024-09-19T05:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe26dffb640 1 -- 172.21.15.64:0/592901364 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe28003a040 con 0x5571ec4b3d40 2024-09-19T05:13:55.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 10 client.17295 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 20 client.17295 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec533740 0x5571ec535b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 4 client.17295 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:13:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.081+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571ec415d00 con 0x5571ec533740 2024-09-19T05:13:55.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.082+0000 7fe28659c640 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec533740 0x5571ec535b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.082+0000 7fe28659c640 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec533740 0x5571ec535b60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2c88eb6cf3de0b server_cookie=41c5f831513c1b3a in_seq=0 out_seq=0 2024-09-19T05:13:55.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.082+0000 7fe26dffb640 10 client.17295 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe26dffb640 1 -- 172.21.15.64:0/592901364 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571ec415d00 con 0x5571ec533740 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe26dffb640 10 client.17295 handle_command_reply: tid=0 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 client.17295 shutdown 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 msgr2=0x5571ec4bcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4bcb90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe2800092a0 tx=0x7fe2800045f0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 shutdown_connections 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571ec533740 0x5571ec535b60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571ec4b3d40 0x5571ec4bcb90 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571ec4a9480 0x5571ec4bc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 --2- 172.21.15.64:0/592901364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571ec4a8b60 0x5571ec4bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe26dffb640 0 client.17295 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 >> 172.21.15.64:0/592901364 conn(0x5571ec4a50f0 msgr2=0x5571ec4aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 shutdown_connections 2024-09-19T05:13:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 1 -- 172.21.15.64:0/592901364 wait complete. 2024-09-19T05:13:55.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.083+0000 7fe28ecfa740 20 client.17295 trim_cache size 0 max 16384 2024-09-19T05:13:55.097 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:13:55.097 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:13:55.097 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "0bd84f4a-10b2-4046-be21-2b3e87923711", 2024-09-19T05:13:55.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:13:55.097 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:13:55.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2740a33b0 msgr2=0x7fe2740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2740a33b0 0x7fe2740a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe280031440 tx=0x7fe28002a410 comp rx=0 tx=0).stop 2024-09-19T05:13:55.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 msgr2=0x7fe288199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:55.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 0x7fe288199450 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe27800ba50 tx=0x7fe27800bf10 comp rx=0 tx=0).stop 2024-09-19T05:13:55.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 shutdown_connections 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe2740a33b0 0x7fe2740a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288106050 0x7fe2881a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe288101df0 0x7fe288199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 --2- 172.21.15.64:0/2161178753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe288101440 0x7fe288199450 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 >> 172.21.15.64:0/2161178753 conn(0x7fe288077210 msgr2=0x7fe2881042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 shutdown_connections 2024-09-19T05:13:55.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:55.084+0000 7fe28c921640 1 -- 172.21.15.64:0/2161178753 wait complete. 2024-09-19T05:13:55.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:55 smithi174 ceph-mon[34488]: pgmap v245: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:55.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:55 smithi064 ceph-mon[30960]: pgmap v245: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:55 smithi103 ceph-mon[34446]: pgmap v245: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:55.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:56.055 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:56.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:56 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:13:56.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:56 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:56.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:56 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:13:56.430 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:56 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 -- 172.21.15.64:0/2668723417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c1049c0 msgr2=0x7fc93c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 --2- 172.21.15.64:0/2668723417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c1049c0 0x7fc93c108a10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402a410 comp rx=0 tx=0).stop 2024-09-19T05:13:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 -- 172.21.15.64:0/2668723417 shutdown_connections 2024-09-19T05:13:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 --2- 172.21.15.64:0/2668723417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc93c109140 0x7fc93c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 --2- 172.21.15.64:0/2668723417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c1049c0 0x7fc93c108a10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 --2- 172.21.15.64:0/2668723417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c104010 0x7fc93c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 -- 172.21.15.64:0/2668723417 >> 172.21.15.64:0/2668723417 conn(0x7fc93c0ffa50 msgr2=0x7fc93c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 -- 172.21.15.64:0/2668723417 shutdown_connections 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.512+0000 7fc9419bf640 1 -- 172.21.15.64:0/2668723417 wait complete. 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 Processor -- start 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 -- start start 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc93c104010 0x7fc93c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c1049c0 0x7fc93c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 0x7fc93c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c114010 con 0x7fc93c109140 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c113e90 con 0x7fc93c104010 2024-09-19T05:13:56.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc9419bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c114190 con 0x7fc93c1049c0 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc93b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 0x7fc93c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc93b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 0x7fc93c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43246/0 (socket says 172.21.15.64:43246) 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc93b7fe640 1 -- 172.21.15.64:0/1930690091 learned_addr learned my addr 172.21.15.64:0/1930690091 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc93affd640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc93c104010 0x7fc93c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.513+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c1049c0 0x7fc93c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93b7fe640 1 -- 172.21.15.64:0/1930690091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c1049c0 msgr2=0x7fc93c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93b7fe640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c1049c0 0x7fc93c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93b7fe640 1 -- 172.21.15.64:0/1930690091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc93c104010 msgr2=0x7fc93c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93b7fe640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc93c104010 0x7fc93c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93b7fe640 1 -- 172.21.15.64:0/1930690091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c19f580 con 0x7fc93c109140 2024-09-19T05:13:56.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c1049c0 0x7fc93c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc93b7fe640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 0x7fc93c19eea0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c970 tx=0x7fc93400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc9409bd640 1 -- 172.21.15.64:0/1930690091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934007bb0 con 0x7fc93c109140 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc9409bd640 1 -- 172.21.15.64:0/1930690091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc934007d50 con 0x7fc93c109140 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c19f870 con 0x7fc93c109140 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc9409bd640 1 -- 172.21.15.64:0/1930690091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934018700 con 0x7fc93c109140 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.514+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c1a7150 con 0x7fc93c109140 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.516+0000 7fc9409bd640 1 -- 172.21.15.64:0/1930690091 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc93400fa20 con 0x7fc93c109140 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 Processor -- start 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 -- start start 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c52880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560e28bb82e0 con 0x560e28c524a0 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e28b5fce0 con 0x560e28c5d680 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc943d98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e28bb42a0 con 0x560e28c52dc0 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c52880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c52880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43248/0 (socket says 172.21.15.64:43248) 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 -- 172.21.15.64:0/4094426502 learned_addr learned my addr 172.21.15.64:0/4094426502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93affd640 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc9409bd640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9280a33b0 0x7fc9280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 -- 172.21.15.64:0/4094426502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 msgr2=0x560e28c5d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 -- 172.21.15.64:0/4094426502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 msgr2=0x560e28c5fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c5fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93a7fc640 1 -- 172.21.15.64:0/4094426502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e28b44440 con 0x560e28c524a0 2024-09-19T05:13:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.517+0000 7fc93b7fe640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9280a33b0 0x7fc9280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.518+0000 7fc9409bd640 1 -- 172.21.15.64:0/1930690091 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fc9340cfdc0 con 0x7fc93c109140 2024-09-19T05:13:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.518+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c52880 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc924005ec0 tx=0x7fc924009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fd67c1e75a576a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.518+0000 7fc931ffb640 1 -- 172.21.15.64:0/4094426502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924043020 con 0x560e28c524a0 2024-09-19T05:13:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.518+0000 7fc931ffb640 1 -- 172.21.15.64:0/4094426502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc924031800 con 0x560e28c524a0 2024-09-19T05:13:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.518+0000 7fc931ffb640 1 -- 172.21.15.64:0/4094426502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035050 con 0x560e28c524a0 2024-09-19T05:13:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- 172.21.15.64:0/4094426502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 msgr2=0x560e28c52880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc93b7fe640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9280a33b0 0x7fc9280a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9340cdae0 tx=0x7fc93400fd10 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c52880 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc924005ec0 tx=0x7fc924009560 comp rx=0 tx=0).stop 2024-09-19T05:13:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- 172.21.15.64:0/4094426502 shutdown_connections 2024-09-19T05:13:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- 172.21.15.64:0/4094426502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c52880 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- 172.21.15.64:0/4094426502 >> 172.21.15.64:0/4094426502 conn(0x560e28c4ea30 msgr2=0x560e28c4ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- 172.21.15.64:0/4094426502 shutdown_connections 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- 172.21.15.64:0/4094426502 wait complete. 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- start start 2024-09-19T05:13:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 Processor -- start 2024-09-19T05:13:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28c5cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c5a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560e28c60250 con 0x560e28c5d680 2024-09-19T05:13:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e28b73050 con 0x560e28c524a0 2024-09-19T05:13:56.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.519+0000 7fc943d98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e28c603d0 con 0x560e28c52dc0 2024-09-19T05:13:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc943d98740 10 client.0 _ll_register_callbacks cb 0x560e28bb3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c5a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c5a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43260/0 (socket says 172.21.15.64:43260) 2024-09-19T05:13:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 -- 172.21.15.64:0/2137978153 learned_addr learned my addr 172.21.15.64:0/2137978153 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28c5cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93affd640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 -- 172.21.15.64:0/2137978153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 msgr2=0x560e28c5a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 -- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 msgr2=0x560e28c5cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28c5cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 -- 172.21.15.64:0/2137978153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e289293c0 con 0x560e28c5d680 2024-09-19T05:13:56.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c5a7e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc924000c00 tx=0x7fc9240319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57633df90b67b536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:56.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc91affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:56.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc91affd640 1 -- 172.21.15.64:0/2137978153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035070 con 0x560e28c5d680 2024-09-19T05:13:56.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc924031800 con 0x560e28c5d680 2024-09-19T05:13:56.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.520+0000 7fc91affd640 1 -- 172.21.15.64:0/2137978153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fc924004b00 con 0x560e28c5d680 2024-09-19T05:13:56.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc91affd640 1 -- 172.21.15.64:0/2137978153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc924043040 con 0x560e28c5d680 2024-09-19T05:13:56.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 10 client.17307 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e28c6bc20 con 0x560e28c5d680 2024-09-19T05:13:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc91affd640 1 -- 172.21.15.64:0/2137978153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc924031800 con 0x560e28c5d680 2024-09-19T05:13:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc91affd640 1 -- 172.21.15.64:0/2137978153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc924003780 con 0x560e28c5d680 2024-09-19T05:13:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 10 client.17307 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:56.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 20 client.17307 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28c6cf70 0x560e28c6f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 4 client.17307 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e28bc5850 con 0x560e28c6cf70 2024-09-19T05:13:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.521+0000 7fc93affd640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28c6cf70 0x560e28c6f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.522+0000 7fc93affd640 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28c6cf70 0x560e28c6f360 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e6420b3c5d612e server_cookie=dcccd7c5a5a42436 in_seq=0 out_seq=0 2024-09-19T05:13:56.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.522+0000 7fc91affd640 10 client.17307 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:56.538 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:56 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:13:56.538 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:56 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:56.539 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:56 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:13:56.539 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:56 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.522+0000 7fc91affd640 1 -- 172.21.15.64:0/2137978153 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e28bc5850 con 0x560e28c6cf70 2024-09-19T05:13:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.522+0000 7fc91affd640 10 client.17307 handle_command_reply: tid=0 2024-09-19T05:13:56.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.522+0000 7fc943d98740 1 client.17307 shutdown 2024-09-19T05:13:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 msgr2=0x560e28c5a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c5a7e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc924000c00 tx=0x7fc9240319b0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 shutdown_connections 2024-09-19T05:13:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28c6cf70 0x560e28c6f360 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c5a7e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c5a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 --2- 172.21.15.64:0/2137978153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28c5cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc91affd640 0 client.17307 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:56.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 >> 172.21.15.64:0/2137978153 conn(0x560e28c4ea30 msgr2=0x560e28c54760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:56.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 shutdown_connections 2024-09-19T05:13:56.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 1 -- 172.21.15.64:0/2137978153 wait complete. 2024-09-19T05:13:56.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.523+0000 7fc943d98740 20 client.17307 trim_cache size 0 max 16384 2024-09-19T05:13:56.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 Processor -- start 2024-09-19T05:13:56.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 -- start start 2024-09-19T05:13:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c72390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1b2270 con 0x560e28c524a0 2024-09-19T05:13:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1afb40 con 0x560e28c5d680 2024-09-19T05:13:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc943d98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1ad9b0 con 0x560e28c52dc0 2024-09-19T05:13:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c4da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43276/0 (socket says 172.21.15.64:43276) 2024-09-19T05:13:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc93a7fc640 1 -- 172.21.15.64:0/1261547489 learned_addr learned my addr 172.21.15.64:0/1261547489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.540+0000 7fc93affd640 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c72390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 -- 172.21.15.64:0/1261547489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 msgr2=0x560e28c72390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c72390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 -- 172.21.15.64:0/1261547489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 msgr2=0x560e28c728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c728d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 -- 172.21.15.64:0/1261547489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e28c5d340 con 0x560e28c524a0 2024-09-19T05:13:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c4da80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc924005ec0 tx=0x7fc924001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3548dab3aefb97f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc9317fa640 1 -- 172.21.15.64:0/1261547489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035070 con 0x560e28c524a0 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc9317fa640 1 -- 172.21.15.64:0/1261547489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc924009520 con 0x560e28c524a0 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.541+0000 7fc9317fa640 1 -- 172.21.15.64:0/1261547489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9240035f0 con 0x560e28c524a0 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 -- 172.21.15.64:0/1261547489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 msgr2=0x560e28c4da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c4da80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc924005ec0 tx=0x7fc924001e50 comp rx=0 tx=0).stop 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 -- 172.21.15.64:0/1261547489 shutdown_connections 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c5d680 0x560e28c728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28c72390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 --2- 172.21.15.64:0/1261547489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c524a0 0x560e28c4da80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 -- 172.21.15.64:0/1261547489 >> 172.21.15.64:0/1261547489 conn(0x560e28c4ea30 msgr2=0x560e28c4d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 -- 172.21.15.64:0/1261547489 shutdown_connections 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 -- 172.21.15.64:0/1261547489 wait complete. 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 -- start start 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.542+0000 7fc943d98740 1 Processor -- start 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28ccfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28cd03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c71bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1ae000 con 0x560e28c5d680 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1af470 con 0x560e28c524a0 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1ae6d0 con 0x560e28c52dc0 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc943d98740 10 client.0 _ll_register_callbacks cb 0x7fc92c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c71bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c71bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43292/0 (socket says 172.21.15.64:43292) 2024-09-19T05:13:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 -- 172.21.15.64:0/3389849312 learned_addr learned my addr 172.21.15.64:0/3389849312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93affd640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28cd03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28ccfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 -- 172.21.15.64:0/3389849312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 msgr2=0x560e28cd03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28cd03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 -- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 msgr2=0x560e28ccfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28ccfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 -- 172.21.15.64:0/3389849312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e28c722e0 con 0x560e28c5d680 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc93a7fc640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c71bf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc924032ad0 tx=0x7fc92402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f72659e4112f43b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc91affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.543+0000 7fc91affd640 1 -- 172.21.15.64:0/3389849312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035070 con 0x560e28c5d680 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc91affd640 1 -- 172.21.15.64:0/3389849312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8571+0+0 (secure 0 0 0) 0x7fc92404d040 con 0x560e28c5d680 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc924009520 con 0x560e28c5d680 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc91affd640 1 -- 172.21.15.64:0/3389849312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9240317c0 con 0x560e28c5d680 2024-09-19T05:13:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 10 client.17313 fetch_fsmap learned FSMap version 25 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e28cd76d0 con 0x560e28c5d680 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc91affd640 1 -- 172.21.15.64:0/3389849312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc924009520 con 0x560e28c5d680 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc91affd640 1 -- 172.21.15.64:0/3389849312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc924067050 con 0x560e28c5d680 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 10 client.17313 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 20 client.17313 populate_metadata read hostname 'smithi064' 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28cd7af0 0x560e28cd9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 4 client.17313 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e28bbfd60 con 0x560e28cd7af0 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.544+0000 7fc93affd640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28cd7af0 0x560e28cd9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc93affd640 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28cd7af0 0x560e28cd9ee0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd0568410804604 server_cookie=3dc547b0ceb70ed3 in_seq=0 out_seq=0 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc91affd640 10 client.17313 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc91affd640 1 -- 172.21.15.64:0/3389849312 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e28bbfd60 con 0x560e28cd7af0 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc91affd640 10 client.17313 handle_command_reply: tid=0 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 client.17313 shutdown 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 msgr2=0x560e28c71bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c71bf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc924032ad0 tx=0x7fc92402a410 comp rx=0 tx=0).stop 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 shutdown_connections 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x560e28cd7af0 0x560e28cd9ee0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560e28c5d680 0x560e28c71bf0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e28c52dc0 0x560e28cd03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc91affd640 0 client.17313 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 --2- 172.21.15.64:0/3389849312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e28c524a0 0x560e28ccfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 >> 172.21.15.64:0/3389849312 conn(0x560e28c4ea30 msgr2=0x560e28c5bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 shutdown_connections 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 1 -- 172.21.15.64:0/3389849312 wait complete. 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.545+0000 7fc943d98740 20 client.17313 trim_cache size 0 max 16384 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:13:56.563 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout: "0bd84f4a-10b2-4046-be21-2b3e87923711": { 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "0bd84f4a-10b2-4046-be21-2b3e87923711", 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:13:56.564 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:13:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9280a33b0 msgr2=0x7fc9280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9280a33b0 0x7fc9280a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9340cdae0 tx=0x7fc93400fd10 comp rx=0 tx=0).stop 2024-09-19T05:13:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 msgr2=0x7fc93c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 0x7fc93c19eea0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c970 tx=0x7fc93400ce30 comp rx=0 tx=0).stop 2024-09-19T05:13:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 shutdown_connections 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc9280a33b0 0x7fc9280a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc93c109140 0x7fc93c19eea0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc93c1049c0 0x7fc93c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 --2- 172.21.15.64:0/1930690091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc93c104010 0x7fc93c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 >> 172.21.15.64:0/1930690091 conn(0x7fc93c0ffa50 msgr2=0x7fc93c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 shutdown_connections 2024-09-19T05:13:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:13:56.546+0000 7fc9419bf640 1 -- 172.21.15.64:0/1930690091 wait complete. 2024-09-19T05:13:56.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:56 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:13:56.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:56 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:56.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:56 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:13:56.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:56 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:57.351 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:57 smithi064 ceph-mon[30960]: pgmap v246: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:57.381 INFO:tasks.cephfs.filesystem:scrub status for tag:0bd84f4a-10b2-4046-be21-2b3e87923711 - {'path': '/', 'tag': '0bd84f4a-10b2-4046-be21-2b3e87923711', 'options': 'recursive,force'} 2024-09-19T05:13:57.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:57 smithi174 ceph-mon[34488]: pgmap v246: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:57.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:57 smithi103 ceph-mon[34446]: pgmap v246: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:13:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:59 smithi174 ceph-mon[34488]: pgmap v247: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:13:59 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:13:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:59 smithi064 ceph-mon[30960]: pgmap v247: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:59.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:13:59 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:13:59.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:59 smithi103 ceph-mon[34446]: pgmap v247: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:13:59.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:13:59 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:14:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:01 smithi174 ceph-mon[34488]: pgmap v248: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:01.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:01 smithi064 ceph-mon[30960]: pgmap v248: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:01 smithi103 ceph-mon[34446]: pgmap v248: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:03.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:03 smithi103 ceph-mon[34446]: pgmap v249: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:03.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:03 smithi174 ceph-mon[34488]: pgmap v249: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:03 smithi064 ceph-mon[30960]: pgmap v249: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:05 smithi174 ceph-mon[34488]: pgmap v250: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:05.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:05 smithi064 ceph-mon[30960]: pgmap v250: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:05.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:05 smithi103 ceph-mon[34446]: pgmap v250: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:07.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:07 smithi174 ceph-mon[34488]: pgmap v251: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:14:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:07 smithi064 ceph-mon[30960]: pgmap v251: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:14:07.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:07 smithi103 ceph-mon[34446]: pgmap v251: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:14:08.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:14:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:14:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:14:08.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:08 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:14:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:14:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:14:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:14:08.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:08 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:14:08.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:14:08.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:14:08.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:14:08.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:08 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:14:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:09 smithi174 ceph-mon[34488]: pgmap v252: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-19T05:14:09.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:09 smithi064 ceph-mon[30960]: pgmap v252: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-19T05:14:09.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:09 smithi103 ceph-mon[34446]: pgmap v252: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-19T05:14:10.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:10 smithi174 ceph-mon[34488]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:14:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:10 smithi064 ceph-mon[30960]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:14:10.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:10 smithi103 ceph-mon[34446]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:14:11.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:11 smithi174 ceph-mon[34488]: pgmap v254: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:14:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:11 smithi064 ceph-mon[30960]: pgmap v254: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:14:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:11 smithi103 ceph-mon[34446]: pgmap v254: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:14:13.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:13 smithi064 ceph-mon[30960]: pgmap v255: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:14:13.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:13 smithi064 ceph-mon[30960]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:14:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:13 smithi103 ceph-mon[34446]: pgmap v255: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:14:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:13 smithi103 ceph-mon[34446]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:14:13.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:13 smithi174 ceph-mon[34488]: pgmap v255: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:14:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:13 smithi174 ceph-mon[34488]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:14:15.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:15 smithi064 ceph-mon[30960]: pgmap v257: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-19T05:14:15.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:15 smithi103 ceph-mon[34446]: pgmap v257: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-19T05:14:15.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:15 smithi174 ceph-mon[34488]: pgmap v257: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-19T05:14:17.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:17 smithi064 ceph-mon[30960]: pgmap v258: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:14:17.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:17 smithi103 ceph-mon[34446]: pgmap v258: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:14:17.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:17 smithi174 ceph-mon[34488]: pgmap v258: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-19T05:14:19.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:19 smithi064 ceph-mon[30960]: pgmap v259: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 238 B/s wr, 7 op/s 2024-09-19T05:14:19.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:19 smithi103 ceph-mon[34446]: pgmap v259: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 238 B/s wr, 7 op/s 2024-09-19T05:14:19.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:19 smithi174 ceph-mon[34488]: pgmap v259: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 238 B/s wr, 7 op/s 2024-09-19T05:14:21.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:21 smithi064 ceph-mon[30960]: pgmap v260: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:14:21.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:21 smithi103 ceph-mon[34446]: pgmap v260: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:14:21.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:21 smithi174 ceph-mon[34488]: pgmap v260: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:14:23.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:23 smithi064 ceph-mon[30960]: pgmap v261: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:14:23.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:23 smithi103 ceph-mon[34446]: pgmap v261: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:14:23.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:23 smithi174 ceph-mon[34488]: pgmap v261: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:14:25.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:25 smithi064 ceph-mon[30960]: pgmap v262: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 87 B/s wr, 5 op/s 2024-09-19T05:14:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:25 smithi103 ceph-mon[34446]: pgmap v262: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 87 B/s wr, 5 op/s 2024-09-19T05:14:25.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:25 smithi174 ceph-mon[34488]: pgmap v262: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 87 B/s wr, 5 op/s 2024-09-19T05:14:27.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:27.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:27 smithi064 ceph-mon[30960]: pgmap v263: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:14:27.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:27 smithi103 ceph-mon[34446]: pgmap v263: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:14:27.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:27 smithi174 ceph-mon[34488]: pgmap v263: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-19T05:14:27.657 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:28.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.141+0000 7f99452c4640 1 -- 172.21.15.64:0/1355705011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940106f10 msgr2=0x7f994010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.141+0000 7f99452c4640 1 --2- 172.21.15.64:0/1355705011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940106f10 0x7f994010ea90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9938009ca0 tx=0x7f993802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 -- 172.21.15.64:0/1355705011 shutdown_connections 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 --2- 172.21.15.64:0/1355705011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940106f10 0x7f994010ea90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 --2- 172.21.15.64:0/1355705011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9940102790 0x7f99401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 --2- 172.21.15.64:0/1355705011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940101de0 0x7f99401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 -- 172.21.15.64:0/1355705011 >> 172.21.15.64:0/1355705011 conn(0x7f99400fd8c0 msgr2=0x7f99400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 -- 172.21.15.64:0/1355705011 shutdown_connections 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 -- 172.21.15.64:0/1355705011 wait complete. 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 Processor -- start 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.142+0000 7f99452c4640 1 -- start start 2024-09-19T05:14:28.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f99452c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 0x7f9940198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f99452c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940102790 0x7f99401988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f99452c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9940106f10 0x7f994019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f99452c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9940111e90 con 0x7f9940101de0 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f99452c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9940111d10 con 0x7f9940102790 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f99452c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9940112010 con 0x7f9940106f10 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 0x7f9940198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 0x7f9940198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38400/0 (socket says 172.21.15.64:38400) 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 -- 172.21.15.64:0/2800310596 learned_addr learned my addr 172.21.15.64:0/2800310596 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9940106f10 0x7f994019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993e7fc640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940102790 0x7f99401988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 -- 172.21.15.64:0/2800310596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9940106f10 msgr2=0x7f994019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9940106f10 0x7f994019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 -- 172.21.15.64:0/2800310596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940102790 msgr2=0x7f99401988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940102790 0x7f99401988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 -- 172.21.15.64:0/2800310596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994019d340 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f993effd640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 0x7f9940198390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f993000d950 tx=0x7f993000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.143+0000 7f9927fff640 1 -- 172.21.15.64:0/2800310596 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930017070 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.144+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994019d630 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.144+0000 7f9927fff640 1 -- 172.21.15.64:0/2800310596 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9930009c30 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.144+0000 7f9927fff640 1 -- 172.21.15.64:0/2800310596 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000f620 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.144+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a4f10 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f9927fff640 1 -- 172.21.15.64:0/2800310596 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f99300109a0 con 0x7f9940101de0 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 Processor -- start 2024-09-19T05:14:28.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 -- start start 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fc0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fcb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fcda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c00f262e0 con 0x555c00fc0dc0 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c00ecdce0 con 0x555c00fc04a0 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f994769d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c00f222a0 con 0x555c00fcb680 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f993effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fcb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fc0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fc0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:60730/0 (socket says 172.21.15.64:60730) 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f993f7fe640 1 -- 172.21.15.64:0/2061611205 learned_addr learned my addr 172.21.15.64:0/2061611205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f993e7fc640 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fcda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.147+0000 7f9927fff640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99280a33b0 0x7f99280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 -- 172.21.15.64:0/2061611205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 msgr2=0x555c00fcda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fcda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 -- 172.21.15.64:0/2061611205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 msgr2=0x555c00fcb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fcb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 -- 172.21.15.64:0/2061611205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c00eb2440 con 0x555c00fc04a0 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993effd640 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fcb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f9927fff640 1 -- 172.21.15.64:0/2800310596 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f99300cfa50 con 0x7f9940101de0 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99280a33b0 0x7f99280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fc0880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9938009430 tx=0x7f9938031bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8edb78bd1a3bfbb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f99257fa640 1 -- 172.21.15.64:0/2061611205 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938031750 con 0x555c00fc04a0 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f99257fa640 1 -- 172.21.15.64:0/2061611205 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9938036040 con 0x555c00fc04a0 2024-09-19T05:14:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.148+0000 7f99257fa640 1 -- 172.21.15.64:0/2061611205 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380307c0 con 0x555c00fc04a0 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 -- 172.21.15.64:0/2061611205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 msgr2=0x555c00fc0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99280a33b0 0x7f99280a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f993804c000 tx=0x7f9938049000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fc0880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9938009430 tx=0x7f9938031bc0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 -- 172.21.15.64:0/2061611205 shutdown_connections 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fcda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fcb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 --2- 172.21.15.64:0/2061611205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fc0880 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 -- 172.21.15.64:0/2061611205 >> 172.21.15.64:0/2061611205 conn(0x555c00fbca30 msgr2=0x555c00fbce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 -- 172.21.15.64:0/2061611205 shutdown_connections 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 -- 172.21.15.64:0/2061611205 wait complete. 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.149+0000 7f994769d740 1 -- start start 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 Processor -- start 2024-09-19T05:14:28.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 0x555c00fcaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc0dc0 0x555c00fc8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fc8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c00fce0d0 con 0x555c00fc04a0 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c00ee1050 con 0x555c00fc0dc0 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c00fce250 con 0x555c00fcb680 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f994769d740 10 client.0 _ll_register_callbacks cb 0x555c00f21160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 0x555c00fcaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 0x555c00fcaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38406/0 (socket says 172.21.15.64:38406) 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 -- 172.21.15.64:0/3233419837 learned_addr learned my addr 172.21.15.64:0/3233419837 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993f7fe640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fc8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993e7fc640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc0dc0 0x555c00fc8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 -- 172.21.15.64:0/3233419837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 msgr2=0x555c00fc8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fc8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 -- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc0dc0 msgr2=0x555c00fc8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc0dc0 0x555c00fc8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 -- 172.21.15.64:0/3233419837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c00c973c0 con 0x555c00fc04a0 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993e7fc640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc0dc0 0x555c00fc8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f993effd640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 0x555c00fcaf70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c00 tx=0x7f993009bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb26208e7bec8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.150+0000 7f9922ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f9922ff5640 1 -- 172.21.15.64:0/3233419837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300d6060 con 0x555c00fc04a0 2024-09-19T05:14:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f9922ff5640 1 -- 172.21.15.64:0/3233419837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f993009b150 con 0x555c00fc04a0 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9938036040 con 0x555c00fc04a0 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f9922ff5640 1 -- 172.21.15.64:0/3233419837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9930020040 con 0x555c00fc04a0 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 10 client.17325 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c00fd9aa0 con 0x555c00fc04a0 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f9922ff5640 1 -- 172.21.15.64:0/3233419837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9938036040 con 0x555c00fc04a0 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f9922ff5640 1 -- 172.21.15.64:0/3233419837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9930097a50 con 0x555c00fc04a0 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 10 client.17325 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 20 client.17325 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c00fdaf00 0x555c00fdd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 4 client.17325 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.151+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c00f33850 con 0x555c00fdaf00 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.152+0000 7f993e7fc640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c00fdaf00 0x555c00fdd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.152+0000 7f993e7fc640 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c00fdaf00 0x555c00fdd370 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5e886c49255fc7 server_cookie=d74ff7172891e0a4 in_seq=0 out_seq=0 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.152+0000 7f9922ff5640 10 client.17325 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f9922ff5640 1 -- 172.21.15.64:0/3233419837 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c00f33850 con 0x555c00fdaf00 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f9922ff5640 10 client.17325 handle_command_reply: tid=0 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 client.17325 shutdown 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 msgr2=0x555c00fcaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 0x555c00fcaf70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c00 tx=0x7f993009bc00 comp rx=0 tx=0).stop 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 shutdown_connections 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fcb680 0x555c00fc8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc0dc0 0x555c00fc8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c00fdaf00 0x555c00fdd370 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 --2- 172.21.15.64:0/3233419837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc04a0 0x555c00fcaf70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f9922ff5640 0 client.17325 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 >> 172.21.15.64:0/3233419837 conn(0x555c00fbca30 msgr2=0x555c00fbbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 shutdown_connections 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 1 -- 172.21.15.64:0/3233419837 wait complete. 2024-09-19T05:14:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.153+0000 7f994769d740 20 client.17325 trim_cache size 0 max 16384 2024-09-19T05:14:28.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.170+0000 7f994769d740 1 Processor -- start 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.170+0000 7f994769d740 1 -- start start 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.170+0000 7f994769d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fbba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f994769d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc0dc0 0x555c00fe2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f994769d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 0x555c00fe3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f994769d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99401b0200 con 0x555c00fcb680 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f994769d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99401adad0 con 0x555c00fc04a0 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f994769d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ab940 con 0x555c00fc0dc0 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 0x555c00fe3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 0x555c00fe3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38416/0 (socket says 172.21.15.64:38416) 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 -- 172.21.15.64:0/942658501 learned_addr learned my addr 172.21.15.64:0/942658501 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993effd640 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fbba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993e7fc640 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc0dc0 0x555c00fe2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 -- 172.21.15.64:0/942658501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc0dc0 msgr2=0x555c00fe2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc0dc0 0x555c00fe2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 -- 172.21.15.64:0/942658501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 msgr2=0x555c00fbba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fbba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 -- 172.21.15.64:0/942658501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c00fcaa90 con 0x555c00fcb680 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f993f7fe640 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 0x555c00fe3180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f993802ade0 tx=0x7f9938009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52e28679e07ff2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:28.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f9924ff9640 1 -- 172.21.15.64:0/942658501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938031750 con 0x555c00fcb680 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f9924ff9640 1 -- 172.21.15.64:0/942658501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9938036030 con 0x555c00fcb680 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.171+0000 7f9924ff9640 1 -- 172.21.15.64:0/942658501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938004040 con 0x555c00fcb680 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 -- 172.21.15.64:0/942658501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 msgr2=0x555c00fe3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 0x555c00fe3180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f993802ade0 tx=0x7f9938009360 comp rx=0 tx=0).stop 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 -- 172.21.15.64:0/942658501 shutdown_connections 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fcb680 0x555c00fe3180 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc0dc0 0x555c00fe2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 --2- 172.21.15.64:0/942658501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fc04a0 0x555c00fbba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 -- 172.21.15.64:0/942658501 >> 172.21.15.64:0/942658501 conn(0x555c00fbca30 msgr2=0x555c00fbb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 -- 172.21.15.64:0/942658501 shutdown_connections 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 -- 172.21.15.64:0/942658501 wait complete. 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.172+0000 7f994769d740 1 -- start start 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 Processor -- start 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc04a0 0x555c00fd3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fd0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fcb680 0x555c00fd11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99401abf90 con 0x555c00fc0dc0 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ad400 con 0x555c00fcb680 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ac660 con 0x555c00fc04a0 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f994769d740 10 client.0 _ll_register_callbacks cb 0x7f993000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fd0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc04a0 0x555c00fd3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fd0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38430/0 (socket says 172.21.15.64:38430) 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 -- 172.21.15.64:0/2220309597 learned_addr learned my addr 172.21.15.64:0/2220309597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993f7fe640 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fcb680 0x555c00fd11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 -- 172.21.15.64:0/2220309597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc04a0 msgr2=0x555c00fd3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc04a0 0x555c00fd3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 -- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fcb680 msgr2=0x555c00fd11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fcb680 0x555c00fd11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.173+0000 7f993e7fc640 1 -- 172.21.15.64:0/2220309597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c00fd16e0 con 0x555c00fc0dc0 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f993e7fc640 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fd0c60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f993400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6aef74f5724faa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f9922ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:28.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f9922ff5640 1 -- 172.21.15.64:0/2220309597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934015d20 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9938036030 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f9922ff5640 1 -- 172.21.15.64:0/2220309597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f99340233e0 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f9922ff5640 1 -- 172.21.15.64:0/2220309597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99340235e0 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 10 client.17331 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c010459a0 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f9922ff5640 1 -- 172.21.15.64:0/2220309597 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9938036030 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f9922ff5640 1 -- 172.21.15.64:0/2220309597 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9934022070 con 0x555c00fc0dc0 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 10 client.17331 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:28.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 20 client.17331 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c01046140 0x555c01048560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 4 client.17331 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.174+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c00fc8af0 con 0x555c01046140 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f993effd640 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c01046140 0x555c01048560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f993effd640 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c01046140 0x555c01048560 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98f146bdad1f6bf server_cookie=34e00d916fb61a18 in_seq=0 out_seq=0 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f9922ff5640 10 client.17331 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f9922ff5640 1 -- 172.21.15.64:0/2220309597 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555c00fc8af0 con 0x555c01046140 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f9922ff5640 10 client.17331 handle_command_reply: tid=0 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 client.17331 shutdown 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 msgr2=0x555c00fd0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fd0c60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9934000c00 tx=0x7f993400b9d0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 shutdown_connections 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c00fcb680 0x555c00fd11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555c01046140 0x555c01048560 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c00fc0dc0 0x555c00fd0c60 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 --2- 172.21.15.64:0/2220309597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c00fc04a0 0x555c00fd3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f9922ff5640 0 client.17331 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.175+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 >> 172.21.15.64:0/2220309597 conn(0x555c00fbca30 msgr2=0x555c00fca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 shutdown_connections 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f994769d740 1 -- 172.21.15.64:0/2220309597 wait complete. 2024-09-19T05:14:28.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f994769d740 20 client.17331 trim_cache size 0 max 16384 2024-09-19T05:14:28.191 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:14:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:14:28.191 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99280a33b0 msgr2=0x7f99280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99280a33b0 0x7f99280a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f993804c000 tx=0x7f9938049000 comp rx=0 tx=0).stop 2024-09-19T05:14:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 msgr2=0x7f9940198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 0x7f9940198390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f993000d950 tx=0x7f993000de10 comp rx=0 tx=0).stop 2024-09-19T05:14:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 shutdown_connections 2024-09-19T05:14:28.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99280a33b0 0x7f99280a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9940106f10 0x7f994019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940102790 0x7f99401988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 --2- 172.21.15.64:0/2800310596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9940101de0 0x7f9940198390 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 >> 172.21.15.64:0/2800310596 conn(0x7f99400fd8c0 msgr2=0x7f99400fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.176+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 shutdown_connections 2024-09-19T05:14:28.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:28.177+0000 7f99452c4640 1 -- 172.21.15.64:0/2800310596 wait complete. 2024-09-19T05:14:29.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:14:29.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:14:29.354 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:29.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:29 smithi064 ceph-mon[30960]: pgmap v264: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:29.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:29 smithi103 ceph-mon[34446]: pgmap v264: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:29.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:29 smithi174 ceph-mon[34488]: pgmap v264: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 -- 172.21.15.64:0/3984056435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908101de0 msgr2=0x7fa9081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/3984056435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908101de0 0x7fa9081021c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009900 tx=0x7fa8fc02a410 comp rx=0 tx=0).stop 2024-09-19T05:14:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 -- 172.21.15.64:0/3984056435 shutdown_connections 2024-09-19T05:14:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/3984056435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa908106f10 0x7fa90810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/3984056435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa908102790 0x7fa9081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/3984056435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908101de0 0x7fa9081021c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 -- 172.21.15.64:0/3984056435 >> 172.21.15.64:0/3984056435 conn(0x7fa9080fd8c0 msgr2=0x7fa9080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 -- 172.21.15.64:0/3984056435 shutdown_connections 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.811+0000 7fa90ebf2640 1 -- 172.21.15.64:0/3984056435 wait complete. 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 Processor -- start 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 -- start start 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa908101de0 0x7fa908198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa908102790 0x7fa9081987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 0x7fa90819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa908111de0 con 0x7fa908106f10 2024-09-19T05:14:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa908111c60 con 0x7fa908102790 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90ebf2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa908111f60 con 0x7fa908101de0 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.812+0000 7fa90d18f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 0x7fa90819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 0x7fa90819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38454/0 (socket says 172.21.15.64:38454) 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 -- 172.21.15.64:0/1980265979 learned_addr learned my addr 172.21.15.64:0/1980265979 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa907fff640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa908102790 0x7fa9081987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90c98e640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa908101de0 0x7fa908198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 -- 172.21.15.64:0/1980265979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa908101de0 msgr2=0x7fa908198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa908101de0 0x7fa908198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 -- 172.21.15.64:0/1980265979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa908102790 msgr2=0x7fa9081987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa908102790 0x7fa9081987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 -- 172.21.15.64:0/1980265979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90819d210 con 0x7fa908106f10 2024-09-19T05:14:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90d18f640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 0x7fa90819cb30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa90000cce0 tx=0x7fa90000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90000f450 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90000faa0 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90819d500 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900018a80 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.813+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9081a4de0 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.814+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9081021c0 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.815+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa90000fc50 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.816+0000 7fa905ffb640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa8f80a3430 0x7fa8f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.816+0000 7fa90c98e640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa8f80a3430 0x7fa8f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.816+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa9000d0180 con 0x7fa908106f10 2024-09-19T05:14:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.817+0000 7fa90c98e640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa8f80a3430 0x7fa8f80a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc031440 tx=0x7fa8fc0095f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:29.820+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa900014030 con 0x7fa908106f10 2024-09-19T05:14:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.033+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa90802d7c0 con 0x7fa908106f10 2024-09-19T05:14:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.033+0000 7fa905ffb640 1 -- 172.21.15.64:0/1980265979 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fa90009cad0 con 0x7fa908106f10 2024-09-19T05:14:30.035 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:14:30.035 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:14:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:14:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa8f80a3430 msgr2=0x7fa8f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa8f80a3430 0x7fa8f80a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc031440 tx=0x7fa8fc0095f0 comp rx=0 tx=0).stop 2024-09-19T05:14:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 msgr2=0x7fa90819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 0x7fa90819cb30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa90000cce0 tx=0x7fa90000a620 comp rx=0 tx=0).stop 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 shutdown_connections 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa8f80a3430 0x7fa8f80a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa908106f10 0x7fa90819cb30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa908102790 0x7fa9081987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 --2- 172.21.15.64:0/1980265979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa908101de0 0x7fa908198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 >> 172.21.15.64:0/1980265979 conn(0x7fa9080fd8c0 msgr2=0x7fa9080febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 shutdown_connections 2024-09-19T05:14:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:30.036+0000 7fa90ebf2640 1 -- 172.21.15.64:0/1980265979 wait complete. 2024-09-19T05:14:30.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:30 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1980265979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:30.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:30 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1980265979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:30.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:30 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1980265979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:30.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:14:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:31.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:31 smithi064 ceph-mon[30960]: pgmap v265: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:14:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 -- 172.21.15.64:0/3889533510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd570102d30 msgr2=0x7fd570103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 --2- 172.21.15.64:0/3889533510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd570102d30 0x7fd570103110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009930 tx=0x7fd56c02a420 comp rx=0 tx=0).stop 2024-09-19T05:14:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 -- 172.21.15.64:0/3889533510 shutdown_connections 2024-09-19T05:14:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 --2- 172.21.15.64:0/3889533510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5701082a0 0x7fd57010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 --2- 172.21.15.64:0/3889533510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5701036e0 0x7fd570107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 --2- 172.21.15.64:0/3889533510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd570102d30 0x7fd570103110 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 -- 172.21.15.64:0/3889533510 >> 172.21.15.64:0/3889533510 conn(0x7fd5700fc7d0 msgr2=0x7fd5700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 -- 172.21.15.64:0/3889533510 shutdown_connections 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.419+0000 7fd577a54640 1 -- 172.21.15.64:0/3889533510 wait complete. 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 Processor -- start 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 -- start start 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd570102d30 0x7fd570100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5701036e0 0x7fd570100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 0x7fd570101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd570111f50 con 0x7fd5701082a0 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd570111dd0 con 0x7fd5701036e0 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd577a54640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701120d0 con 0x7fd570102d30 2024-09-19T05:14:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd575ff1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 0x7fd570101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd575ff1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 0x7fd570101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38460/0 (socket says 172.21.15.64:38460) 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd575ff1640 1 -- 172.21.15.64:0/744008929 learned_addr learned my addr 172.21.15.64:0/744008929 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd5757f0640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd570102d30 0x7fd570100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.420+0000 7fd574fef640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5701036e0 0x7fd570100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd575ff1640 1 -- 172.21.15.64:0/744008929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd570102d30 msgr2=0x7fd570100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd575ff1640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd570102d30 0x7fd570100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd575ff1640 1 -- 172.21.15.64:0/744008929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5701036e0 msgr2=0x7fd570100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd575ff1640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5701036e0 0x7fd570100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd575ff1640 1 -- 172.21.15.64:0/744008929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5701a69f0 con 0x7fd5701082a0 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd575ff1640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 0x7fd570101410 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd568009fd0 tx=0x7fd56800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd5627fc640 1 -- 172.21.15.64:0/744008929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568017070 con 0x7fd5701082a0 2024-09-19T05:14:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5701a6ce0 con 0x7fd5701082a0 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd5627fc640 1 -- 172.21.15.64:0/744008929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd568003e20 con 0x7fd5701082a0 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd5627fc640 1 -- 172.21.15.64:0/744008929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568010460 con 0x7fd5701082a0 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.421+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5701a7220 con 0x7fd5701082a0 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.423+0000 7fd5627fc640 1 -- 172.21.15.64:0/744008929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd568003970 con 0x7fd5701082a0 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 Processor -- start 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 -- start start 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 0x56453bc05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc055d0 0x56453bc05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56453bb6b2f0 con 0x56453bc06b40 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56453bb12cf0 con 0x56453bc10370 2024-09-19T05:14:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.424+0000 7fd579e2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56453bb672b0 con 0x56453bc055d0 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 0x56453bc05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 0x56453bc05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38464/0 (socket says 172.21.15.64:38464) 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 -- 172.21.15.64:0/1927447819 learned_addr learned my addr 172.21.15.64:0/1927447819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd5627fc640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd55c0a33b0 0x7fd55c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd5757f0640 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc055d0 0x56453bc05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd575ff1640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd55c0a33b0 0x7fd55c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 -- 172.21.15.64:0/1927447819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc055d0 msgr2=0x56453bc05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc055d0 0x56453bc05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 -- 172.21.15.64:0/1927447819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 msgr2=0x56453bc12760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc12760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 -- 172.21.15.64:0/1927447819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453baf73a0 con 0x56453bc06b40 2024-09-19T05:14:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd574fef640 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 0x56453bc05090 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd564004650 tx=0x7fd5640041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3163ca02d738a3d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd5627fc640 1 -- 172.21.15.64:0/744008929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fd5680cfa40 con 0x7fd5701082a0 2024-09-19T05:14:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd575ff1640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd55c0a33b0 0x7fd55c0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd56809baa0 tx=0x7fd56809bbf0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.425+0000 7fd55b7fe640 1 -- 172.21.15.64:0/1927447819 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564005020 con 0x56453bc06b40 2024-09-19T05:14:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd55b7fe640 1 -- 172.21.15.64:0/1927447819 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd564005670 con 0x56453bc06b40 2024-09-19T05:14:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd55b7fe640 1 -- 172.21.15.64:0/1927447819 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640104c0 con 0x56453bc06b40 2024-09-19T05:14:31.432 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:31 smithi174 ceph-mon[34488]: pgmap v265: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:14:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 -- 172.21.15.64:0/1927447819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 msgr2=0x56453bc05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 0x56453bc05090 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd564004650 tx=0x7fd5640041d0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 -- 172.21.15.64:0/1927447819 shutdown_connections 2024-09-19T05:14:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc12760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc055d0 0x56453bc05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 --2- 172.21.15.64:0/1927447819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc06b40 0x56453bc05090 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 -- 172.21.15.64:0/1927447819 >> 172.21.15.64:0/1927447819 conn(0x56453bbce310 msgr2=0x56453bc01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 -- 172.21.15.64:0/1927447819 shutdown_connections 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.426+0000 7fd579e2d740 1 -- 172.21.15.64:0/1927447819 wait complete. 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 -- start start 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 Processor -- start 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 0x56453bc0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc0cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc0d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56453bc13010 con 0x56453bc055d0 2024-09-19T05:14:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56453bb26060 con 0x56453bc10370 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56453bc13190 con 0x56453bc06b40 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd579e2d740 10 client.0 _ll_register_callbacks cb 0x56453b8798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 0x56453bc0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 0x56453bc0fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38470/0 (socket says 172.21.15.64:38470) 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd574fef640 1 -- 172.21.15.64:0/2100132804 learned_addr learned my addr 172.21.15.64:0/2100132804 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.427+0000 7fd5757f0640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc0cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc0d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 -- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 msgr2=0x56453bc0cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc0cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 -- 172.21.15.64:0/2100132804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 msgr2=0x56453bc0d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc0d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 -- 172.21.15.64:0/2100132804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453b8dc3c0 con 0x56453bc055d0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd5757f0640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc0cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd574fef640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 0x56453bc0fce0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b3f0 tx=0x7fd564004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723800122c3af89b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd558ff9640 1 -- 172.21.15.64:0/2100132804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564005200 con 0x56453bc055d0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd558ff9640 1 -- 172.21.15.64:0/2100132804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fd5640053a0 con 0x56453bc055d0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd564005670 con 0x56453bc055d0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd558ff9640 1 -- 172.21.15.64:0/2100132804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd564020480 con 0x56453bc055d0 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd579e2d740 10 client.17349 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.428+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56453bc1ea20 con 0x56453bc055d0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd558ff9640 1 -- 172.21.15.64:0/2100132804 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd564005670 con 0x56453bc055d0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd558ff9640 1 -- 172.21.15.64:0/2100132804 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd564010460 con 0x56453bc055d0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd579e2d740 10 client.17349 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd579e2d740 10 client.17349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd579e2d740 20 client.17349 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd579e2d740 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc1fdd0 0x56453bc221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd579e2d740 4 client.17349 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56453bb78860 con 0x56453bc1fdd0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.429+0000 7fd5757f0640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc1fdd0 0x56453bc221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.430+0000 7fd5757f0640 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc1fdd0 0x56453bc221c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=721f39857777ae65 server_cookie=eaed2723c3983ddf in_seq=0 out_seq=0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.430+0000 7fd558ff9640 10 client.17349 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.430+0000 7fd558ff9640 1 -- 172.21.15.64:0/2100132804 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56453bb78860 con 0x56453bc1fdd0 2024-09-19T05:14:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.430+0000 7fd558ff9640 10 client.17349 handle_command_reply: tid=0 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.430+0000 7fd579e2d740 1 client.17349 shutdown 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 msgr2=0x56453bc0fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 0x56453bc0fce0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b3f0 tx=0x7fd564004cb0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 shutdown_connections 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc1fdd0 0x56453bc221c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc10370 0x56453bc0d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc0cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 --2- 172.21.15.64:0/2100132804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc055d0 0x56453bc0fce0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 >> 172.21.15.64:0/2100132804 conn(0x56453bbce310 msgr2=0x56453bc07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd558ff9640 0 client.17349 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 shutdown_connections 2024-09-19T05:14:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 1 -- 172.21.15.64:0/2100132804 wait complete. 2024-09-19T05:14:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.431+0000 7fd579e2d740 20 client.17349 trim_cache size 0 max 16384 2024-09-19T05:14:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 Processor -- start 2024-09-19T05:14:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 -- start start 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc1c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc04c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701b2320 con 0x56453bc10370 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701afbf0 con 0x56453bc055d0 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd579e2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701ada60 con 0x56453bc06b40 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc0be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38484/0 (socket says 172.21.15.64:38484) 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd574fef640 1 -- 172.21.15.64:0/3714161697 learned_addr learned my addr 172.21.15.64:0/3714161697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd574fef640 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc1c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.444+0000 7fd5757f0640 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc04c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd574fef640 1 -- 172.21.15.64:0/3714161697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 msgr2=0x56453bc04c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd574fef640 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc04c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd574fef640 1 -- 172.21.15.64:0/3714161697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 msgr2=0x56453bc1c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd574fef640 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc1c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd574fef640 1 -- 172.21.15.64:0/3714161697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453bc12f50 con 0x56453bc10370 2024-09-19T05:14:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd574fef640 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc0be40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b890 tx=0x7fd564015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fd6980103f5816 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd55affd640 1 -- 172.21.15.64:0/3714161697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564011e60 con 0x56453bc10370 2024-09-19T05:14:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd55affd640 1 -- 172.21.15.64:0/3714161697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd564010460 con 0x56453bc10370 2024-09-19T05:14:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd55affd640 1 -- 172.21.15.64:0/3714161697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564020d00 con 0x56453bc10370 2024-09-19T05:14:31.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd579e2d740 1 -- 172.21.15.64:0/3714161697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 msgr2=0x56453bc0be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.445+0000 7fd579e2d740 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc0be40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b890 tx=0x7fd564015040 comp rx=0 tx=0).stop 2024-09-19T05:14:31.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- 172.21.15.64:0/3714161697 shutdown_connections 2024-09-19T05:14:31.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc0be40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc04c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 --2- 172.21.15.64:0/3714161697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc1c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- 172.21.15.64:0/3714161697 >> 172.21.15.64:0/3714161697 conn(0x56453bbce310 msgr2=0x56453bc0b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- 172.21.15.64:0/3714161697 shutdown_connections 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- 172.21.15.64:0/3714161697 wait complete. 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- start start 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 Processor -- start 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc18b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701ae0b0 con 0x56453bc10370 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701af520 con 0x56453bc055d0 2024-09-19T05:14:31.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.446+0000 7fd579e2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701ae780 con 0x56453bc06b40 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd579e2d740 10 client.0 _ll_register_callbacks cb 0x7fd56c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc097c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38492/0 (socket says 172.21.15.64:38492) 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 -- 172.21.15.64:0/884884693 learned_addr learned my addr 172.21.15.64:0/884884693 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd5757f0640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc18b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 -- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 msgr2=0x56453bc18b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc18b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 -- 172.21.15.64:0/884884693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 msgr2=0x56453bc185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc185d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 -- 172.21.15.64:0/884884693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453bc02960 con 0x56453bc10370 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd574fef640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc097c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x56453bc284b0 tx=0x7fd56400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d51880a5268525 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd558ff9640 1 -- 172.21.15.64:0/884884693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564020620 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd558ff9640 1 -- 172.21.15.64:0/884884693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fd564010aa0 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.447+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd564010460 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd558ff9640 1 -- 172.21.15.64:0/884884693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd564010ce0 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 10 client.17355 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56453bc8a120 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd558ff9640 1 -- 172.21.15.64:0/884884693 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd564010460 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd558ff9640 1 -- 172.21.15.64:0/884884693 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd564047020 con 0x56453bc10370 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 10 client.17355 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 10 client.17355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 20 client.17355 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc8a7e0 0x56453bc8cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 4 client.17355 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56453bb72d70 con 0x56453bc8a7e0 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.448+0000 7fd5757f0640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc8a7e0 0x56453bc8cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd558ff9640 10 client.17355 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd5757f0640 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc8a7e0 0x56453bc8cbd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=918fb3579c51fea5 server_cookie=4927b769785752a2 in_seq=0 out_seq=0 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd558ff9640 1 -- 172.21.15.64:0/884884693 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56453bb72d70 con 0x56453bc8a7e0 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd558ff9640 10 client.17355 handle_command_reply: tid=0 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 client.17355 shutdown 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 msgr2=0x56453bc097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc097c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x56453bc284b0 tx=0x7fd56400b950 comp rx=0 tx=0).stop 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 shutdown_connections 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x56453bc8a7e0 0x56453bc8cbd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56453bc10370 0x56453bc097c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56453bc06b40 0x56453bc18b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 --2- 172.21.15.64:0/884884693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56453bc055d0 0x56453bc185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 >> 172.21.15.64:0/884884693 conn(0x56453bbce310 msgr2=0x56453bc0b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd558ff9640 0 client.17355 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 shutdown_connections 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 1 -- 172.21.15.64:0/884884693 wait complete. 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.449+0000 7fd579e2d740 20 client.17355 trim_cache size 0 max 16384 2024-09-19T05:14:31.457 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd55c0a33b0 msgr2=0x7fd55c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd55c0a33b0 0x7fd55c0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd56809baa0 tx=0x7fd56809bbf0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 msgr2=0x7fd570101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 0x7fd570101410 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd568009fd0 tx=0x7fd56800b260 comp rx=0 tx=0).stop 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 shutdown_connections 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd55c0a33b0 0x7fd55c0a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5701082a0 0x7fd570101410 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5701036e0 0x7fd570100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 --2- 172.21.15.64:0/744008929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd570102d30 0x7fd570100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 >> 172.21.15.64:0/744008929 conn(0x7fd5700fc7d0 msgr2=0x7fd570105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 shutdown_connections 2024-09-19T05:14:31.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:31.450+0000 7fd577a54640 1 -- 172.21.15.64:0/744008929 wait complete. 2024-09-19T05:14:31.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:31 smithi103 ceph-mon[34446]: pgmap v265: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:14:32.231 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:14:32.487 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:33.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.038+0000 7fa778482640 1 -- 172.21.15.64:0/1496092519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 msgr2=0x7fa7701021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.038+0000 7fa778482640 1 --2- 172.21.15.64:0/1496092519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701021b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009900 tx=0x7fa76c02a410 comp rx=0 tx=0).stop 2024-09-19T05:14:33.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.038+0000 7fa778482640 1 -- 172.21.15.64:0/1496092519 shutdown_connections 2024-09-19T05:14:33.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 --2- 172.21.15.64:0/1496092519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770106f00 0x7fa77010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 --2- 172.21.15.64:0/1496092519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770102780 0x7fa7701067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 --2- 172.21.15.64:0/1496092519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701021b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 -- 172.21.15.64:0/1496092519 >> 172.21.15.64:0/1496092519 conn(0x7fa7700fd8b0 msgr2=0x7fa7700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 -- 172.21.15.64:0/1496092519 shutdown_connections 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 -- 172.21.15.64:0/1496092519 wait complete. 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 Processor -- start 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.039+0000 7fa778482640 1 -- start start 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770102780 0x7fa770198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770106f00 0x7fa77019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa770111eb0 con 0x7fa770101dd0 2024-09-19T05:14:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa770111d30 con 0x7fa770106f00 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa770112030 con 0x7fa770102780 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38522/0 (socket says 172.21.15.64:38522) 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 -- 172.21.15.64:0/2736107541 learned_addr learned my addr 172.21.15.64:0/2736107541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa776a1f640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770106f00 0x7fa77019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa775a1d640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770102780 0x7fa770198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 -- 172.21.15.64:0/2736107541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770102780 msgr2=0x7fa770198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770102780 0x7fa770198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 -- 172.21.15.64:0/2736107541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770106f00 msgr2=0x7fa77019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770106f00 0x7fa77019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 -- 172.21.15.64:0/2736107541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77019d570 con 0x7fa770101dd0 2024-09-19T05:14:33.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa776a1f640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770106f00 0x7fa77019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa775a1d640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770102780 0x7fa770198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa77621e640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701985c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa76c006010 tx=0x7fa76c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa7677fe640 1 -- 172.21.15.64:0/2736107541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c03f030 con 0x7fa770101dd0 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77019d800 con 0x7fa770101dd0 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa7677fe640 1 -- 172.21.15.64:0/2736107541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa76c0035f0 con 0x7fa770101dd0 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.040+0000 7fa7677fe640 1 -- 172.21.15.64:0/2736107541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c035070 con 0x7fa770101dd0 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.041+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7701a5140 con 0x7fa770101dd0 2024-09-19T05:14:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.043+0000 7fa7677fe640 1 -- 172.21.15.64:0/2736107541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa76c003b10 con 0x7fa770101dd0 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.043+0000 7fa77a85b740 1 Processor -- start 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.043+0000 7fa77a85b740 1 -- start start 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67eb40 0x55af8e67d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 0x55af8e67da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e68a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77a85b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55af8e5e32f0 con 0x55af8e67d5d0 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77a85b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af8e58acf0 con 0x55af8e67eb40 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77a85b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55af8e5df2b0 con 0x55af8e688370 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 0x55af8e67da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa7677fe640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa75c0a33b0 0x7fa75c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 0x55af8e67da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38538/0 (socket says 172.21.15.64:38538) 2024-09-19T05:14:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 -- 172.21.15.64:0/2650029718 learned_addr learned my addr 172.21.15.64:0/2650029718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa776a1f640 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67eb40 0x55af8e67d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa776a1f640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa75c0a33b0 0x7fa75c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa775a1d640 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e68a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa7677fe640 1 -- 172.21.15.64:0/2736107541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa76c0f0760 con 0x7fa770101dd0 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 -- 172.21.15.64:0/2650029718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 msgr2=0x55af8e68a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e68a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 -- 172.21.15.64:0/2650029718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67eb40 msgr2=0x55af8e67d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67eb40 0x55af8e67d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.044+0000 7fa77621e640 1 -- 172.21.15.64:0/2650029718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8e56f3a0 con 0x55af8e67d5d0 2024-09-19T05:14:33.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa776a1f640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa75c0a33b0 0x7fa75c0a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa768009fd0 tx=0x7fa768008220 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa77621e640 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 0x55af8e67da10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0edee0 tx=0x7fa76c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c66cf3fd1c60749 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa764ff9640 1 -- 172.21.15.64:0/2650029718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c049070 con 0x55af8e67d5d0 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa764ff9640 1 -- 172.21.15.64:0/2650029718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa76c0bd110 con 0x55af8e67d5d0 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa764ff9640 1 -- 172.21.15.64:0/2650029718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0ee260 con 0x55af8e67d5d0 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa77a85b740 1 -- 172.21.15.64:0/2650029718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 msgr2=0x55af8e67da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.045+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 0x55af8e67da10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0edee0 tx=0x7fa76c02c000 comp rx=0 tx=0).stop 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- 172.21.15.64:0/2650029718 shutdown_connections 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e68a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67d5d0 0x55af8e67da10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2650029718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67eb40 0x55af8e67d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- 172.21.15.64:0/2650029718 >> 172.21.15.64:0/2650029718 conn(0x55af8e646310 msgr2=0x55af8e679c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- 172.21.15.64:0/2650029718 shutdown_connections 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- 172.21.15.64:0/2650029718 wait complete. 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- start start 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 Processor -- start 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67d5d0 0x55af8e687d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e684fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e6855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55af8e68ae30 con 0x55af8e67eb40 2024-09-19T05:14:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af8e59e060 con 0x55af8e67d5d0 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55af8e68afb0 con 0x55af8e688370 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa77a85b740 10 client.0 _ll_register_callbacks cb 0x55af8e2f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa775a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e684fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa775a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e684fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38542/0 (socket says 172.21.15.64:38542) 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.046+0000 7fa775a1d640 1 -- 172.21.15.64:0/2925240196 learned_addr learned my addr 172.21.15.64:0/2925240196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa77621e640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67d5d0 0x55af8e687d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa775a1d640 1 -- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 msgr2=0x55af8e6855d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa775a1d640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e6855d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa775a1d640 1 -- 172.21.15.64:0/2925240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67d5d0 msgr2=0x55af8e687d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa775a1d640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67d5d0 0x55af8e687d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa775a1d640 1 -- 172.21.15.64:0/2925240196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8e3543c0 con 0x55af8e67eb40 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa77621e640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67d5d0 0x55af8e687d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa775a1d640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e684fe0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa76000f4c0 tx=0x7fa76000f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf043c5c621f995d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa759ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa759ffb640 1 -- 172.21.15.64:0/2925240196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760013850 con 0x55af8e67eb40 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa759ffb640 1 -- 172.21.15.64:0/2925240196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa7600139f0 con 0x55af8e67eb40 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76c0bd110 con 0x55af8e67eb40 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa759ffb640 1 -- 172.21.15.64:0/2925240196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa760013bf0 con 0x55af8e67eb40 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa77a85b740 10 client.17367 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:33.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55af8e6967e0 con 0x55af8e67eb40 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.047+0000 7fa759ffb640 1 -- 172.21.15.64:0/2925240196 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa76c0bd110 con 0x55af8e67eb40 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa759ffb640 1 -- 172.21.15.64:0/2925240196 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa760022070 con 0x55af8e67eb40 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77a85b740 10 client.17367 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77a85b740 10 client.17367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77a85b740 20 client.17367 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e697cf0 0x55af8e69a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77a85b740 4 client.17367 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af8e5f0860 con 0x55af8e697cf0 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77621e640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e697cf0 0x55af8e69a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa759ffb640 10 client.17367 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.048+0000 7fa77621e640 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e697cf0 0x55af8e69a140 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52f660710e930f5c server_cookie=d444a321fb54f80d in_seq=0 out_seq=0 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa759ffb640 1 -- 172.21.15.64:0/2925240196 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af8e5f0860 con 0x55af8e697cf0 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa759ffb640 10 client.17367 handle_command_reply: tid=0 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 client.17367 shutdown 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 msgr2=0x55af8e684fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e684fe0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa76000f4c0 tx=0x7fa76000f980 comp rx=0 tx=0).stop 2024-09-19T05:14:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 shutdown_connections 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e688370 0x55af8e6855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e697cf0 0x55af8e69a140 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e684fe0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 --2- 172.21.15.64:0/2925240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e67d5d0 0x55af8e687d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 >> 172.21.15.64:0/2925240196 conn(0x55af8e646310 msgr2=0x55af8e67f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa759ffb640 0 client.17367 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 shutdown_connections 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.049+0000 7fa77a85b740 1 -- 172.21.15.64:0/2925240196 wait complete. 2024-09-19T05:14:33.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.050+0000 7fa77a85b740 20 client.17367 trim_cache size 0 max 16384 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 Processor -- start 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 -- start start 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e686f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e680120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e680660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701b02d0 con 0x55af8e67eb40 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701adba0 con 0x55af8e688370 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77a85b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701aba10 con 0x55af8e67d5d0 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e680120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e680120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38556/0 (socket says 172.21.15.64:38556) 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 -- 172.21.15.64:0/3188548515 learned_addr learned my addr 172.21.15.64:0/3188548515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:33.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa77621e640 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e686f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa776a1f640 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e680660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 -- 172.21.15.64:0/3188548515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 msgr2=0x55af8e686f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e686f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 -- 172.21.15.64:0/3188548515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 msgr2=0x55af8e680660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e680660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 -- 172.21.15.64:0/3188548515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8e67a960 con 0x55af8e67eb40 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.063+0000 7fa775a1d640 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e680120 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa760000c00 tx=0x7fa76000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cf80449aa0ec0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa75bfff640 1 -- 172.21.15.64:0/3188548515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760033070 con 0x55af8e67eb40 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa75bfff640 1 -- 172.21.15.64:0/3188548515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa760023550 con 0x55af8e67eb40 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa75bfff640 1 -- 172.21.15.64:0/3188548515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760028070 con 0x55af8e67eb40 2024-09-19T05:14:33.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 -- 172.21.15.64:0/3188548515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 msgr2=0x55af8e680120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e680120 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa760000c00 tx=0x7fa76000ae90 comp rx=0 tx=0).stop 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 -- 172.21.15.64:0/3188548515 shutdown_connections 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e680660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e680120 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3188548515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e686f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 -- 172.21.15.64:0/3188548515 >> 172.21.15.64:0/3188548515 conn(0x55af8e646310 msgr2=0x55af8e686840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 -- 172.21.15.64:0/3188548515 shutdown_connections 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.064+0000 7fa77a85b740 1 -- 172.21.15.64:0/3188548515 wait complete. 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 -- start start 2024-09-19T05:14:33.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 Processor -- start 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e6908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e68bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e68c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701ac060 con 0x55af8e67eb40 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701ad4d0 con 0x55af8e688370 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7701ac730 con 0x55af8e67d5d0 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77a85b740 10 client.0 _ll_register_callbacks cb 0x7fa76000a510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e68bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e68bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38558/0 (socket says 172.21.15.64:38558) 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 -- 172.21.15.64:0/3697607108 learned_addr learned my addr 172.21.15.64:0/3697607108 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa77621e640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e6908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa776a1f640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e68c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 -- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 msgr2=0x55af8e6908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e6908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 -- 172.21.15.64:0/3697607108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 msgr2=0x55af8e68c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.065+0000 7fa775a1d640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e68c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa775a1d640 1 -- 172.21.15.64:0/3697607108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8e68c9d0 con 0x55af8e67eb40 2024-09-19T05:14:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77621e640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e6908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa776a1f640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e68c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa775a1d640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e68bc20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa76000f4c0 tx=0x7fa7600233e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad843f4ad4a06bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa759ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa759ffb640 1 -- 172.21.15.64:0/3697607108 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76003c060 con 0x55af8e67eb40 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa759ffb640 1 -- 172.21.15.64:0/3697607108 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa760023700 con 0x55af8e67eb40 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa760023550 con 0x55af8e67eb40 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa759ffb640 1 -- 172.21.15.64:0/3697607108 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa760033050 con 0x55af8e67eb40 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 10 client.17373 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55af8e702ac0 con 0x55af8e67eb40 2024-09-19T05:14:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa759ffb640 1 -- 172.21.15.64:0/3697607108 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa760023550 con 0x55af8e67eb40 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa759ffb640 1 -- 172.21.15.64:0/3697607108 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa760022040 con 0x55af8e67eb40 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 10 client.17373 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 10 client.17373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 20 client.17373 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e703180 0x55af8e7055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 4 client.17373 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.066+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af8e5ead70 con 0x55af8e703180 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77621e640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e703180 0x55af8e7055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77621e640 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e703180 0x55af8e7055a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c89da15ea72fd58 server_cookie=6d11932f8567925a in_seq=0 out_seq=0 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa759ffb640 10 client.17373 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa759ffb640 1 -- 172.21.15.64:0/3697607108 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af8e5ead70 con 0x55af8e703180 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa759ffb640 10 client.17373 handle_command_reply: tid=0 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 client.17373 shutdown 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 msgr2=0x55af8e68bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e68bc20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa76000f4c0 tx=0x7fa7600233e0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 shutdown_connections 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af8e688370 0x55af8e68c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55af8e703180 0x55af8e7055a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55af8e67eb40 0x55af8e68bc20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 --2- 172.21.15.64:0/3697607108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af8e67d5d0 0x55af8e6908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.130 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:33.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa759ffb640 0 client.17373 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.067+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 >> 172.21.15.64:0/3697607108 conn(0x55af8e646310 msgr2=0x55af8e687060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 shutdown_connections 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa77a85b740 1 -- 172.21.15.64:0/3697607108 wait complete. 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa77a85b740 20 client.17373 trim_cache size 0 max 16384 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa75c0a33b0 msgr2=0x7fa75c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa75c0a33b0 0x7fa75c0a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa768009fd0 tx=0x7fa768008220 comp rx=0 tx=0).stop 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 msgr2=0x7fa7701985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701985c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa76c006010 tx=0x7fa76c004270 comp rx=0 tx=0).stop 2024-09-19T05:14:33.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 shutdown_connections 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa75c0a33b0 0x7fa75c0a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa770106f00 0x7fa77019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa770102780 0x7fa770198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 --2- 172.21.15.64:0/2736107541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa770101dd0 0x7fa7701985c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 >> 172.21.15.64:0/2736107541 conn(0x7fa7700fd8b0 msgr2=0x7fa7700fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 shutdown_connections 2024-09-19T05:14:33.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:33.068+0000 7fa778482640 1 -- 172.21.15.64:0/2736107541 wait complete. 2024-09-19T05:14:33.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:33 smithi064 ceph-mon[30960]: pgmap v266: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:33.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:33 smithi103 ceph-mon[34446]: pgmap v266: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:33.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:33 smithi174 ceph-mon[34488]: pgmap v266: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:33.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:14:34.041 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:34.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 -- 172.21.15.64:0/2686396893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594106a90 msgr2=0x7fc594106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2686396893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594106a90 0x7fc594106e70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc590009930 tx=0x7fc59002a420 comp rx=0 tx=0).stop 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 -- 172.21.15.64:0/2686396893 shutdown_connections 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2686396893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc594101430 0x7fc59410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2686396893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc594100a90 0x7fc594100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2686396893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594106a90 0x7fc594106e70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 -- 172.21.15.64:0/2686396893 >> 172.21.15.64:0/2686396893 conn(0x7fc5940fc7d0 msgr2=0x7fc5940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 -- 172.21.15.64:0/2686396893 shutdown_connections 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.496+0000 7fc59adc2640 1 -- 172.21.15.64:0/2686396893 wait complete. 2024-09-19T05:14:34.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 Processor -- start 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 -- start start 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc594100a90 0x7fc5941972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 0x7fc594197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc594106a90 0x7fc59419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110d60 con 0x7fc594101430 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110be0 con 0x7fc594106a90 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59adc2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc594110ee0 con 0x7fc594100a90 2024-09-19T05:14:34.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 0x7fc594197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 0x7fc594197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35346/0 (socket says 172.21.15.64:35346) 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc58bfff640 1 -- 172.21.15.64:0/2937307869 learned_addr learned my addr 172.21.15.64:0/2937307869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc598b5e640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc594100a90 0x7fc5941972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.497+0000 7fc59935f640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc594106a90 0x7fc59419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc58bfff640 1 -- 172.21.15.64:0/2937307869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc594100a90 msgr2=0x7fc5941972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc58bfff640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc594100a90 0x7fc5941972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc58bfff640 1 -- 172.21.15.64:0/2937307869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc594106a90 msgr2=0x7fc59419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc58bfff640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc594106a90 0x7fc59419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc58bfff640 1 -- 172.21.15.64:0/2937307869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59419c270 con 0x7fc594101430 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc58bfff640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 0x7fc594197800 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc58400b4a0 tx=0x7fc58400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc589ffb640 1 -- 172.21.15.64:0/2937307869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584014070 con 0x7fc594101430 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc589ffb640 1 -- 172.21.15.64:0/2937307869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc584010ac0 con 0x7fc594101430 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59419c560 con 0x7fc594101430 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc589ffb640 1 -- 172.21.15.64:0/2937307869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584017c30 con 0x7fc594101430 2024-09-19T05:14:34.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.498+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5941a3e40 con 0x7fc594101430 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc589ffb640 1 -- 172.21.15.64:0/2937307869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc584017070 con 0x7fc594101430 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 Processor -- start 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 -- start start 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5d5d0 0x5600a6c5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c6c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5600a6bc32f0 con 0x5600a6c6a580 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600a6b6acf0 con 0x5600a6c5d5d0 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.501+0000 7fc59d19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600a6bbf2b0 con 0x5600a6c5eb40 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c6c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c6c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35354/0 (socket says 172.21.15.64:35354) 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 -- 172.21.15.64:0/2921905193 learned_addr learned my addr 172.21.15.64:0/2921905193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc589ffb640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc57c0a3430 0x7fc57c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc598b5e640 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5d5d0 0x5600a6c5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc59935f640 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc59935f640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc57c0a3430 0x7fc57c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc589ffb640 1 -- 172.21.15.64:0/2937307869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc5840cf1c0 con 0x7fc594101430 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 -- 172.21.15.64:0/2921905193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 msgr2=0x5600a6c5d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c5d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 -- 172.21.15.64:0/2921905193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5d5d0 msgr2=0x5600a6c5da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5d5d0 0x5600a6c5da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 -- 172.21.15.64:0/2921905193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a6b4f3a0 con 0x5600a6c6a580 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc598b5e640 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5d5d0 0x5600a6c5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:34.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc58bfff640 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c6c970 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc584069c70 tx=0x7fc58400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c70ba90a80ee236 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc59935f640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc57c0a3430 0x7fc57c0a58f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc58c007fd0 tx=0x7fc58c007520 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.502+0000 7fc57affd640 1 -- 172.21.15.64:0/2921905193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58401e070 con 0x5600a6c6a580 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc57affd640 1 -- 172.21.15.64:0/2921905193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58409b630 con 0x5600a6c6a580 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc57affd640 1 -- 172.21.15.64:0/2921905193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584097ca0 con 0x5600a6c6a580 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 -- 172.21.15.64:0/2921905193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 msgr2=0x5600a6c6c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c6c970 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc584069c70 tx=0x7fc58400bf90 comp rx=0 tx=0).stop 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 -- 172.21.15.64:0/2921905193 shutdown_connections 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c6c970 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5d5d0 0x5600a6c5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 --2- 172.21.15.64:0/2921905193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 -- 172.21.15.64:0/2921905193 >> 172.21.15.64:0/2921905193 conn(0x5600a6c26310 msgr2=0x5600a6c59c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 -- 172.21.15.64:0/2921905193 shutdown_connections 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.503+0000 7fc59d19b740 1 -- 172.21.15.64:0/2921905193 wait complete. 2024-09-19T05:14:34.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 -- start start 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 Processor -- start 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c65f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5600a6c60690 con 0x5600a6c5d5d0 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600a6b7e060 con 0x5600a6c6a580 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600a6c60810 con 0x5600a6c5eb40 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.504+0000 7fc59d19b740 10 client.0 _ll_register_callbacks cb 0x5600a68d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c65a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35370/0 (socket says 172.21.15.64:35370) 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 -- 172.21.15.64:0/3249765068 learned_addr learned my addr 172.21.15.64:0/3249765068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc59935f640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc58bfff640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c65f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 -- 172.21.15.64:0/3249765068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 msgr2=0x5600a6c65f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c65f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 -- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 msgr2=0x5600a6c78350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 -- 172.21.15.64:0/3249765068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a69343c0 con 0x5600a6c5d5d0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc58bfff640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c65f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc598b5e640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c65a10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc590009930 tx=0x7fc5900316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7f8140c5ffffb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc573fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc573fff640 1 -- 172.21.15.64:0/3249765068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590035040 con 0x5600a6c5d5d0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc58409b630 con 0x5600a6c5d5d0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.505+0000 7fc573fff640 1 -- 172.21.15.64:0/3249765068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc590004240 con 0x5600a6c5d5d0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc573fff640 1 -- 172.21.15.64:0/3249765068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc590004440 con 0x5600a6c5d5d0 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 10 client.17385 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:34.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600a6c7af70 con 0x5600a6c5d5d0 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc573fff640 1 -- 172.21.15.64:0/3249765068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58409b630 con 0x5600a6c5d5d0 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc573fff640 1 -- 172.21.15.64:0/3249765068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc59004e070 con 0x5600a6c5d5d0 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 10 client.17385 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 20 client.17385 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6c7c590 0x5600a6c7ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 4 client.17385 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600a6bd0860 con 0x5600a6c7c590 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.506+0000 7fc58bfff640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6c7c590 0x5600a6c7ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.507+0000 7fc58bfff640 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6c7c590 0x5600a6c7ea30 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f184760499f96d server_cookie=66b7591383966bbe in_seq=0 out_seq=0 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.507+0000 7fc573fff640 10 client.17385 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.507+0000 7fc573fff640 1 -- 172.21.15.64:0/3249765068 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600a6bd0860 con 0x5600a6c7c590 2024-09-19T05:14:34.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.507+0000 7fc573fff640 10 client.17385 handle_command_reply: tid=0 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 client.17385 shutdown 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 msgr2=0x5600a6c65a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c65a10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc590009930 tx=0x7fc5900316d0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 shutdown_connections 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c65f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6c7c590 0x5600a6c7ea30 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 --2- 172.21.15.64:0/3249765068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c65a10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 >> 172.21.15.64:0/3249765068 conn(0x5600a6c26310 msgr2=0x5600a6c5f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc573fff640 0 client.17385 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 shutdown_connections 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 1 -- 172.21.15.64:0/3249765068 wait complete. 2024-09-19T05:14:34.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.508+0000 7fc59d19b740 20 client.17385 trim_cache size 0 max 16384 2024-09-19T05:14:34.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 Processor -- start 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 -- start start 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5d5d0 0x5600a6c841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5eb40 0x5600a6c846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941adf50 con 0x5600a6c6a580 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ab820 con 0x5600a6c5eb40 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59d19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941a9690 con 0x5600a6c5d5d0 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35372/0 (socket says 172.21.15.64:35372) 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 -- 172.21.15.64:0/1997641328 learned_addr learned my addr 172.21.15.64:0/1997641328 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc598b5e640 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5d5d0 0x5600a6c841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc58bfff640 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5eb40 0x5600a6c846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 -- 172.21.15.64:0/1997641328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5d5d0 msgr2=0x5600a6c841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5d5d0 0x5600a6c841a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 -- 172.21.15.64:0/1997641328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5eb40 msgr2=0x5600a6c846e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.521+0000 7fc59935f640 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5eb40 0x5600a6c846e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.522+0000 7fc59935f640 1 -- 172.21.15.64:0/1997641328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a6c66bb0 con 0x5600a6c6a580 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.522+0000 7fc598b5e640 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5d5d0 0x5600a6c841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.522+0000 7fc59935f640 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0145b0 tx=0x7fc58c007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4af24be9a718537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.522+0000 7fc57a7fc640 1 -- 172.21.15.64:0/1997641328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c026070 con 0x5600a6c6a580 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.522+0000 7fc57a7fc640 1 -- 172.21.15.64:0/1997641328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58c00f480 con 0x5600a6c6a580 2024-09-19T05:14:34.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.522+0000 7fc57a7fc640 1 -- 172.21.15.64:0/1997641328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c005220 con 0x5600a6c6a580 2024-09-19T05:14:34.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- 172.21.15.64:0/1997641328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 msgr2=0x5600a6c78350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0145b0 tx=0x7fc58c007940 comp rx=0 tx=0).stop 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- 172.21.15.64:0/1997641328 shutdown_connections 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c6a580 0x5600a6c78350 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c5eb40 0x5600a6c846e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1997641328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5d5d0 0x5600a6c841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- 172.21.15.64:0/1997641328 >> 172.21.15.64:0/1997641328 conn(0x5600a6c26310 msgr2=0x5600a6c64920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- 172.21.15.64:0/1997641328 shutdown_connections 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- 172.21.15.64:0/1997641328 wait complete. 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- start start 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 Processor -- start 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c73a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941a9ce0 con 0x5600a6c5d5d0 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941ab150 con 0x5600a6c6a580 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941aa3b0 con 0x5600a6c5eb40 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.523+0000 7fc59d19b740 10 client.0 _ll_register_callbacks cb 0x7fc590009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c784b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35384/0 (socket says 172.21.15.64:35384) 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 -- 172.21.15.64:0/1274322233 learned_addr learned my addr 172.21.15.64:0/1274322233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc59935f640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c73a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc58bfff640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 -- 172.21.15.64:0/1274322233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 msgr2=0x5600a6c789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 -- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 msgr2=0x5600a6c73a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c73a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 -- 172.21.15.64:0/1274322233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a6c74230 con 0x5600a6c5d5d0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc598b5e640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c784b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc590009930 tx=0x7fc590032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ad3ad4fd2f611a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc573fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc573fff640 1 -- 172.21.15.64:0/1274322233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590035070 con 0x5600a6c5d5d0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc573fff640 1 -- 172.21.15.64:0/1274322233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fc590005220 con 0x5600a6c5d5d0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc58c00f480 con 0x5600a6c5d5d0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc573fff640 1 -- 172.21.15.64:0/1274322233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc590054070 con 0x5600a6c5d5d0 2024-09-19T05:14:34.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.524+0000 7fc59d19b740 10 client.17391 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600a6ce6df0 con 0x5600a6c5d5d0 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc573fff640 1 -- 172.21.15.64:0/1274322233 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc58c00f480 con 0x5600a6c5d5d0 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc573fff640 1 -- 172.21.15.64:0/1274322233 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc59004e070 con 0x5600a6c5d5d0 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 10 client.17391 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 20 client.17391 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6ce7510 0x5600a6ce9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 4 client.17391 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600a6c662e0 con 0x5600a6ce7510 2024-09-19T05:14:34.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.525+0000 7fc58bfff640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6ce7510 0x5600a6ce9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc573fff640 10 client.17391 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc58bfff640 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6ce7510 0x5600a6ce9900 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b99fb724630b1c0 server_cookie=bd9c33473d6738da in_seq=0 out_seq=0 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc573fff640 1 -- 172.21.15.64:0/1274322233 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600a6c662e0 con 0x5600a6ce7510 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc573fff640 10 client.17391 handle_command_reply: tid=0 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 client.17391 shutdown 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 msgr2=0x5600a6c784b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c784b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc590009930 tx=0x7fc590032d60 comp rx=0 tx=0).stop 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 shutdown_connections 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600a6c6a580 0x5600a6c73a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600a6c5eb40 0x5600a6c789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5600a6ce7510 0x5600a6ce9900 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc573fff640 0 client.17391 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:34.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 --2- 172.21.15.64:0/1274322233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600a6c5d5d0 0x5600a6c784b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 >> 172.21.15.64:0/1274322233 conn(0x5600a6c26310 msgr2=0x5600a6c5f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:34.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 shutdown_connections 2024-09-19T05:14:34.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 1 -- 172.21.15.64:0/1274322233 wait complete. 2024-09-19T05:14:34.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.526+0000 7fc59d19b740 20 client.17391 trim_cache size 0 max 16384 2024-09-19T05:14:34.535 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc57c0a3430 msgr2=0x7fc57c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc57c0a3430 0x7fc57c0a58f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc58c007fd0 tx=0x7fc58c007520 comp rx=0 tx=0).stop 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 msgr2=0x7fc594197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 0x7fc594197800 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc58400b4a0 tx=0x7fc58400b960 comp rx=0 tx=0).stop 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 shutdown_connections 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc57c0a3430 0x7fc57c0a58f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc594106a90 0x7fc59419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc594101430 0x7fc594197800 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 --2- 172.21.15.64:0/2937307869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc594100a90 0x7fc5941972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:34.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 >> 172.21.15.64:0/2937307869 conn(0x7fc5940fc7d0 msgr2=0x7fc5940fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:34.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 shutdown_connections 2024-09-19T05:14:34.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:34.527+0000 7fc59adc2640 1 -- 172.21.15.64:0/2937307869 wait complete. 2024-09-19T05:14:35.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:35 smithi064 ceph-mon[30960]: pgmap v267: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:35.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:35 smithi103 ceph-mon[34446]: pgmap v267: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:35 smithi174 ceph-mon[34488]: pgmap v267: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:36.239 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:14:36.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:14:36.507 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:36.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 -- 172.21.15.64:0/512514729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 msgr2=0x7f6604110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:36.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 --2- 172.21.15.64:0/512514729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f6604110ca0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009ca0 tx=0x7f65fc02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 -- 172.21.15.64:0/512514729 shutdown_connections 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 --2- 172.21.15.64:0/512514729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f6604110ca0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 --2- 172.21.15.64:0/512514729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66041049a0 0x7f66041089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 --2- 172.21.15.64:0/512514729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6604103ff0 0x7f66041043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 -- 172.21.15.64:0/512514729 >> 172.21.15.64:0/512514729 conn(0x7f66040ffa90 msgr2=0x7f6604101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 -- 172.21.15.64:0/512514729 shutdown_connections 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.966+0000 7f660937e640 1 -- 172.21.15.64:0/512514729 wait complete. 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 Processor -- start 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 -- start start 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6604103ff0 0x7f660419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66041049a0 0x7f660419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f660419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6604114000 con 0x7f6604109120 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6604113e80 con 0x7f6604103ff0 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f660937e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6604114180 con 0x7f66041049a0 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f660419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f660419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35398/0 (socket says 172.21.15.64:35398) 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 -- 172.21.15.64:0/338697975 learned_addr learned my addr 172.21.15.64:0/338697975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f65fbfff640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66041049a0 0x7f660419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f6602ffd640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6604103ff0 0x7f660419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 -- 172.21.15.64:0/338697975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66041049a0 msgr2=0x7f660419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66041049a0 0x7f660419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 -- 172.21.15.64:0/338697975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6604103ff0 msgr2=0x7f660419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6604103ff0 0x7f660419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f66037fe640 1 -- 172.21.15.64:0/338697975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660419f490 con 0x7f6604109120 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.967+0000 7f65fbfff640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66041049a0 0x7f660419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.968+0000 7f66037fe640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f660419edb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009dc0 tx=0x7f65fc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.968+0000 7f6600ff9640 1 -- 172.21.15.64:0/338697975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc003c30 con 0x7f6604109120 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.968+0000 7f6600ff9640 1 -- 172.21.15.64:0/338697975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65fc003dd0 con 0x7f6604109120 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.968+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660419f720 con 0x7f6604109120 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.968+0000 7f6600ff9640 1 -- 172.21.15.64:0/338697975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc030aa0 con 0x7f6604109120 2024-09-19T05:14:36.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.968+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66041a7060 con 0x7f6604109120 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.970+0000 7f6600ff9640 1 -- 172.21.15.64:0/338697975 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f65fc004680 con 0x7f6604109120 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 Processor -- start 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 -- start start 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 0x555fa459af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fa4500280 con 0x555fa459ab60 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fa44a7b10 con 0x555fa45a5d40 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f660b757740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fa44fc240 con 0x555fa459b480 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f6600ff9640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f65f00a33b0 0x7f65f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 0x555fa459af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 0x555fa459af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35404/0 (socket says 172.21.15.64:35404) 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f65fbfff640 1 -- 172.21.15.64:0/3482244956 learned_addr learned my addr 172.21.15.64:0/3482244956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f66037fe640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f65f00a33b0 0x7f65f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f6600ff9640 1 -- 172.21.15.64:0/338697975 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f65fc0f1300 con 0x7f6604109120 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f6602ffd640 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.971+0000 7f65fbfff640 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65fbfff640 1 -- 172.21.15.64:0/3482244956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 msgr2=0x555fa45a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65fbfff640 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65fbfff640 1 -- 172.21.15.64:0/3482244956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 msgr2=0x555fa45a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65fbfff640 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65fbfff640 1 -- 172.21.15.64:0/3482244956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa44b41f0 con 0x555fa459ab60 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f6602ffd640 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f66037fe640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f65f00a33b0 0x7f65f00a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0b9b50 tx=0x7f65fc0c4040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65fbfff640 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 0x555fa459af40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f65ec022d70 tx=0x7f65ec009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c7089010ec8cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65f97fa640 1 -- 172.21.15.64:0/3482244956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec004030 con 0x555fa459ab60 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65f97fa640 1 -- 172.21.15.64:0/3482244956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65ec005020 con 0x555fa459ab60 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.972+0000 7f65f97fa640 1 -- 172.21.15.64:0/3482244956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0049a0 con 0x555fa459ab60 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 -- 172.21.15.64:0/3482244956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 msgr2=0x555fa459af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 0x555fa459af40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f65ec022d70 tx=0x7f65ec009d80 comp rx=0 tx=0).stop 2024-09-19T05:14:36.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 -- 172.21.15.64:0/3482244956 shutdown_connections 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 --2- 172.21.15.64:0/3482244956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459ab60 0x555fa459af40 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 -- 172.21.15.64:0/3482244956 >> 172.21.15.64:0/3482244956 conn(0x555fa45970f0 msgr2=0x555fa45974d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 -- 172.21.15.64:0/3482244956 shutdown_connections 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 -- 172.21.15.64:0/3482244956 wait complete. 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 -- start start 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.973+0000 7f660b757740 1 Processor -- start 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa45a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fa45a88b0 con 0x555fa45a5d40 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fa44baf10 con 0x555fa459ab60 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fa45a8a30 con 0x555fa459b480 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f660b757740 10 client.0 _ll_register_callbacks cb 0x555fa4483a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35420/0 (socket says 172.21.15.64:35420) 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 -- 172.21.15.64:0/1923214901 learned_addr learned my addr 172.21.15.64:0/1923214901 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa45a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 -- 172.21.15.64:0/1923214901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 msgr2=0x555fa45a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 -- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 msgr2=0x555fa45a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:36.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa45a2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 -- 172.21.15.64:0/1923214901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa42716a0 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45a3690 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005660 tx=0x7f65ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1df337742126d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.974+0000 7f65ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1923214901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec037060 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1923214901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f65ec00fe20 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65ec005020 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1923214901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65ec0046c0 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f660b757740 10 client.17403 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fa45b44b0 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1923214901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65ec005020 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1923214901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65ec02f850 con 0x555fa45a5d40 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f660b757740 10 client.17403 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:36.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.975+0000 7f660b757740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f660b757740 20 client.17403 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f660b757740 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa45b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f660b757740 4 client.17403 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa450d7f0 con 0x555fa45b5910 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa45b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa45b7d00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f31f8a956dc769 server_cookie=761bf0d2705115be in_seq=0 out_seq=0 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.976+0000 7f65ea7fc640 10 client.17403 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1923214901 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fa450d7f0 con 0x555fa45b5910 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f65ea7fc640 10 client.17403 handle_command_reply: tid=0 2024-09-19T05:14:36.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 client.17403 shutdown 2024-09-19T05:14:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 msgr2=0x555fa45a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45a3690 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005660 tx=0x7f65ec009260 comp rx=0 tx=0).stop 2024-09-19T05:14:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 shutdown_connections 2024-09-19T05:14:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa45b7d00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f65ea7fc640 0 client.17403 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:36.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45a3690 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 --2- 172.21.15.64:0/1923214901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa45a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 >> 172.21.15.64:0/1923214901 conn(0x555fa45970f0 msgr2=0x555fa459cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 shutdown_connections 2024-09-19T05:14:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 1 -- 172.21.15.64:0/1923214901 wait complete. 2024-09-19T05:14:36.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.977+0000 7f660b757740 20 client.17403 trim_cache size 0 max 16384 2024-09-19T05:14:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 Processor -- start 2024-09-19T05:14:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 -- start start 2024-09-19T05:14:36.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa459d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66041adf70 con 0x555fa45a5d40 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66041ab840 con 0x555fa459ab60 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f660b757740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66041a96b0 con 0x555fa459b480 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45bd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35428/0 (socket says 172.21.15.64:35428) 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 -- 172.21.15.64:0/1482094548 learned_addr learned my addr 172.21.15.64:0/1482094548 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa459d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 -- 172.21.15.64:0/1482094548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 msgr2=0x555fa45a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 -- 172.21.15.64:0/1482094548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 msgr2=0x555fa459d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa459d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.995+0000 7f65fbfff640 1 -- 172.21.15.64:0/1482094548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa45b8070 con 0x555fa45a5d40 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.996+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45bd2a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f65ec001eb0 tx=0x7f65ec013800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d0f4b789a158ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.996+0000 7f65f8ff9640 1 -- 172.21.15.64:0/1482094548 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec037070 con 0x555fa45a5d40 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.996+0000 7f65f8ff9640 1 -- 172.21.15.64:0/1482094548 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65ec009e50 con 0x555fa45a5d40 2024-09-19T05:14:36.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.996+0000 7f65f8ff9640 1 -- 172.21.15.64:0/1482094548 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec040040 con 0x555fa45a5d40 2024-09-19T05:14:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.996+0000 7f660b757740 1 -- 172.21.15.64:0/1482094548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 msgr2=0x555fa45bd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.996+0000 7f660b757740 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45bd2a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f65ec001eb0 tx=0x7f65ec013800 comp rx=0 tx=0).stop 2024-09-19T05:14:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- 172.21.15.64:0/1482094548 shutdown_connections 2024-09-19T05:14:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa45a5d40 0x555fa45bd2a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459b480 0x555fa45a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 --2- 172.21.15.64:0/1482094548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa459ab60 0x555fa459d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- 172.21.15.64:0/1482094548 >> 172.21.15.64:0/1482094548 conn(0x555fa45970f0 msgr2=0x555fa45a13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- 172.21.15.64:0/1482094548 shutdown_connections 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- 172.21.15.64:0/1482094548 wait complete. 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- start start 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 Processor -- start 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459ab60 0x555fa45a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 0x555fa45a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66041a9d00 con 0x555fa459b480 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66041ab170 con 0x555fa45a5d40 2024-09-19T05:14:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66041aa3d0 con 0x555fa459ab60 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f660b757740 10 client.0 _ll_register_callbacks cb 0x7f65ec0135a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f6602ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 0x555fa45a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f6602ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 0x555fa45a9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35442/0 (socket says 172.21.15.64:35442) 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.997+0000 7f6602ffd640 1 -- 172.21.15.64:0/1903889494 learned_addr learned my addr 172.21.15.64:0/1903889494 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459ab60 0x555fa45a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f6602ffd640 1 -- 172.21.15.64:0/1903889494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459ab60 msgr2=0x555fa45a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459ab60 0x555fa45a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f6602ffd640 1 -- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 msgr2=0x555fa45aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f6602ffd640 1 -- 172.21.15.64:0/1903889494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa461f2a0 con 0x555fa459b480 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f6602ffd640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 0x555fa45a9f30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x555fa45a24a0 tx=0x7f65f4012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ae526718903751 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1903889494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f40122d0 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1903889494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f65f4004260 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65ec009e50 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1903889494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65f4004460 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f660b757740 10 client.17409 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fa4620630 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.998+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1903889494 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65ec009e50 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1903889494 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65f4003a80 con 0x555fa459b480 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f660b757740 10 client.17409 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f660b757740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f660b757740 20 client.17409 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f660b757740 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa4622e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f660b757740 4 client.17409 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:14:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555fa4507d00 con 0x555fa45b5910 2024-09-19T05:14:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:36.999+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa4622e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.000+0000 7f65fbfff640 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa4622e70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca51aa9245e76438 server_cookie=df0a471abb5ed414 in_seq=0 out_seq=0 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.000+0000 7f65ea7fc640 10 client.17409 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f65ea7fc640 1 -- 172.21.15.64:0/1903889494 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555fa4507d00 con 0x555fa45b5910 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f65ea7fc640 10 client.17409 handle_command_reply: tid=0 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 client.17409 shutdown 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 msgr2=0x555fa45a9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 0x555fa45a9f30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x555fa45a24a0 tx=0x7f65f4012d50 comp rx=0 tx=0).stop 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 shutdown_connections 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fa45a5d40 0x555fa45aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x555fa45b5910 0x555fa4622e70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa459b480 0x555fa45a9f30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 --2- 172.21.15.64:0/1903889494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa459ab60 0x555fa45a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 >> 172.21.15.64:0/1903889494 conn(0x555fa45970f0 msgr2=0x555fa4596270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f65ea7fc640 0 client.17409 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 shutdown_connections 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 1 -- 172.21.15.64:0/1903889494 wait complete. 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.001+0000 7f660b757740 20 client.17409 trim_cache size 0 max 16384 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:37.009 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:14:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "b0a3c740-f788-492b-a6ff-d26a48e84f45", 2024-09-19T05:14:37.010 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:14:37.010 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:37.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f65f00a33b0 msgr2=0x7f65f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f65f00a33b0 0x7f65f00a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0b9b50 tx=0x7f65fc0c4040 comp rx=0 tx=0).stop 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 msgr2=0x7f660419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f660419edb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009dc0 tx=0x7f65fc032ff0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 shutdown_connections 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f65f00a33b0 0x7f65f00a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6604109120 0x7f660419edb0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66041049a0 0x7f660419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 --2- 172.21.15.64:0/338697975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6604103ff0 0x7f660419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 >> 172.21.15.64:0/338697975 conn(0x7f66040ffa90 msgr2=0x7f6604100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 shutdown_connections 2024-09-19T05:14:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:37.002+0000 7f660937e640 1 -- 172.21.15.64:0/338697975 wait complete. 2024-09-19T05:14:37.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:37 smithi064 ceph-mon[30960]: pgmap v268: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:14:37.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:37 smithi103 ceph-mon[34446]: pgmap v268: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:14:37.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:37 smithi174 ceph-mon[34488]: pgmap v268: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:14:37.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:38 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:14:38.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:38 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:38 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:14:38.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:38 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:38 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:14:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:38 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:38 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:14:38.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:38 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:38.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:38 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:14:38.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:38 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:38.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:38 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:14:38.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:38 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 -- 172.21.15.64:0/494667820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64101de0 msgr2=0x7f2b641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 --2- 172.21.15.64:0/494667820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64101de0 0x7f2b641021c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002a410 comp rx=0 tx=0).stop 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 -- 172.21.15.64:0/494667820 shutdown_connections 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 --2- 172.21.15.64:0/494667820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b64106f10 0x7f2b6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 --2- 172.21.15.64:0/494667820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b64102790 0x7f2b641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 --2- 172.21.15.64:0/494667820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64101de0 0x7f2b641021c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 -- 172.21.15.64:0/494667820 >> 172.21.15.64:0/494667820 conn(0x7f2b640fd8c0 msgr2=0x7f2b640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 -- 172.21.15.64:0/494667820 shutdown_connections 2024-09-19T05:14:38.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.660+0000 7f2b6be33640 1 -- 172.21.15.64:0/494667820 wait complete. 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 Processor -- start 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 -- start start 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b64101de0 0x7f2b641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 0x7f2b64198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b64106f10 0x7f2b6419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64111f10 con 0x7f2b64102790 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64111d90 con 0x7f2b64101de0 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b6be33640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64112090 con 0x7f2b64106f10 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 0x7f2b64198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 0x7f2b64198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35454/0 (socket says 172.21.15.64:35454) 2024-09-19T05:14:38.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b693ce640 1 -- 172.21.15.64:0/2049905170 learned_addr learned my addr 172.21.15.64:0/2049905170 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.661+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b64101de0 0x7f2b641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b6a3d0640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b64106f10 0x7f2b6419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b693ce640 1 -- 172.21.15.64:0/2049905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b64106f10 msgr2=0x7f2b6419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b693ce640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b64106f10 0x7f2b6419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b693ce640 1 -- 172.21.15.64:0/2049905170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b64101de0 msgr2=0x7f2b641984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b693ce640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b64101de0 0x7f2b641984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b693ce640 1 -- 172.21.15.64:0/2049905170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6419d480 con 0x7f2b64102790 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b693ce640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 0x7f2b64198a10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b4a0 tx=0x7f2b5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b5affd640 1 -- 172.21.15.64:0/2049905170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54014070 con 0x7f2b64102790 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6419d770 con 0x7f2b64102790 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b5affd640 1 -- 172.21.15.64:0/2049905170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b54010ac0 con 0x7f2b64102790 2024-09-19T05:14:38.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b5affd640 1 -- 172.21.15.64:0/2049905170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54017c30 con 0x7f2b64102790 2024-09-19T05:14:38.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.662+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a5050 con 0x7f2b64102790 2024-09-19T05:14:38.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.664+0000 7f2b5affd640 1 -- 172.21.15.64:0/2049905170 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2b54017070 con 0x7f2b64102790 2024-09-19T05:14:38.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 Processor -- start 2024-09-19T05:14:38.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 -- start start 2024-09-19T05:14:38.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e4f4a0 0x563e69e4f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e5a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e5a680 0x563e69e5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563e69db52e0 con 0x563e69e5a680 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e69d5cce0 con 0x563e69e4f4a0 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e69db12a0 con 0x563e69e4fdc0 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.665+0000 7f2b5affd640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b500a33b0 0x7f2b500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e5a680 0x563e69e5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b6a3d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e4f4a0 0x563e69e4f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e5a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e5a680 0x563e69e5ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35456/0 (socket says 172.21.15.64:35456) 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b6a3d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e4f4a0 0x563e69e4f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.64:58468/0 (socket says 172.21.15.64:58468) 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b693ce640 1 -- 172.21.15.64:0/3406655592 learned_addr learned my addr 172.21.15.64:0/3406655592 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:38.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b6a3d0640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b500a33b0 0x7f2b500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3406655592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e4f4a0 msgr2=0x563e69e4f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e4f4a0 0x563e69e4f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3406655592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e5a680 msgr2=0x563e69e5ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e5a680 0x563e69e5ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3406655592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e69d41440 con 0x563e69e4fdc0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b5affd640 1 -- 172.21.15.64:0/2049905170 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f2b540cf320 con 0x7f2b64102790 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b6a3d0640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b500a33b0 0x7f2b500a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0070b0 tx=0x7f2b5c0093d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e5a140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002aa00 tx=0x7f2b60005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6c203a89bf81774 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.666+0000 7f2b4ffff640 1 -- 172.21.15.64:0/3406655592 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60040070 con 0x563e69e4fdc0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b4ffff640 1 -- 172.21.15.64:0/3406655592 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b600047a0 con 0x563e69e4fdc0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b4ffff640 1 -- 172.21.15.64:0/3406655592 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60035040 con 0x563e69e4fdc0 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3406655592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 msgr2=0x563e69e5a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e5a140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002aa00 tx=0x7f2b60005e80 comp rx=0 tx=0).stop 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3406655592 shutdown_connections 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e5a680 0x563e69e5ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e5a140 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3406655592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e4f4a0 0x563e69e4f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3406655592 >> 172.21.15.64:0/3406655592 conn(0x563e69e4ba30 msgr2=0x563e69e4be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3406655592 shutdown_connections 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.667+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3406655592 wait complete. 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 -- start start 2024-09-19T05:14:38.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 Processor -- start 2024-09-19T05:14:38.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 0x563e69e59fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e57290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563e69e5d250 con 0x563e69e4f4a0 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e69d70050 con 0x563e69e5a680 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e69e5d3d0 con 0x563e69e4fdc0 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.668+0000 7f2b6e20c740 10 client.0 _ll_register_callbacks cb 0x563e69db0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 0x563e69e59fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 0x563e69e59fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35468/0 (socket says 172.21.15.64:35468) 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3579940555 learned_addr learned my addr 172.21.15.64:0/3579940555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b6a3d0640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b693ce640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e57290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3579940555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 msgr2=0x563e69e57290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e57290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 msgr2=0x563e69e577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 -- 172.21.15.64:0/3579940555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e69b263c0 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 0x563e69e59fe0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2b60006010 tx=0x7f2b60004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d0b393bc3c0d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3579940555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6004d060 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3579940555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f2b60005430 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.669+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b600047a0 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3579940555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b60040070 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 10 client.17421 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e69e68c20 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3579940555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b600047a0 con 0x563e69e4f4a0 2024-09-19T05:14:38.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3579940555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b600314a0 con 0x563e69e4f4a0 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 10 client.17421 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 20 client.17421 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69e69f70 0x563e69e6c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 4 client.17421 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e69dc2850 con 0x563e69e69f70 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.670+0000 7f2b693ce640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69e69f70 0x563e69e6c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.671+0000 7f2b4d7fa640 10 client.17421 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.671+0000 7f2b693ce640 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69e69f70 0x563e69e6c360 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9557c594157c3d83 server_cookie=b7e78d552644650c in_seq=0 out_seq=0 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.671+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3579940555 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e69dc2850 con 0x563e69e69f70 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b4d7fa640 10 client.17421 handle_command_reply: tid=0 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 client.17421 shutdown 2024-09-19T05:14:38.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 msgr2=0x563e69e59fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 0x563e69e59fe0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f2b60006010 tx=0x7f2b60004c00 comp rx=0 tx=0).stop 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 shutdown_connections 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4fdc0 0x563e69e57290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69e69f70 0x563e69e6c360 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3579940555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4f4a0 0x563e69e59fe0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b4d7fa640 0 client.17421 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 >> 172.21.15.64:0/3579940555 conn(0x563e69e4ba30 msgr2=0x563e69e51710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 shutdown_connections 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3579940555 wait complete. 2024-09-19T05:14:38.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.672+0000 7f2b6e20c740 20 client.17421 trim_cache size 0 max 16384 2024-09-19T05:14:38.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 Processor -- start 2024-09-19T05:14:38.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 -- start start 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e71c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e72140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ae0b0 con 0x563e69e4fdc0 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ab980 con 0x563e69e5a680 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.689+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641a97f0 con 0x563e69e4f4a0 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e71c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e71c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35480/0 (socket says 172.21.15.64:35480) 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 -- 172.21.15.64:0/2599751904 learned_addr learned my addr 172.21.15.64:0/2599751904 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 -- 172.21.15.64:0/2599751904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 msgr2=0x563e69e664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b6a3d0640 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e72140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 -- 172.21.15.64:0/2599751904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 msgr2=0x563e69e72140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e72140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 -- 172.21.15.64:0/2599751904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e69e57b60 con 0x563e69e4fdc0 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b693ce640 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e71c00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2b54017480 tx=0x7f2b540bb910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df0133ee1412a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b4f7fe640 1 -- 172.21.15.64:0/2599751904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54082070 con 0x563e69e4fdc0 2024-09-19T05:14:38.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b4f7fe640 1 -- 172.21.15.64:0/2599751904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b5409b6e0 con 0x563e69e4fdc0 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.690+0000 7f2b4f7fe640 1 -- 172.21.15.64:0/2599751904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401e040 con 0x563e69e4fdc0 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 -- 172.21.15.64:0/2599751904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 msgr2=0x563e69e71c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e71c00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2b54017480 tx=0x7f2b540bb910 comp rx=0 tx=0).stop 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 -- 172.21.15.64:0/2599751904 shutdown_connections 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e72140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e71c00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/2599751904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 -- 172.21.15.64:0/2599751904 >> 172.21.15.64:0/2599751904 conn(0x563e69e4ba30 msgr2=0x563e69e56380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 -- 172.21.15.64:0/2599751904 shutdown_connections 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 -- 172.21.15.64:0/2599751904 wait complete. 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 -- start start 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.691+0000 7f2b6e20c740 1 Processor -- start 2024-09-19T05:14:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e5fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641a9e40 con 0x563e69e4fdc0 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ab2b0 con 0x563e69e5a680 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641aa510 con 0x563e69e4f4a0 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6e20c740 10 client.0 _ll_register_callbacks cb 0x7f2b54097260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e62c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35482/0 (socket says 172.21.15.64:35482) 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 -- 172.21.15.64:0/3206231472 learned_addr learned my addr 172.21.15.64:0/3206231472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b6a3d0640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e5fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 -- 172.21.15.64:0/3206231472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 msgr2=0x563e69e626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 -- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 msgr2=0x563e69e5fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e5fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.692+0000 7f2b693ce640 1 -- 172.21.15.64:0/3206231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e69e60350 con 0x563e69e4fdc0 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b693ce640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e62c20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009420 tx=0x7f2b540a4350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c98cb03140eef81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3206231472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54082040 con 0x563e69e4fdc0 2024-09-19T05:14:38.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5409b6e0 con 0x563e69e4fdc0 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3206231472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f2b54017dd0 con 0x563e69e4fdc0 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3206231472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b5401e040 con 0x563e69e4fdc0 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b6e20c740 10 client.17427 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e69ed45e0 con 0x563e69e4fdc0 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3206231472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b5409b6e0 con 0x563e69e4fdc0 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3206231472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b540a3ec0 con 0x563e69e4fdc0 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b6e20c740 10 client.17427 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b6e20c740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.693+0000 7f2b6e20c740 20 client.17427 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69ed4c20 0x563e69ed7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b6e20c740 4 client.17427 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e69dbcd60 con 0x563e69ed4c20 2024-09-19T05:14:38.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69ed4c20 0x563e69ed7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b4d7fa640 10 client.17427 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:38.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b69bcf640 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69ed4c20 0x563e69ed7040 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbb8ae85c4e822d server_cookie=dc2f023db3bfd244 in_seq=0 out_seq=0 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b4d7fa640 1 -- 172.21.15.64:0/3206231472 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563e69dbcd60 con 0x563e69ed4c20 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b4d7fa640 10 client.17427 handle_command_reply: tid=0 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.694+0000 7f2b6e20c740 1 client.17427 shutdown 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 msgr2=0x563e69e62c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e62c20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009420 tx=0x7f2b540a4350 comp rx=0 tx=0).stop 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 shutdown_connections 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e69e5a680 0x563e69e5fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563e69ed4c20 0x563e69ed7040 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e69e4fdc0 0x563e69e62c20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 --2- 172.21.15.64:0/3206231472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e69e4f4a0 0x563e69e626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b4d7fa640 0 client.17427 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:38.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 >> 172.21.15.64:0/3206231472 conn(0x563e69e4ba30 msgr2=0x563e69e4a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 shutdown_connections 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 1 -- 172.21.15.64:0/3206231472 wait complete. 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6e20c740 20 client.17427 trim_cache size 0 max 16384 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:14:38.704 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b500a33b0 msgr2=0x7f2b500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6be33640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b500a33b0 0x7f2b500a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0070b0 tx=0x7f2b5c0093d0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 msgr2=0x7f2b64198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.695+0000 7f2b6be33640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 0x7f2b64198a10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b4a0 tx=0x7f2b5400b960 comp rx=0 tx=0).stop 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 shutdown_connections 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2b500a33b0 0x7f2b500a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b64106f10 0x7f2b6419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b64102790 0x7f2b64198a10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 --2- 172.21.15.64:0/2049905170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b64101de0 0x7f2b641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 >> 172.21.15.64:0/2049905170 conn(0x7f2b640fd8c0 msgr2=0x7f2b640fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 shutdown_connections 2024-09-19T05:14:38.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:38.696+0000 7f2b6be33640 1 -- 172.21.15.64:0/2049905170 wait complete. 2024-09-19T05:14:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:39 smithi064 ceph-mon[30960]: pgmap v269: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:39.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:39 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:14:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:39 smithi103 ceph-mon[34446]: pgmap v269: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:39.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:39 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:14:39.516 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:14:39.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:14:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:39 smithi174 ceph-mon[34488]: pgmap v269: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:14:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:39 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:14:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.245+0000 7f788ba55640 1 -- 172.21.15.64:0/494893106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884103660 msgr2=0x7f788410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.245+0000 7f788ba55640 1 --2- 172.21.15.64:0/494893106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884103660 0x7f788410bb50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7878009900 tx=0x7f787802a480 comp rx=0 tx=0).stop 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 -- 172.21.15.64:0/494893106 shutdown_connections 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 --2- 172.21.15.64:0/494893106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f788410c170 0x7f788410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 --2- 172.21.15.64:0/494893106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884103660 0x7f788410bb50 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 --2- 172.21.15.64:0/494893106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884102d40 0x7f7884103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 -- 172.21.15.64:0/494893106 >> 172.21.15.64:0/494893106 conn(0x7f78840fc7a0 msgr2=0x7f78840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:40.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 -- 172.21.15.64:0/494893106 shutdown_connections 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 -- 172.21.15.64:0/494893106 wait complete. 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 Processor -- start 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.246+0000 7f788ba55640 1 -- start start 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 0x7f78841009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884103660 0x7f7884100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f788410c170 0x7f7884101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7884111e60 con 0x7f7884102d40 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7884111ce0 con 0x7f788410c170 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7884111fe0 con 0x7f7884103660 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 0x7f78841009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 0x7f78841009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35502/0 (socket says 172.21.15.64:35502) 2024-09-19T05:14:40.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 -- 172.21.15.64:0/644288233 learned_addr learned my addr 172.21.15.64:0/644288233 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f7889ff2640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f788410c170 0x7f7884101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f7888ff0640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884103660 0x7f7884100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 -- 172.21.15.64:0/644288233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884103660 msgr2=0x7f7884100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884103660 0x7f7884100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 -- 172.21.15.64:0/644288233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f788410c170 msgr2=0x7f7884101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f788410c170 0x7f7884101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 -- 172.21.15.64:0/644288233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78841a68c0 con 0x7f7884102d40 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f7888ff0640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884103660 0x7f7884100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78897f1640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 0x7f78841009a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f787400ba50 tx=0x7f787400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874019070 con 0x7f7884102d40 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f787400cbe0 con 0x7f7884102d40 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841a6bb0 con 0x7f7884102d40 2024-09-19T05:14:40.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740075d0 con 0x7f7884102d40 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.247+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78841a70f0 con 0x7f7884102d40 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.249+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7884103120 con 0x7f7884102d40 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.250+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f787401c070 con 0x7f7884102d40 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.250+0000 7f78827fc640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f786c0a33b0 0x7f786c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.250+0000 7f7888ff0640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f786c0a33b0 0x7f786c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.251+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f78740cf1f0 con 0x7f7884102d40 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.251+0000 7f7888ff0640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f786c0a33b0 0x7f786c0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f7878031400 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:40.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.255+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f787409b660 con 0x7f7884102d40 2024-09-19T05:14:40.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.471+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f788402d7c0 con 0x7f7884102d40 2024-09-19T05:14:40.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.472+0000 7f78827fc640 1 -- 172.21.15.64:0/644288233 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f787409f960 con 0x7f7884102d40 2024-09-19T05:14:40.473 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:14:40.474 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:14:40.474 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:14:40.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.475+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f786c0a33b0 msgr2=0x7f786c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.475+0000 7f788ba55640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f786c0a33b0 0x7f786c0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f7878031400 comp rx=0 tx=0).stop 2024-09-19T05:14:40.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.475+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 msgr2=0x7f78841009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.475+0000 7f788ba55640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 0x7f78841009a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f787400ba50 tx=0x7f787400bf10 comp rx=0 tx=0).stop 2024-09-19T05:14:40.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.475+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 shutdown_connections 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f786c0a33b0 0x7f786c0a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f788410c170 0x7f7884101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7884103660 0x7f7884100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 --2- 172.21.15.64:0/644288233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7884102d40 0x7f78841009a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 >> 172.21.15.64:0/644288233 conn(0x7f78840fc7a0 msgr2=0x7f78840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 shutdown_connections 2024-09-19T05:14:40.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:40.476+0000 7f788ba55640 1 -- 172.21.15.64:0/644288233 wait complete. 2024-09-19T05:14:41.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:14:41.352 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:41.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:41 smithi064 ceph-mon[30960]: pgmap v270: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:41.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:41 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/644288233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:41.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:41 smithi103 ceph-mon[34446]: pgmap v270: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:41.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:41 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/644288233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:41.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:41 smithi174 ceph-mon[34488]: pgmap v270: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:41.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:41 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/644288233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 -- 172.21.15.64:0/1666711068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8106a80 msgr2=0x7f10a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 --2- 172.21.15.64:0/1666711068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8106a80 0x7f10a8106e60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009930 tx=0x7f10a402a420 comp rx=0 tx=0).stop 2024-09-19T05:14:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 -- 172.21.15.64:0/1666711068 shutdown_connections 2024-09-19T05:14:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 --2- 172.21.15.64:0/1666711068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8101420 0x7f10a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 --2- 172.21.15.64:0/1666711068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8100a80 0x7f10a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 --2- 172.21.15.64:0/1666711068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8106a80 0x7f10a8106e60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 -- 172.21.15.64:0/1666711068 >> 172.21.15.64:0/1666711068 conn(0x7f10a80fc7a0 msgr2=0x7f10a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.938+0000 7f10afe2f640 1 -- 172.21.15.64:0/1666711068 shutdown_connections 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 -- 172.21.15.64:0/1666711068 wait complete. 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 Processor -- start 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 -- start start 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8100a80 0x7f10a80fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 0x7f10a80fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8106a80 0x7f10a80fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8110cf0 con 0x7f10a8101420 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8110b70 con 0x7f10a8100a80 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.939+0000 7f10afe2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8110e70 con 0x7f10a8106a80 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 0x7f10a80fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 0x7f10a80fd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35530/0 (socket says 172.21.15.64:35530) 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 -- 172.21.15.64:0/2690648154 learned_addr learned my addr 172.21.15.64:0/2690648154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10adbcb640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8100a80 0x7f10a80fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8106a80 0x7f10a80fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 -- 172.21.15.64:0/2690648154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8106a80 msgr2=0x7f10a80fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8106a80 0x7f10a80fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 -- 172.21.15.64:0/2690648154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8100a80 msgr2=0x7f10a80fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8100a80 0x7f10a80fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 -- 172.21.15.64:0/2690648154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a81a6760 con 0x7f10a8101420 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10adbcb640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8100a80 0x7f10a80fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8106a80 0x7f10a80fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:41.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 0x7f10a80fd370 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f109800c920 tx=0x7f109800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f109effd640 1 -- 172.21.15.64:0/2690648154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098007c70 con 0x7f10a8101420 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f109effd640 1 -- 172.21.15.64:0/2690648154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1098007e10 con 0x7f10a8101420 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81a6a50 con 0x7f10a8101420 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f109effd640 1 -- 172.21.15.64:0/2690648154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098018820 con 0x7f10a8101420 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.940+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81a6f90 con 0x7f10a8101420 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.942+0000 7f109effd640 1 -- 172.21.15.64:0/2690648154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f10980189c0 con 0x7f10a8101420 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 Processor -- start 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 -- start start 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562fbfd902f0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562fbfd37cf0 con 0x562fbfe35370 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10b2208740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562fbfd8c2b0 con 0x562fbfe2bb40 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10adbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10adbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe2aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35534/0 (socket says 172.21.15.64:35534) 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10adbcb640 1 -- 172.21.15.64:0/1845623369 learned_addr learned my addr 172.21.15.64:0/1845623369 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.943+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f109effd640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f10940a33b0 0x7f10940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10adbcb640 1 -- 172.21.15.64:0/1845623369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 msgr2=0x562fbfe2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10adbcb640 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10adbcb640 1 -- 172.21.15.64:0/1845623369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 msgr2=0x562fbfe37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10adbcb640 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10adbcb640 1 -- 172.21.15.64:0/1845623369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fbfd1c3a0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f10940a33b0 0x7f10940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f109effd640 1 -- 172.21.15.64:0/2690648154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f10980cffd0 con 0x7f10a8101420 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10adbcb640 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe2aa10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f10a4031440 tx=0x7f10a4032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c673f16eb215006 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f1093fff640 1 -- 172.21.15.64:0/1845623369 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a403f070 con 0x562fbfe2a5d0 2024-09-19T05:14:41.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f1093fff640 1 -- 172.21.15.64:0/1845623369 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a40316a0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f1093fff640 1 -- 172.21.15.64:0/1845623369 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4035070 con 0x562fbfe2a5d0 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.944+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f10940a33b0 0x7f10940a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005fd0 tx=0x7f10a0004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 -- 172.21.15.64:0/1845623369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 msgr2=0x562fbfe2aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe2aa10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f10a4031440 tx=0x7f10a4032dc0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 -- 172.21.15.64:0/1845623369 shutdown_connections 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe2aa10 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 --2- 172.21.15.64:0/1845623369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 -- 172.21.15.64:0/1845623369 >> 172.21.15.64:0/1845623369 conn(0x562fbfdf3310 msgr2=0x562fbfe26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 -- 172.21.15.64:0/1845623369 shutdown_connections 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 -- 172.21.15.64:0/1845623369 wait complete. 2024-09-19T05:14:41.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.945+0000 7f10b2208740 1 -- start start 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 Processor -- start 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe2a5d0 0x562fbfe34d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 0x562fbfe31fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe35370 0x562fbfe325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562fbfe38010 con 0x562fbfe2bb40 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562fbfd4b060 con 0x562fbfe2a5d0 2024-09-19T05:14:41.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562fbfe38190 con 0x562fbfe35370 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10b2208740 10 client.0 _ll_register_callbacks cb 0x562fbfa9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 0x562fbfe31fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 0x562fbfe31fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35544/0 (socket says 172.21.15.64:35544) 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 -- 172.21.15.64:0/4021629260 learned_addr learned my addr 172.21.15.64:0/4021629260 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe35370 0x562fbfe325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10adbcb640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe2a5d0 0x562fbfe34d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 -- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe35370 msgr2=0x562fbfe325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe35370 0x562fbfe325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 -- 172.21.15.64:0/4021629260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe2a5d0 msgr2=0x562fbfe34d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe2a5d0 0x562fbfe34d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10ad3ca640 1 -- 172.21.15.64:0/4021629260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fbfb013c0 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.946+0000 7f10adbcb640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe2a5d0 0x562fbfe34d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 0x562fbfe31fe0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f10980cd6f0 tx=0x7f109809bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03388e0adf42a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10917fa640 1 -- 172.21.15.64:0/4021629260 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980bae60 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10917fa640 1 -- 172.21.15.64:0/4021629260 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f10980bc0b0 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a40316a0 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10917fa640 1 -- 172.21.15.64:0/4021629260 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10980cdb10 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10b2208740 10 client.17445 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562fbfe43aa0 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10917fa640 1 -- 172.21.15.64:0/4021629260 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a40316a0 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10917fa640 1 -- 172.21.15.64:0/4021629260 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10980a4a00 con 0x562fbfe2bb40 2024-09-19T05:14:41.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10b2208740 10 client.17445 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:41.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.947+0000 7f10b2208740 10 client.17445 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:14:41.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10b2208740 20 client.17445 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:41.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10b2208740 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfe45000 0x562fbfe473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10b2208740 4 client.17445 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:41.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fbfd9d860 con 0x562fbfe45000 2024-09-19T05:14:41.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10adbcb640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfe45000 0x562fbfe473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10917fa640 10 client.17445 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:41.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.948+0000 7f10adbcb640 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfe45000 0x562fbfe473f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1514009854168cdb server_cookie=1ad6009cc42ce190 in_seq=0 out_seq=0 2024-09-19T05:14:41.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10917fa640 1 -- 172.21.15.64:0/4021629260 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fbfd9d860 con 0x562fbfe45000 2024-09-19T05:14:41.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10917fa640 10 client.17445 handle_command_reply: tid=0 2024-09-19T05:14:41.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 client.17445 shutdown 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 msgr2=0x562fbfe31fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 0x562fbfe31fe0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f10980cd6f0 tx=0x7f109809bc40 comp rx=0 tx=0).stop 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 shutdown_connections 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfe45000 0x562fbfe473f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe35370 0x562fbfe325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2bb40 0x562fbfe31fe0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10917fa640 0 client.17445 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 --2- 172.21.15.64:0/4021629260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe2a5d0 0x562fbfe34d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 >> 172.21.15.64:0/4021629260 conn(0x562fbfdf3310 msgr2=0x562fbfe2c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:41.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 shutdown_connections 2024-09-19T05:14:41.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.949+0000 7f10b2208740 1 -- 172.21.15.64:0/4021629260 wait complete. 2024-09-19T05:14:41.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.950+0000 7f10b2208740 20 client.17445 trim_cache size 0 max 16384 2024-09-19T05:14:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.962+0000 7f10b2208740 1 Processor -- start 2024-09-19T05:14:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.962+0000 7f10b2208740 1 -- start start 2024-09-19T05:14:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10b2208740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10b2208740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe33f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10b2208740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10b2208740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81b2290 con 0x562fbfe2a5d0 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10b2208740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81afb60 con 0x562fbfe35370 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10b2208740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81ad9d0 con 0x562fbfe2bb40 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35552/0 (socket says 172.21.15.64:35552) 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 -- 172.21.15.64:0/1517047899 learned_addr learned my addr 172.21.15.64:0/1517047899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe33f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 -- 172.21.15.64:0/1517047899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 msgr2=0x562fbfe33f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe33f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 -- 172.21.15.64:0/1517047899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 msgr2=0x562fbfe4cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe4cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 -- 172.21.15.64:0/1517047899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fbfe30530 con 0x562fbfe2a5d0 2024-09-19T05:14:41.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe4cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10adbcb640 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe412e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f10a402a940 tx=0x7f10a402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4456429e6507685d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10937fe640 1 -- 172.21.15.64:0/1517047899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4052070 con 0x562fbfe2a5d0 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10937fe640 1 -- 172.21.15.64:0/1517047899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a40054f0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.963+0000 7f10937fe640 1 -- 172.21.15.64:0/1517047899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a403f040 con 0x562fbfe2a5d0 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 -- 172.21.15.64:0/1517047899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 msgr2=0x562fbfe412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe412e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f10a402a940 tx=0x7f10a402a420 comp rx=0 tx=0).stop 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 -- 172.21.15.64:0/1517047899 shutdown_connections 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe4cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe33f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 --2- 172.21.15.64:0/1517047899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe412e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 -- 172.21.15.64:0/1517047899 >> 172.21.15.64:0/1517047899 conn(0x562fbfdf3310 msgr2=0x562fbfe33840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 -- 172.21.15.64:0/1517047899 shutdown_connections 2024-09-19T05:14:41.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 -- 172.21.15.64:0/1517047899 wait complete. 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 -- start start 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.964+0000 7f10b2208740 1 Processor -- start 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe3db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe2f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81ae020 con 0x562fbfe2a5d0 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81af490 con 0x562fbfe35370 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81ae6f0 con 0x562fbfe2bb40 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10b2208740 10 client.0 _ll_register_callbacks cb 0x7f109800c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe3db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe3db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35554/0 (socket says 172.21.15.64:35554) 2024-09-19T05:14:41.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 -- 172.21.15.64:0/3717652006 learned_addr learned my addr 172.21.15.64:0/3717652006 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 -- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 msgr2=0x562fbfe2ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 -- 172.21.15.64:0/3717652006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 msgr2=0x562fbfe2f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe2f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 -- 172.21.15.64:0/3717652006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fbfe2f960 con 0x562fbfe2a5d0 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10ae3cc640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe2f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10adbcb640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe3db40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f10a4005270 tx=0x7f10a40052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d6d46a48cca1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.965+0000 7f10917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:41.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10917fa640 1 -- 172.21.15.64:0/3717652006 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4052040 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10917fa640 1 -- 172.21.15.64:0/3717652006 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f10a4031cf0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a40054f0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10917fa640 1 -- 172.21.15.64:0/3717652006 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10a403f040 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 10 client.17451 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562fbfeafc20 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10917fa640 1 -- 172.21.15.64:0/3717652006 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a40054f0 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10917fa640 1 -- 172.21.15.64:0/3717652006 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10a404d450 con 0x562fbfe2a5d0 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 10 client.17451 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 10 client.17451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:14:41.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 20 client.17451 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfeafea0 0x562fbfeb2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 4 client.17451 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.966+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fbfd97d70 con 0x562fbfeafea0 2024-09-19T05:14:41.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfeafea0 0x562fbfeb2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10917fa640 10 client.17451 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10ad3ca640 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfeafea0 0x562fbfeb2290 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94e3b7de90ec6615 server_cookie=9cadda24c0e469f4 in_seq=0 out_seq=0 2024-09-19T05:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10917fa640 1 -- 172.21.15.64:0/3717652006 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fbfd97d70 con 0x562fbfeafea0 2024-09-19T05:14:41.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10917fa640 10 client.17451 handle_command_reply: tid=0 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10b2208740 1 client.17451 shutdown 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 msgr2=0x562fbfe3db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.967+0000 7f10b2208740 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe3db40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f10a4005270 tx=0x7f10a40052a0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 shutdown_connections 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x562fbfeafea0 0x562fbfeb2290 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562fbfe35370 0x562fbfe2f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10917fa640 0 client.17451 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fbfe2bb40 0x562fbfe2ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 --2- 172.21.15.64:0/3717652006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fbfe2a5d0 0x562fbfe3db40 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 >> 172.21.15.64:0/3717652006 conn(0x562fbfdf3310 msgr2=0x562fbfe2c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 shutdown_connections 2024-09-19T05:14:41.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 1 -- 172.21.15.64:0/3717652006 wait complete. 2024-09-19T05:14:41.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10b2208740 20 client.17451 trim_cache size 0 max 16384 2024-09-19T05:14:41.977 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f10940a33b0 msgr2=0x7f10940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f10940a33b0 0x7f10940a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005fd0 tx=0x7f10a0004a40 comp rx=0 tx=0).stop 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 msgr2=0x7f10a80fd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 0x7f10a80fd370 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f109800c920 tx=0x7f109800cde0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 shutdown_connections 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f10940a33b0 0x7f10940a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a8106a80 0x7f10a80fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10a8101420 0x7f10a80fd370 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 --2- 172.21.15.64:0/2690648154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a8100a80 0x7f10a80fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 >> 172.21.15.64:0/2690648154 conn(0x7f10a80fc7a0 msgr2=0x7f10a806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 shutdown_connections 2024-09-19T05:14:41.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:41.968+0000 7f10afe2f640 1 -- 172.21.15.64:0/2690648154 wait complete. 2024-09-19T05:14:42.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:14:42.878 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:43.252 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:43 smithi064 ceph-mon[30960]: pgmap v271: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:43.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 -- 172.21.15.64:0/2254462415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 msgr2=0x7f2c601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 --2- 172.21.15.64:0/2254462415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c601067e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009900 tx=0x7f2c5402a410 comp rx=0 tx=0).stop 2024-09-19T05:14:43.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 -- 172.21.15.64:0/2254462415 shutdown_connections 2024-09-19T05:14:43.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 --2- 172.21.15.64:0/2254462415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c60106f10 0x7f2c6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 --2- 172.21.15.64:0/2254462415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c601067e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 --2- 172.21.15.64:0/2254462415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60101de0 0x7f2c601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 -- 172.21.15.64:0/2254462415 >> 172.21.15.64:0/2254462415 conn(0x7f2c600fd8c0 msgr2=0x7f2c600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:43.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 -- 172.21.15.64:0/2254462415 shutdown_connections 2024-09-19T05:14:43.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 -- 172.21.15.64:0/2254462415 wait complete. 2024-09-19T05:14:43.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 Processor -- start 2024-09-19T05:14:43.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 -- start start 2024-09-19T05:14:43.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.329+0000 7f2c68088640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c60101de0 0x7f2c600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60106f10 0x7f2c60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60111e40 con 0x7f2c60102790 2024-09-19T05:14:43.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60111cc0 con 0x7f2c60101de0 2024-09-19T05:14:43.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60111fc0 con 0x7f2c60106f10 2024-09-19T05:14:43.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35574/0 (socket says 172.21.15.64:35574) 2024-09-19T05:14:43.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 -- 172.21.15.64:0/2519922724 learned_addr learned my addr 172.21.15.64:0/2519922724 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:43.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c66625640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60106f10 0x7f2c60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65e24640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c60101de0 0x7f2c600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 -- 172.21.15.64:0/2519922724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60106f10 msgr2=0x7f2c60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60106f10 0x7f2c60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 -- 172.21.15.64:0/2519922724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c60101de0 msgr2=0x7f2c600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c60101de0 0x7f2c600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 -- 172.21.15.64:0/2519922724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c601a68b0 con 0x7f2c60102790 2024-09-19T05:14:43.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c66625640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60106f10 0x7f2c60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:43.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c65623640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c60074a30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2c54006010 tx=0x7f2c54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:43.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c52ffd640 1 -- 172.21.15.64:0/2519922724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5403f030 con 0x7f2c60102790 2024-09-19T05:14:43.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c601a6b40 con 0x7f2c60102790 2024-09-19T05:14:43.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c52ffd640 1 -- 172.21.15.64:0/2519922724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c540035f0 con 0x7f2c60102790 2024-09-19T05:14:43.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c52ffd640 1 -- 172.21.15.64:0/2519922724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54035070 con 0x7f2c60102790 2024-09-19T05:14:43.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.330+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c601a7020 con 0x7f2c60102790 2024-09-19T05:14:43.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c52ffd640 1 -- 172.21.15.64:0/2519922724 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2c54003b10 con 0x7f2c60102790 2024-09-19T05:14:43.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 Processor -- start 2024-09-19T05:14:43.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 -- start start 2024-09-19T05:14:43.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c130090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c130a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 0x556b4c13d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b4c0962f0 con 0x556b4c13b370 2024-09-19T05:14:43.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556b4c03dcf0 con 0x556b4c131b40 2024-09-19T05:14:43.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.333+0000 7f2c6a461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b4c0922b0 con 0x556b4c1305d0 2024-09-19T05:14:43.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 0x556b4c13d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c52ffd640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2c4c0a33b0 0x7f2c4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 0x556b4c13d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35576/0 (socket says 172.21.15.64:35576) 2024-09-19T05:14:43.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 -- 172.21.15.64:0/3917873599 learned_addr learned my addr 172.21.15.64:0/3917873599 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:43.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65e24640 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c130a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c66625640 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c66625640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2c4c0a33b0 0x7f2c4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c52ffd640 1 -- 172.21.15.64:0/2519922724 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f2c540f09e0 con 0x7f2c60102790 2024-09-19T05:14:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 -- 172.21.15.64:0/3917873599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 msgr2=0x556b4c130a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c130a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 -- 172.21.15.64:0/3917873599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 msgr2=0x556b4c130090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c130090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 -- 172.21.15.64:0/3917873599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4c0223a0 con 0x556b4c13b370 2024-09-19T05:14:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65e24640 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c130a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c65623640 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 0x556b4c13d760 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2c540edfe0 tx=0x7f2c5402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8a975b2187671b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:43.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.334+0000 7f2c66625640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2c4c0a33b0 0x7f2c4c0a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c58013bc0 tx=0x7f2c5800a370 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c4bfff640 1 -- 172.21.15.64:0/3917873599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54049030 con 0x556b4c13b370 2024-09-19T05:14:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c4bfff640 1 -- 172.21.15.64:0/3917873599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c540bcc30 con 0x556b4c13b370 2024-09-19T05:14:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c4bfff640 1 -- 172.21.15.64:0/3917873599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c540ee320 con 0x556b4c13b370 2024-09-19T05:14:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 -- 172.21.15.64:0/3917873599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 msgr2=0x556b4c13d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 0x556b4c13d760 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2c540edfe0 tx=0x7f2c5402a410 comp rx=0 tx=0).stop 2024-09-19T05:14:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 -- 172.21.15.64:0/3917873599 shutdown_connections 2024-09-19T05:14:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c13b370 0x556b4c13d760 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c130a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3917873599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 -- 172.21.15.64:0/3917873599 >> 172.21.15.64:0/3917873599 conn(0x556b4c0f9310 msgr2=0x556b4c12cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:43.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 -- 172.21.15.64:0/3917873599 shutdown_connections 2024-09-19T05:14:43.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.335+0000 7f2c6a461740 1 -- 172.21.15.64:0/3917873599 wait complete. 2024-09-19T05:14:43.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 -- start start 2024-09-19T05:14:43.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 Processor -- start 2024-09-19T05:14:43.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c13adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c131b40 0x556b4c1380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c1387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b4c13de30 con 0x556b4c1305d0 2024-09-19T05:14:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556b4c051060 con 0x556b4c13b370 2024-09-19T05:14:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b4c13dfb0 con 0x556b4c131b40 2024-09-19T05:14:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c6a461740 10 client.0 _ll_register_callbacks cb 0x556b4bda48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:43.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c65e24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c13adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c65e24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c13adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35588/0 (socket says 172.21.15.64:35588) 2024-09-19T05:14:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c65e24640 1 -- 172.21.15.64:0/1771659935 learned_addr learned my addr 172.21.15.64:0/1771659935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.336+0000 7f2c66625640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c1387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65623640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c131b40 0x556b4c1380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65e24640 1 -- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c131b40 msgr2=0x556b4c1380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65e24640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c131b40 0x556b4c1380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65e24640 1 -- 172.21.15.64:0/1771659935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 msgr2=0x556b4c1387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65e24640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c1387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65e24640 1 -- 172.21.15.64:0/1771659935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4be073c0 con 0x556b4c1305d0 2024-09-19T05:14:43.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c65e24640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c13adf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00d470 tx=0x7f2c5c00d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e06ad47b89b6645 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c497fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c497fa640 1 -- 172.21.15.64:0/1771659935 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c015750 con 0x556b4c1305d0 2024-09-19T05:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c497fa640 1 -- 172.21.15.64:0/1771659935 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f2c5c0158f0 con 0x556b4c1305d0 2024-09-19T05:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c540bcc30 con 0x556b4c1305d0 2024-09-19T05:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c497fa640 1 -- 172.21.15.64:0/1771659935 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c5c024050 con 0x556b4c1305d0 2024-09-19T05:14:43.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c6a461740 10 client.17463 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.337+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b4c1498b0 con 0x556b4c1305d0 2024-09-19T05:14:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c497fa640 1 -- 172.21.15.64:0/1771659935 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c540bcc30 con 0x556b4c1305d0 2024-09-19T05:14:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c497fa640 1 -- 172.21.15.64:0/1771659935 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c5c01f030 con 0x556b4c1305d0 2024-09-19T05:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c6a461740 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c6a461740 10 client.17463 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c6a461740 20 client.17463 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c6a461740 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c14adc0 0x556b4c14d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c6a461740 4 client.17463 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b4c0a3860 con 0x556b4c14adc0 2024-09-19T05:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c65623640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c14adc0 0x556b4c14d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c497fa640 10 client.17463 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.338+0000 7f2c65623640 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c14adc0 0x556b4c14d230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea97628d47346686 server_cookie=8df30e2c68853b83 in_seq=0 out_seq=0 2024-09-19T05:14:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c497fa640 1 -- 172.21.15.64:0/1771659935 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b4c0a3860 con 0x556b4c14adc0 2024-09-19T05:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c497fa640 10 client.17463 handle_command_reply: tid=0 2024-09-19T05:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 client.17463 shutdown 2024-09-19T05:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 msgr2=0x556b4c13adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c13adf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00d470 tx=0x7f2c5c00d870 comp rx=0 tx=0).stop 2024-09-19T05:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 shutdown_connections 2024-09-19T05:14:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c1387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c131b40 0x556b4c1380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c14adc0 0x556b4c14d230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 --2- 172.21.15.64:0/1771659935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c13adf0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c497fa640 0 client.17463 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.339+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 >> 172.21.15.64:0/1771659935 conn(0x556b4c0f9310 msgr2=0x556b4c132990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.340+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 shutdown_connections 2024-09-19T05:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.340+0000 7f2c6a461740 1 -- 172.21.15.64:0/1771659935 wait complete. 2024-09-19T05:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.340+0000 7f2c6a461740 20 client.17463 trim_cache size 0 max 16384 2024-09-19T05:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.352+0000 7f2c6a461740 1 Processor -- start 2024-09-19T05:14:43.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 -- start start 2024-09-19T05:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c137960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c13b370 0x556b4c136f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601b2390 con 0x556b4c1305d0 2024-09-19T05:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601afc60 con 0x556b4c131b40 2024-09-19T05:14:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c6a461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601adad0 con 0x556b4c13b370 2024-09-19T05:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35596/0 (socket says 172.21.15.64:35596) 2024-09-19T05:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 -- 172.21.15.64:0/3686089909 learned_addr learned my addr 172.21.15.64:0/3686089909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c66625640 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c13b370 0x556b4c136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65623640 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 -- 172.21.15.64:0/3686089909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c13b370 msgr2=0x556b4c136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c13b370 0x556b4c136f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 -- 172.21.15.64:0/3686089909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 msgr2=0x556b4c146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 -- 172.21.15.64:0/3686089909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4c14d8b0 con 0x556b4c1305d0 2024-09-19T05:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c65e24640 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c137960 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00b2d0 tx=0x7f2c5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49bc010c851b2e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.353+0000 7f2c4b7fe640 1 -- 172.21.15.64:0/3686089909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c02e030 con 0x556b4c1305d0 2024-09-19T05:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c4b7fe640 1 -- 172.21.15.64:0/3686089909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c5c0163e0 con 0x556b4c1305d0 2024-09-19T05:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c4b7fe640 1 -- 172.21.15.64:0/3686089909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c016c20 con 0x556b4c1305d0 2024-09-19T05:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 -- 172.21.15.64:0/3686089909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 msgr2=0x556b4c137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c137960 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00b2d0 tx=0x7f2c5c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 -- 172.21.15.64:0/3686089909 shutdown_connections 2024-09-19T05:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c13b370 0x556b4c136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c131b40 0x556b4c146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 --2- 172.21.15.64:0/3686089909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c1305d0 0x556b4c137960 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 -- 172.21.15.64:0/3686089909 >> 172.21.15.64:0/3686089909 conn(0x556b4c0f9310 msgr2=0x556b4c136840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 -- 172.21.15.64:0/3686089909 shutdown_connections 2024-09-19T05:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.354+0000 7f2c6a461740 1 -- 172.21.15.64:0/3686089909 wait complete. 2024-09-19T05:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 -- start start 2024-09-19T05:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 Processor -- start 2024-09-19T05:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c143420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 0x556b4c143960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c13e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ae120 con 0x556b4c131b40 2024-09-19T05:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601af590 con 0x556b4c13b370 2024-09-19T05:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ae7f0 con 0x556b4c1305d0 2024-09-19T05:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c6a461740 10 client.0 _ll_register_callbacks cb 0x7f2c5c00b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:43.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c65623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 0x556b4c143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c65623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 0x556b4c143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35600/0 (socket says 172.21.15.64:35600) 2024-09-19T05:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c65623640 1 -- 172.21.15.64:0/611953674 learned_addr learned my addr 172.21.15.64:0/611953674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c65e24640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.355+0000 7f2c66625640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c13e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c65623640 1 -- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 msgr2=0x556b4c143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c65623640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c65623640 1 -- 172.21.15.64:0/611953674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 msgr2=0x556b4c13e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c65623640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c13e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c65623640 1 -- 172.21.15.64:0/611953674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4c13f1d0 con 0x556b4c131b40 2024-09-19T05:14:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c65623640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 0x556b4c143960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2c540bc8e0 tx=0x7f2c540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b902710fba92a997 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c497fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c497fa640 1 -- 172.21.15.64:0/611953674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54049070 con 0x556b4c131b40 2024-09-19T05:14:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c497fa640 1 -- 172.21.15.64:0/611953674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f2c540ee320 con 0x556b4c131b40 2024-09-19T05:14:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5c0163e0 con 0x556b4c131b40 2024-09-19T05:14:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c497fa640 1 -- 172.21.15.64:0/611953674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c540bca00 con 0x556b4c131b40 2024-09-19T05:14:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c6a461740 10 client.17469 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b4c1b5780 con 0x556b4c131b40 2024-09-19T05:14:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c497fa640 1 -- 172.21.15.64:0/611953674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c5c0163e0 con 0x556b4c131b40 2024-09-19T05:14:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.356+0000 7f2c497fa640 1 -- 172.21.15.64:0/611953674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c54031ab0 con 0x556b4c131b40 2024-09-19T05:14:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 10 client.17469 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 10 client.17469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:14:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 20 client.17469 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c1b5b40 0x556b4c1b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 4 client.17469 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b4c09dd70 con 0x556b4c1b5b40 2024-09-19T05:14:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c65e24640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c1b5b40 0x556b4c1b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c497fa640 10 client.17469 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c65e24640 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c1b5b40 0x556b4c1b7f60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15d31d3fd6fe8c8a server_cookie=b021427ceea5f0fd in_seq=0 out_seq=0 2024-09-19T05:14:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c497fa640 1 -- 172.21.15.64:0/611953674 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b4c09dd70 con 0x556b4c1b5b40 2024-09-19T05:14:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c497fa640 10 client.17469 handle_command_reply: tid=0 2024-09-19T05:14:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.357+0000 7f2c6a461740 1 client.17469 shutdown 2024-09-19T05:14:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 msgr2=0x556b4c143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 0x556b4c143960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2c540bc8e0 tx=0x7f2c540038a0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 shutdown_connections 2024-09-19T05:14:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b4c13b370 0x556b4c13e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556b4c1b5b40 0x556b4c1b7f60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b4c131b40 0x556b4c143960 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c497fa640 0 client.17469 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 --2- 172.21.15.64:0/611953674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4c1305d0 0x556b4c143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 >> 172.21.15.64:0/611953674 conn(0x556b4c0f9310 msgr2=0x556b4c12bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 shutdown_connections 2024-09-19T05:14:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 1 -- 172.21.15.64:0/611953674 wait complete. 2024-09-19T05:14:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c6a461740 20 client.17469 trim_cache size 0 max 16384 2024-09-19T05:14:43.405 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:43.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2c4c0a33b0 msgr2=0x7f2c4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c68088640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2c4c0a33b0 0x7f2c4c0a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c58013bc0 tx=0x7f2c5800a370 comp rx=0 tx=0).stop 2024-09-19T05:14:43.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 msgr2=0x7f2c60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.358+0000 7f2c68088640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c60074a30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2c54006010 tx=0x7f2c54004270 comp rx=0 tx=0).stop 2024-09-19T05:14:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 shutdown_connections 2024-09-19T05:14:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2c4c0a33b0 0x7f2c4c0a5870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c60106f10 0x7f2c60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c60102790 0x7f2c60074a30 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 --2- 172.21.15.64:0/2519922724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c60101de0 0x7f2c600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 >> 172.21.15.64:0/2519922724 conn(0x7f2c600fd8c0 msgr2=0x7f2c600fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 shutdown_connections 2024-09-19T05:14:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:43.359+0000 7f2c68088640 1 -- 172.21.15.64:0/2519922724 wait complete. 2024-09-19T05:14:43.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:43 smithi103 ceph-mon[34446]: pgmap v271: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:43 smithi174 ceph-mon[34488]: pgmap v271: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:44.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:14:44.607 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 -- 172.21.15.64:0/1731135456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 msgr2=0x7fa4d810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 --2- 172.21.15.64:0/1731135456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d810e610 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009ca0 tx=0x7fa4d002a950 comp rx=0 tx=0).stop 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 -- 172.21.15.64:0/1731135456 shutdown_connections 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 --2- 172.21.15.64:0/1731135456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d810e610 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 --2- 172.21.15.64:0/1731135456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8103660 0x7fa4d810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 --2- 172.21.15.64:0/1731135456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8102d40 0x7fa4d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 -- 172.21.15.64:0/1731135456 >> 172.21.15.64:0/1731135456 conn(0x7fa4d80fc7a0 msgr2=0x7fa4d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 -- 172.21.15.64:0/1731135456 shutdown_connections 2024-09-19T05:14:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.097+0000 7fa4dd088640 1 -- 172.21.15.64:0/1731135456 wait complete. 2024-09-19T05:14:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 Processor -- start 2024-09-19T05:14:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 -- start start 2024-09-19T05:14:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8102d40 0x7fa4d819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8103660 0x7fa4d819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8111e20 con 0x7fa4d810c170 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8111ca0 con 0x7fa4d8103660 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4dd088640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8111fa0 con 0x7fa4d8102d40 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50660/0 (socket says 172.21.15.64:50660) 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4d759e640 1 -- 172.21.15.64:0/988902979 learned_addr learned my addr 172.21.15.64:0/988902979 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4d659c640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8103660 0x7fa4d819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.098+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8102d40 0x7fa4d819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d759e640 1 -- 172.21.15.64:0/988902979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8102d40 msgr2=0x7fa4d819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d759e640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8102d40 0x7fa4d819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d759e640 1 -- 172.21.15.64:0/988902979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8103660 msgr2=0x7fa4d819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d759e640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8103660 0x7fa4d819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d759e640 1 -- 172.21.15.64:0/988902979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d819f3a0 con 0x7fa4d810c170 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d659c640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8103660 0x7fa4d819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4d759e640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d819ecc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0006010 tx=0x7fa4d002f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4bffff640 1 -- 172.21.15.64:0/988902979 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0034040 con 0x7fa4d810c170 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d819f630 con 0x7fa4d810c170 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4bffff640 1 -- 172.21.15.64:0/988902979 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d0003e00 con 0x7fa4d810c170 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4bffff640 1 -- 172.21.15.64:0/988902979 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0045480 con 0x7fa4d810c170 2024-09-19T05:14:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.099+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d81a6f70 con 0x7fa4d810c170 2024-09-19T05:14:45.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.102+0000 7fa4df461740 1 Processor -- start 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.102+0000 7fa4bffff640 1 -- 172.21.15.64:0/988902979 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa4d0037020 con 0x7fa4d810c170 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 -- start start 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c3fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ddd5ba52f0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddd5b4ccf0 con 0x55ddd5c4a370 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4df461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddd5ba12b0 con 0x55ddd5c40b40 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c3fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c3fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50676/0 (socket says 172.21.15.64:50676) 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/935728851 learned_addr learned my addr 172.21.15.64:0/935728851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d659c640 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d659c640 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/935728851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 msgr2=0x55ddd5c3f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c3f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/935728851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 msgr2=0x55ddd5c4c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/935728851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd5b313a0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4d659c640 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.103+0000 7fa4bffff640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa4c00a33b0 0x7fa4c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.104+0000 7fa4d659c640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa4c00a33b0 0x7fa4c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.104+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c3fa10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800ad00 tx=0x7fa4c800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3791365b5f0474e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.104+0000 7fa4bffff640 1 -- 172.21.15.64:0/988902979 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa4d00f0cc0 con 0x7fa4d810c170 2024-09-19T05:14:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.104+0000 7fa4bd7fa640 1 -- 172.21.15.64:0/935728851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c801d070 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.104+0000 7fa4bd7fa640 1 -- 172.21.15.64:0/935728851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c8018af0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.104+0000 7fa4bd7fa640 1 -- 172.21.15.64:0/935728851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8010b40 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- 172.21.15.64:0/935728851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 msgr2=0x55ddd5c3fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4d659c640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa4c00a33b0 0x7fa4c00a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ddd5c40070 tx=0x7fa4cc00a250 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c3fa10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800ad00 tx=0x7fa4c800f790 comp rx=0 tx=0).stop 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- 172.21.15.64:0/935728851 shutdown_connections 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c3fa10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- 172.21.15.64:0/935728851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c3f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- 172.21.15.64:0/935728851 >> 172.21.15.64:0/935728851 conn(0x55ddd5c08310 msgr2=0x55ddd5c3bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- 172.21.15.64:0/935728851 shutdown_connections 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- 172.21.15.64:0/935728851 wait complete. 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- start start 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 Processor -- start 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c49d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c47040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c47650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ddd5c4d070 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddd5b60060 con 0x55ddd5c4a370 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddd5c4d1f0 con 0x55ddd5c40b40 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.105+0000 7fa4df461740 10 client.0 _ll_register_callbacks cb 0x55ddd58b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:45.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c49d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c49d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50678/0 (socket says 172.21.15.64:50678) 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/839644670 learned_addr learned my addr 172.21.15.64:0/839644670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d759e640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c47650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d659c640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c47040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/839644670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 msgr2=0x55ddd5c47040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c47040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 msgr2=0x55ddd5c47650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c47650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/839644670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd59163c0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c49d90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800a550 tx=0x7fa4c8018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb70f44adbacc252 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:45.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4baff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4baff5640 1 -- 172.21.15.64:0/839644670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c801d040 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4baff5640 1 -- 172.21.15.64:0/839644670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa4c8010840 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.106+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c8018af0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4baff5640 1 -- 172.21.15.64:0/839644670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa4c8031050 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 10 client.17481 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ddd5c58b40 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4baff5640 1 -- 172.21.15.64:0/839644670 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c8018af0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4baff5640 1 -- 172.21.15.64:0/839644670 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa4c8020030 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 10 client.17481 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:45.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 20 client.17481 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5c5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 4 client.17481 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddd5bb2860 con 0x55ddd5c59f40 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.107+0000 7fa4d659c640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5c5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4d659c640 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5c5c330 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dbd4180f13c5fa2 server_cookie=9da2250309d1eac0 in_seq=0 out_seq=0 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4baff5640 10 client.17481 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4baff5640 1 -- 172.21.15.64:0/839644670 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddd5bb2860 con 0x55ddd5c59f40 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4baff5640 10 client.17481 handle_command_reply: tid=0 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 client.17481 shutdown 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 msgr2=0x55ddd5c49d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c49d90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800a550 tx=0x7fa4c8018670 comp rx=0 tx=0).stop 2024-09-19T05:14:45.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 shutdown_connections 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c4a370 0x55ddd5c47650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c40b40 0x55ddd5c47040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.108+0000 7fa4df461740 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5c5c330 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.109+0000 7fa4baff5640 0 client.17481 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.109+0000 7fa4df461740 1 --2- 172.21.15.64:0/839644670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c49d90 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.109+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 >> 172.21.15.64:0/839644670 conn(0x55ddd5c08310 msgr2=0x55ddd5c41990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.109+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 shutdown_connections 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.109+0000 7fa4df461740 1 -- 172.21.15.64:0/839644670 wait complete. 2024-09-19T05:14:45.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.109+0000 7fa4df461740 20 client.17481 trim_cache size 0 max 16384 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 Processor -- start 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 -- start start 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81b2260 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81afb30 con 0x55ddd5c40b40 2024-09-19T05:14:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4df461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81ad9a0 con 0x55ddd5c4a370 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c56310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50682/0 (socket says 172.21.15.64:50682) 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/3541387436 learned_addr learned my addr 172.21.15.64:0/3541387436 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d759e640 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/3541387436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c4a370 msgr2=0x55ddd5c4b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d659c640 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/3541387436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 msgr2=0x55ddd5c46020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c46020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 -- 172.21.15.64:0/3541387436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd5c4cbd0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d659c640 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.122+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c56310 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800a5c0 tx=0x7fa4c800a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbbbd069602aae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4bcff9640 1 -- 172.21.15.64:0/3541387436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c801d040 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4bcff9640 1 -- 172.21.15.64:0/3541387436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c8028070 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4bcff9640 1 -- 172.21.15.64:0/3541387436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8018940 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 -- 172.21.15.64:0/3541387436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 msgr2=0x55ddd5c56310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c56310 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800a5c0 tx=0x7fa4c800a800 comp rx=0 tx=0).stop 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 -- 172.21.15.64:0/3541387436 shutdown_connections 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c4a370 0x55ddd5c4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 --2- 172.21.15.64:0/3541387436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c56310 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 -- 172.21.15.64:0/3541387436 >> 172.21.15.64:0/3541387436 conn(0x55ddd5c08310 msgr2=0x55ddd5c45910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 -- 172.21.15.64:0/3541387436 shutdown_connections 2024-09-19T05:14:45.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.123+0000 7fa4df461740 1 -- 172.21.15.64:0/3541387436 wait complete. 2024-09-19T05:14:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 -- start start 2024-09-19T05:14:45.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 Processor -- start 2024-09-19T05:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c52950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c44330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 0x55ddd5c44870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81adff0 con 0x55ddd5c4a370 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81af460 con 0x55ddd5c40b40 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d81ae6c0 con 0x55ddd5c3f5d0 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4df461740 10 client.0 _ll_register_callbacks cb 0x7fa4c801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 0x55ddd5c44870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 0x55ddd5c44870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50694/0 (socket says 172.21.15.64:50694) 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 -- 172.21.15.64:0/3283413911 learned_addr learned my addr 172.21.15.64:0/3283413911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c52950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d659c640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c44330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 -- 172.21.15.64:0/3283413911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c3f5d0 msgr2=0x55ddd5c52950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c52950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 -- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 msgr2=0x55ddd5c44330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c44330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d759e640 1 -- 172.21.15.64:0/3283413911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd5c44db0 con 0x55ddd5c4a370 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c52950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.124+0000 7fa4d659c640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c44330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4d759e640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 0x55ddd5c44870 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004430 tx=0x7fa4d00bb620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4692898762bd2502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4baff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4baff5640 1 -- 172.21.15.64:0/3283413911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d002c040 con 0x55ddd5c4a370 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4baff5640 1 -- 172.21.15.64:0/3283413911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fa4d00ee660 con 0x55ddd5c4a370 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c8028070 con 0x55ddd5c4a370 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4baff5640 1 -- 172.21.15.64:0/3283413911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa4d00bd110 con 0x55ddd5c4a370 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 10 client.17487 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:45.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ddd5c61e70 con 0x55ddd5c4a370 2024-09-19T05:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4baff5640 1 -- 172.21.15.64:0/3283413911 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c8028070 con 0x55ddd5c4a370 2024-09-19T05:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4baff5640 1 -- 172.21.15.64:0/3283413911 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa4d004d030 con 0x55ddd5c4a370 2024-09-19T05:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 10 client.17487 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 20 client.17487 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:45.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5cc6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 4 client.17487 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.125+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddd5bacd70 con 0x55ddd5c59f40 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5cc6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4d6d9d640 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5cc6ec0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec36c1604cd33a0 server_cookie=9e6ee55e0b7016e0 in_seq=0 out_seq=0 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4baff5640 10 client.17487 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4baff5640 1 -- 172.21.15.64:0/3283413911 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddd5bacd70 con 0x55ddd5c59f40 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4baff5640 10 client.17487 handle_command_reply: tid=0 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 client.17487 shutdown 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 msgr2=0x55ddd5c44870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 0x55ddd5c44870 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004430 tx=0x7fa4d00bb620 comp rx=0 tx=0).stop 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 shutdown_connections 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55ddd5c59f40 0x55ddd5cc6ec0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ddd5c4a370 0x55ddd5c44870 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddd5c40b40 0x55ddd5c44330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 --2- 172.21.15.64:0/3283413911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddd5c3f5d0 0x55ddd5c52950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 >> 172.21.15.64:0/3283413911 conn(0x55ddd5c08310 msgr2=0x55ddd5c46130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4baff5640 0 client.17487 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 shutdown_connections 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.126+0000 7fa4df461740 1 -- 172.21.15.64:0/3283413911 wait complete. 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4df461740 20 client.17487 trim_cache size 0 max 16384 2024-09-19T05:14:45.135 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:45.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa4c00a33b0 msgr2=0x7fa4c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa4c00a33b0 0x7fa4c00a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55ddd5c40070 tx=0x7fa4cc00a250 comp rx=0 tx=0).stop 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 msgr2=0x7fa4d819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d819ecc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0006010 tx=0x7fa4d002f640 comp rx=0 tx=0).stop 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 shutdown_connections 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa4c00a33b0 0x7fa4c00a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4d810c170 0x7fa4d819ecc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4d8103660 0x7fa4d819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 --2- 172.21.15.64:0/988902979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8102d40 0x7fa4d819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 >> 172.21.15.64:0/988902979 conn(0x7fa4d80fc7a0 msgr2=0x7fa4d80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 shutdown_connections 2024-09-19T05:14:45.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:45.127+0000 7fa4dd088640 1 -- 172.21.15.64:0/988902979 wait complete. 2024-09-19T05:14:45.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:45 smithi064 ceph-mon[30960]: pgmap v272: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:45.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:45 smithi103 ceph-mon[34446]: pgmap v272: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:45.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:45 smithi174 ceph-mon[34488]: pgmap v272: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:46.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:14:46.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:14:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:47.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:47 smithi064 ceph-mon[30960]: pgmap v273: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:47.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:47 smithi103 ceph-mon[34446]: pgmap v273: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:47.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:47 smithi174 ceph-mon[34488]: pgmap v273: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 -- 172.21.15.64:0/2409791300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102ef0 msgr2=0x7f997c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 --2- 172.21.15.64:0/2409791300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102ef0 0x7f997c10da40 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9974009ca0 tx=0x7f997402a950 comp rx=0 tx=0).stop 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 -- 172.21.15.64:0/2409791300 shutdown_connections 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 --2- 172.21.15.64:0/2409791300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c102ef0 0x7f997c10da40 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 --2- 172.21.15.64:0/2409791300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102570 0x7f997c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 --2- 172.21.15.64:0/2409791300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f997c100e20 0x7f997c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 -- 172.21.15.64:0/2409791300 >> 172.21.15.64:0/2409791300 conn(0x7f997c0fc7d0 msgr2=0x7f997c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:47.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 -- 172.21.15.64:0/2409791300 shutdown_connections 2024-09-19T05:14:47.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.582+0000 7f998377d640 1 -- 172.21.15.64:0/2409791300 wait complete. 2024-09-19T05:14:47.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 Processor -- start 2024-09-19T05:14:47.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 -- start start 2024-09-19T05:14:47.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 0x7f997c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f997c102570 0x7f997c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102ef0 0x7f997c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f997c110e30 con 0x7f997c100e20 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f997c110cb0 con 0x7f997c102ef0 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f998377d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f997c110fb0 con 0x7f997c102570 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f9981519640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 0x7f997c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f9981519640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 0x7f997c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50712/0 (socket says 172.21.15.64:50712) 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.583+0000 7f9981519640 1 -- 172.21.15.64:0/3370696389 learned_addr learned my addr 172.21.15.64:0/3370696389 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9980d18640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f997c102570 0x7f997c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981d1a640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102ef0 0x7f997c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981519640 1 -- 172.21.15.64:0/3370696389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f997c102570 msgr2=0x7f997c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981519640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f997c102570 0x7f997c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981519640 1 -- 172.21.15.64:0/3370696389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102ef0 msgr2=0x7f997c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981519640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102ef0 0x7f997c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981519640 1 -- 172.21.15.64:0/3370696389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c19d330 con 0x7f997c100e20 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981d1a640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102ef0 0x7f997c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f9981519640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 0x7f997c198380 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f997800e9f0 tx=0x7f997800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f99727fc640 1 -- 172.21.15.64:0/3370696389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997800cd80 con 0x7f997c100e20 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f99727fc640 1 -- 172.21.15.64:0/3370696389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99780054d0 con 0x7f997c100e20 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c19d620 con 0x7f997c100e20 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f99727fc640 1 -- 172.21.15.64:0/3370696389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978010490 con 0x7f997c100e20 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.584+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c1a4f00 con 0x7f997c100e20 2024-09-19T05:14:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f99727fc640 1 -- 172.21.15.64:0/3370696389 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9978005020 con 0x7f997c100e20 2024-09-19T05:14:47.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 Processor -- start 2024-09-19T05:14:47.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 -- start start 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c1b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d212b73280 con 0x55d212c0db60 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d212b1ab10 con 0x55d212c18d40 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9985b56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d212b6f240 con 0x55d212c0e480 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9981d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9981d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c0df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50724/0 (socket says 172.21.15.64:50724) 2024-09-19T05:14:47.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9981d1a640 1 -- 172.21.15.64:0/249270428 learned_addr learned my addr 172.21.15.64:0/249270428 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.587+0000 7f9981519640 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9980d18640 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c1b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f99727fc640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99680a33b0 0x7f99680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 -- 172.21.15.64:0/249270428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 msgr2=0x55d212c18800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c18800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 -- 172.21.15.64:0/249270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 msgr2=0x55d212c1b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c1b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 -- 172.21.15.64:0/249270428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d212b271f0 con 0x55d212c0db60 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f99727fc640 1 -- 172.21.15.64:0/3370696389 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f99780d0230 con 0x7f997c100e20 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c0df40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9974006010 tx=0x7f9974009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fbae28bae26bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f9981d1a640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99680a33b0 0x7f99680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f99677fe640 1 -- 172.21.15.64:0/249270428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974048040 con 0x55d212c0db60 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.588+0000 7f99677fe640 1 -- 172.21.15.64:0/249270428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f997402f480 con 0x55d212c0db60 2024-09-19T05:14:47.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f99677fe640 1 -- 172.21.15.64:0/249270428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974034050 con 0x55d212c0db60 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 -- 172.21.15.64:0/249270428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 msgr2=0x55d212c0df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9981d1a640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99680a33b0 0x7f99680a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9974043000 tx=0x7f9974043570 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c0df40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9974006010 tx=0x7f9974009a20 comp rx=0 tx=0).stop 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 -- 172.21.15.64:0/249270428 shutdown_connections 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c18800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 --2- 172.21.15.64:0/249270428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c0df40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 -- 172.21.15.64:0/249270428 >> 172.21.15.64:0/249270428 conn(0x55d212c0a0f0 msgr2=0x55d212c0a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 -- 172.21.15.64:0/249270428 shutdown_connections 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.589+0000 7f9985b56740 1 -- 172.21.15.64:0/249270428 wait complete. 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 -- start start 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 Processor -- start 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c15b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d212c1b910 con 0x55d212c0db60 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d212b2df10 con 0x55d212c18d40 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d212c1ba90 con 0x55d212c0e480 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9985b56740 10 client.0 _ll_register_callbacks cb 0x55d212af6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:47.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9981519640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9981519640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50736/0 (socket says 172.21.15.64:50736) 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9981519640 1 -- 172.21.15.64:0/2758673003 learned_addr learned my addr 172.21.15.64:0/2758673003 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9981d1a640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.590+0000 7f9980d18640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c15b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9981519640 1 -- 172.21.15.64:0/2758673003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 msgr2=0x55d212c15b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9981519640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c15b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9981519640 1 -- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 msgr2=0x55d212c162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9981519640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9981519640 1 -- 172.21.15.64:0/2758673003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2128e46a0 con 0x55d212c0db60 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9980d18640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c15b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9981519640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c187f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f997800e9c0 tx=0x7f9978009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955ef40ac45690f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9964ff9640 1 -- 172.21.15.64:0/2758673003 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997809c2b0 con 0x55d212c0db60 2024-09-19T05:14:47.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9964ff9640 1 -- 172.21.15.64:0/2758673003 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f9978098b50 con 0x55d212c0db60 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997402f480 con 0x55d212c0db60 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9964ff9640 1 -- 172.21.15.64:0/2758673003 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f997809be40 con 0x55d212c0db60 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9985b56740 10 client.17499 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d212c274a0 con 0x55d212c0db60 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.591+0000 7f9964ff9640 1 -- 172.21.15.64:0/2758673003 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f997402f480 con 0x55d212c0db60 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9964ff9640 1 -- 172.21.15.64:0/2758673003 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f99780a4cc0 con 0x55d212c0db60 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9985b56740 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9985b56740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9985b56740 20 client.17499 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9985b56740 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c28960 0x55d212c2add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9985b56740 4 client.17499 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d212b807f0 con 0x55d212c28960 2024-09-19T05:14:47.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9980d18640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c28960 0x55d212c2add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9980d18640 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c28960 0x55d212c2add0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511f33f3ea9c3a57 server_cookie=be73fd32681ad8a5 in_seq=0 out_seq=0 2024-09-19T05:14:47.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.592+0000 7f9964ff9640 10 client.17499 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9964ff9640 1 -- 172.21.15.64:0/2758673003 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d212b807f0 con 0x55d212c28960 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9964ff9640 10 client.17499 handle_command_reply: tid=0 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 client.17499 shutdown 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 msgr2=0x55d212c187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c187f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f997800e9c0 tx=0x7f9978009310 comp rx=0 tx=0).stop 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 shutdown_connections 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c18d40 0x55d212c162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0e480 0x55d212c15b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c28960 0x55d212c2add0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9964ff9640 0 client.17499 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 --2- 172.21.15.64:0/2758673003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0db60 0x55d212c187f0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 >> 172.21.15.64:0/2758673003 conn(0x55d212c0a0f0 msgr2=0x55d212c09270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 shutdown_connections 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.593+0000 7f9985b56740 1 -- 172.21.15.64:0/2758673003 wait complete. 2024-09-19T05:14:47.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.594+0000 7f9985b56740 20 client.17499 trim_cache size 0 max 16384 2024-09-19T05:14:47.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 Processor -- start 2024-09-19T05:14:47.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 -- start start 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0db60 0x55d212c306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 0x55d212c30be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c18d40 0x55d212c12da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1b0070 con 0x55d212c0e480 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ad940 con 0x55d212c0db60 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9985b56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ab7b0 con 0x55d212c18d40 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 0x55d212c30be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 0x55d212c30be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50742/0 (socket says 172.21.15.64:50742) 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 -- 172.21.15.64:0/1173095350 learned_addr learned my addr 172.21.15.64:0/1173095350 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9981519640 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0db60 0x55d212c306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9981d1a640 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c18d40 0x55d212c12da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 -- 172.21.15.64:0/1173095350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c18d40 msgr2=0x55d212c12da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c18d40 0x55d212c12da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 -- 172.21.15.64:0/1173095350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0db60 msgr2=0x55d212c306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0db60 0x55d212c306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9980d18640 1 -- 172.21.15.64:0/1173095350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d212c0f070 con 0x55d212c0e480 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.611+0000 7f9981d1a640 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c18d40 0x55d212c12da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9980d18640 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 0x55d212c30be0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x55d212c153d0 tx=0x7f996c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be9532496fe6664 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9966ffd640 1 -- 172.21.15.64:0/1173095350 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c011700 con 0x55d212c0e480 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9966ffd640 1 -- 172.21.15.64:0/1173095350 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f996c011d50 con 0x55d212c0e480 2024-09-19T05:14:47.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9966ffd640 1 -- 172.21.15.64:0/1173095350 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c027db0 con 0x55d212c0e480 2024-09-19T05:14:47.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 -- 172.21.15.64:0/1173095350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 msgr2=0x55d212c30be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 0x55d212c30be0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x55d212c153d0 tx=0x7f996c0128e0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 -- 172.21.15.64:0/1173095350 shutdown_connections 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c18d40 0x55d212c12da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c0e480 0x55d212c30be0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 --2- 172.21.15.64:0/1173095350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0db60 0x55d212c306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 -- 172.21.15.64:0/1173095350 >> 172.21.15.64:0/1173095350 conn(0x55d212c0a0f0 msgr2=0x55d212c14c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 -- 172.21.15.64:0/1173095350 shutdown_connections 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.612+0000 7f9985b56740 1 -- 172.21.15.64:0/1173095350 wait complete. 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 -- start start 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 Processor -- start 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0db60 0x55d212c1d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0e480 0x55d212c1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 0x55d212c92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1abe00 con 0x55d212c18d40 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ad270 con 0x55d212c0e480 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f997c1ac4d0 con 0x55d212c0db60 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9985b56740 10 client.0 _ll_register_callbacks cb 0x7f99780d3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9981d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 0x55d212c92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9981d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 0x55d212c92670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50752/0 (socket says 172.21.15.64:50752) 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.613+0000 7f9981d1a640 1 -- 172.21.15.64:0/819953118 learned_addr learned my addr 172.21.15.64:0/819953118 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9980d18640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0e480 0x55d212c1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981519640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0db60 0x55d212c1d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981d1a640 1 -- 172.21.15.64:0/819953118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0db60 msgr2=0x55d212c1d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981d1a640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0db60 0x55d212c1d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981d1a640 1 -- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0e480 msgr2=0x55d212c1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:47.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981d1a640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0e480 0x55d212c1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981d1a640 1 -- 172.21.15.64:0/819953118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d212c92be0 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9981d1a640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 0x55d212c92670 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9974009a60 tx=0x7f997404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d8c7f128fddd97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9964ff9640 1 -- 172.21.15.64:0/819953118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997405e060 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f996c011d50 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9964ff9640 1 -- 172.21.15.64:0/819953118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f9974004550 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9964ff9640 1 -- 172.21.15.64:0/819953118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9974055040 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9985b56740 10 client.17505 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.614+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d212c93e70 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9964ff9640 1 -- 172.21.15.64:0/819953118 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f996c011d50 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9964ff9640 1 -- 172.21.15.64:0/819953118 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9974058030 con 0x55d212c18d40 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9985b56740 10 client.17505 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:47.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9985b56740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9985b56740 20 client.17505 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9985b56740 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c94150 0x55d212c96540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9985b56740 4 client.17505 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d212b7ad00 con 0x55d212c94150 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9981519640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c94150 0x55d212c96540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9981519640 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c94150 0x55d212c96540 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b1acc79ec857b8 server_cookie=f7cea31d5f370f5a in_seq=0 out_seq=0 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.615+0000 7f9964ff9640 10 client.17505 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.616+0000 7f9964ff9640 1 -- 172.21.15.64:0/819953118 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d212b7ad00 con 0x55d212c94150 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9964ff9640 10 client.17505 handle_command_reply: tid=0 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 client.17505 shutdown 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 msgr2=0x55d212c92670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 0x55d212c92670 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9974009a60 tx=0x7f997404c000 comp rx=0 tx=0).stop 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 shutdown_connections 2024-09-19T05:14:47.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55d212c94150 0x55d212c96540 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9964ff9640 0 client.17505 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d212c18d40 0x55d212c92670 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d212c0e480 0x55d212c1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 --2- 172.21.15.64:0/819953118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d212c0db60 0x55d212c1d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 >> 172.21.15.64:0/819953118 conn(0x55d212c0a0f0 msgr2=0x55d212c08a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 shutdown_connections 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 1 -- 172.21.15.64:0/819953118 wait complete. 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.617+0000 7f9985b56740 20 client.17505 trim_cache size 0 max 16384 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:47.623 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:14:47.624 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "9194de07-e51c-4f95-ac28-41005aafb7c1", 2024-09-19T05:14:47.624 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:14:47.624 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:47.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99680a33b0 msgr2=0x7f99680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99680a33b0 0x7f99680a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9974043000 tx=0x7f9974043570 comp rx=0 tx=0).stop 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 msgr2=0x7f997c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 0x7f997c198380 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f997800e9f0 tx=0x7f997800edf0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 shutdown_connections 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f99680a33b0 0x7f99680a5870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f997c102ef0 0x7f997c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f997c102570 0x7f997c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 --2- 172.21.15.64:0/3370696389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f997c100e20 0x7f997c198380 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 >> 172.21.15.64:0/3370696389 conn(0x7f997c0fc7d0 msgr2=0x7f997c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 shutdown_connections 2024-09-19T05:14:47.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:47.618+0000 7f998377d640 1 -- 172.21.15.64:0/3370696389 wait complete. 2024-09-19T05:14:48.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:48.663 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 -- 172.21.15.64:0/4285301331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 msgr2=0x7fcd00105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 --2- 172.21.15.64:0/4285301331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd00105eb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009900 tx=0x7fcce802a410 comp rx=0 tx=0).stop 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 -- 172.21.15.64:0/4285301331 shutdown_connections 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 --2- 172.21.15.64:0/4285301331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ba0 0x7fcd0010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 --2- 172.21.15.64:0/4285301331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd00101200 0x7fcd00101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 --2- 172.21.15.64:0/4285301331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd00105eb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 -- 172.21.15.64:0/4285301331 >> 172.21.15.64:0/4285301331 conn(0x7fcd00077210 msgr2=0x7fcd000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.256+0000 7fcd04f74640 1 -- 172.21.15.64:0/4285301331 shutdown_connections 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 -- 172.21.15.64:0/4285301331 wait complete. 2024-09-19T05:14:49.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 Processor -- start 2024-09-19T05:14:49.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 -- start start 2024-09-19T05:14:49.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd00101200 0x7fcd00197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ba0 0x7fcd00197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd0019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00110df0 con 0x7fcd00105ad0 2024-09-19T05:14:49.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.257+0000 7fcd04f74640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00110c70 con 0x7fcd00101ba0 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fcd04f74640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00110f70 con 0x7fcd00101200 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd0019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd0019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50764/0 (socket says 172.21.15.64:50764) 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 -- 172.21.15.64:0/3535281572 learned_addr learned my addr 172.21.15.64:0/3535281572 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfe59c640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd00101200 0x7fcd00197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ba0 0x7fcd00197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 -- 172.21.15.64:0/3535281572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd00101200 msgr2=0x7fcd00197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd00101200 0x7fcd00197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 -- 172.21.15.64:0/3535281572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ba0 msgr2=0x7fcd00197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ba0 0x7fcd00197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 -- 172.21.15.64:0/3535281572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0019c1c0 con 0x7fcd00105ad0 2024-09-19T05:14:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccfed9d640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd0019bae0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fccf800dac0 tx=0x7fccf800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccf77fe640 1 -- 172.21.15.64:0/3535281572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8017070 con 0x7fcd00105ad0 2024-09-19T05:14:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccf77fe640 1 -- 172.21.15.64:0/3535281572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccf800be30 con 0x7fcd00105ad0 2024-09-19T05:14:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fccf77fe640 1 -- 172.21.15.64:0/3535281572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf800f820 con 0x7fcd00105ad0 2024-09-19T05:14:49.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0019c4b0 con 0x7fcd00105ad0 2024-09-19T05:14:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.258+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001a3d90 con 0x7fcd00105ad0 2024-09-19T05:14:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fccf77fe640 1 -- 172.21.15.64:0/3535281572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fccf800fa00 con 0x7fcd00105ad0 2024-09-19T05:14:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 Processor -- start 2024-09-19T05:14:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 -- start start 2024-09-19T05:14:49.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 0x558e4551f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e4552c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e4552ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e454852e0 con 0x558e4551f340 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e4542cce0 con 0x558e4552c890 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fcd0734d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e454812a0 con 0x558e4551fc60 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.261+0000 7fccf77fe640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fccec0a3430 0x7fccec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 0x558e4551f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 0x558e4551f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50770/0 (socket says 172.21.15.64:50770) 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1939421243 learned_addr learned my addr 172.21.15.64:0/1939421243 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfed9d640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fccec0a3430 0x7fccec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfe59c640 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e4552c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccf77fe640 1 -- 172.21.15.64:0/3535281572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fccf80cfd10 con 0x7fcd00105ad0 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e4552ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1939421243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 msgr2=0x558e4552c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e4552c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1939421243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 msgr2=0x558e4552ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e4552ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1939421243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e45411440 con 0x558e4551f340 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfed9d640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fccec0a3430 0x7fccec0a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccf809bc60 tx=0x7fccf806dce0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:49.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 0x558e4551f720 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fccf00135a0 tx=0x7fccf000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddebedb3e0deea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.262+0000 7fccf4ff9640 1 -- 172.21.15.64:0/1939421243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0019030 con 0x558e4551f340 2024-09-19T05:14:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fccf4ff9640 1 -- 172.21.15.64:0/1939421243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccf0011970 con 0x558e4551f340 2024-09-19T05:14:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fccf4ff9640 1 -- 172.21.15.64:0/1939421243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf001d850 con 0x558e4551f340 2024-09-19T05:14:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 -- 172.21.15.64:0/1939421243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 msgr2=0x558e4551f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 0x558e4551f720 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fccf00135a0 tx=0x7fccf000b6b0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 -- 172.21.15.64:0/1939421243 shutdown_connections 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e4552ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e4552c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1939421243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551f340 0x558e4551f720 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 -- 172.21.15.64:0/1939421243 >> 172.21.15.64:0/1939421243 conn(0x558e4551ba30 msgr2=0x558e4551be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 -- 172.21.15.64:0/1939421243 shutdown_connections 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.263+0000 7fcd0734d740 1 -- 172.21.15.64:0/1939421243 wait complete. 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 -- start start 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 Processor -- start 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e4552c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551fc60 0x558e455294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e45529a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e4552f460 con 0x558e4552c890 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e45440050 con 0x558e4551fc60 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e4552f5e0 con 0x558e4551f340 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fcd0734d740 10 client.0 _ll_register_callbacks cb 0x558e45480160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fccfdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e45529a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fccfdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e45529a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50786/0 (socket says 172.21.15.64:50786) 2024-09-19T05:14:49.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1359042779 learned_addr learned my addr 172.21.15.64:0/1359042779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e4552c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.264+0000 7fccfe59c640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551fc60 0x558e455294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1359042779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 msgr2=0x558e4552c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e4552c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551fc60 msgr2=0x558e455294a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551fc60 0x558e455294a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fccfdd9b640 1 -- 172.21.15.64:0/1359042779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e451f63c0 con 0x558e4552c890 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e45529a00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fccf001d7a0 tx=0x7fccf000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d345a8d3b78ab27c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:49.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcce5ffb640 1 -- 172.21.15.64:0/1359042779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0019040 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcce5ffb640 1 -- 172.21.15.64:0/1359042779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fccf00114c0 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf0011970 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcce5ffb640 1 -- 172.21.15.64:0/1359042779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fccf0033050 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcd0734d740 10 client.17517 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.265+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e4553ae70 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcce5ffb640 1 -- 172.21.15.64:0/1359042779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccf0011970 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcce5ffb640 1 -- 172.21.15.64:0/1359042779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fccf002b020 con 0x558e4552c890 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcd0734d740 10 client.17517 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:49.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcd0734d740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcd0734d740 20 client.17517 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e4553e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcd0734d740 4 client.17517 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e45492850 con 0x558e4553c2d0 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fccfe59c640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e4553e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fcce5ffb640 10 client.17517 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.266+0000 7fccfe59c640 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e4553e6c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e348b8e9e9beb91 server_cookie=1d075f14a9c99a80 in_seq=0 out_seq=0 2024-09-19T05:14:49.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcce5ffb640 1 -- 172.21.15.64:0/1359042779 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e45492850 con 0x558e4553c2d0 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcce5ffb640 10 client.17517 handle_command_reply: tid=0 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 client.17517 shutdown 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 msgr2=0x558e45529a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e45529a00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fccf001d7a0 tx=0x7fccf000b610 comp rx=0 tx=0).stop 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 shutdown_connections 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e4553e6c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcce5ffb640 0 client.17517 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e45529a00 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551fc60 0x558e455294a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 --2- 172.21.15.64:0/1359042779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e4552c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.267+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 >> 172.21.15.64:0/1359042779 conn(0x558e4551ba30 msgr2=0x558e45521790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.268+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 shutdown_connections 2024-09-19T05:14:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.268+0000 7fcd0734d740 1 -- 172.21.15.64:0/1359042779 wait complete. 2024-09-19T05:14:49.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.268+0000 7fcd0734d740 20 client.17517 trim_cache size 0 max 16384 2024-09-19T05:14:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 Processor -- start 2024-09-19T05:14:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 -- start start 2024-09-19T05:14:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e45538700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 0x558e45543de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e45544320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd001adf20 con 0x558e4551fc60 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd001ab7f0 con 0x558e4552c890 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fcd0734d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd001a9660 con 0x558e4551f340 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 0x558e45543de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 0x558e45543de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50788/0 (socket says 172.21.15.64:50788) 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 -- 172.21.15.64:0/491668622 learned_addr learned my addr 172.21.15.64:0/491668622 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e45538700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e45544320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 -- 172.21.15.64:0/491668622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 msgr2=0x558e45538700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e45538700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 -- 172.21.15.64:0/491668622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 msgr2=0x558e45544320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.285+0000 7fccfe59c640 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e45544320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.286+0000 7fccfe59c640 1 -- 172.21.15.64:0/491668622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4551b4d0 con 0x558e4551fc60 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.286+0000 7fccfe59c640 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 0x558e45543de0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcce802a9d0 tx=0x7fcce8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ded0d48545346fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.286+0000 7fcce7fff640 1 -- 172.21.15.64:0/491668622 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8035040 con 0x558e4551fc60 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.286+0000 7fcce7fff640 1 -- 172.21.15.64:0/491668622 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce8049070 con 0x558e4551fc60 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.286+0000 7fcce7fff640 1 -- 172.21.15.64:0/491668622 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8003850 con 0x558e4551fc60 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- 172.21.15.64:0/491668622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 msgr2=0x558e45543de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 0x558e45543de0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcce802a9d0 tx=0x7fcce8005b50 comp rx=0 tx=0).stop 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- 172.21.15.64:0/491668622 shutdown_connections 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4552c890 0x558e45544320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4551fc60 0x558e45543de0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- 172.21.15.64:0/491668622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551f340 0x558e45538700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- 172.21.15.64:0/491668622 >> 172.21.15.64:0/491668622 conn(0x558e4551ba30 msgr2=0x558e45524b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- 172.21.15.64:0/491668622 shutdown_connections 2024-09-19T05:14:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- 172.21.15.64:0/491668622 wait complete. 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- start start 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 Processor -- start 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551f340 0x558e4552fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e45530330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e455a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd001a9cb0 con 0x558e4552c890 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd001ab120 con 0x558e4551f340 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.287+0000 7fcd0734d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd001aa380 con 0x558e4551fc60 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fcd0734d740 10 client.0 _ll_register_callbacks cb 0x7fcce8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e455a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfdd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e455a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50800/0 (socket says 172.21.15.64:50800) 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfdd9b640 1 -- 172.21.15.64:0/2111931585 learned_addr learned my addr 172.21.15.64:0/2111931585 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e45530330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551f340 0x558e4552fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 -- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551f340 msgr2=0x558e4552fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551f340 0x558e4552fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 -- 172.21.15.64:0/2111931585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 msgr2=0x558e455a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e455a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 -- 172.21.15.64:0/2111931585 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e455a5310 con 0x558e4551fc60 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e455a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fccfe59c640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e45530330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcce802aa00 tx=0x7fcce8004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552da69efb7a7c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fcce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fcce5ffb640 1 -- 172.21.15.64:0/2111931585 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8051030 con 0x558e4551fc60 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8049070 con 0x558e4551fc60 2024-09-19T05:14:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.288+0000 7fcce5ffb640 1 -- 172.21.15.64:0/2111931585 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fcce80054d0 con 0x558e4551fc60 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcce5ffb640 1 -- 172.21.15.64:0/2111931585 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcce8035050 con 0x558e4551fc60 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 10 client.26545 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e45522130 con 0x558e4551fc60 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcce5ffb640 1 -- 172.21.15.64:0/2111931585 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce8049070 con 0x558e4551fc60 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcce5ffb640 1 -- 172.21.15.64:0/2111931585 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcce805c020 con 0x558e4551fc60 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 10 client.26545 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 20 client.26545 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e45522810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 4 client.26545 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e4548cd60 con 0x558e4553c2d0 2024-09-19T05:14:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.289+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e45522810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fccfdd9b640 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e45522810 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ca8d8b2afc5543 server_cookie=6d6fba434e9e1a83 in_seq=0 out_seq=0 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcce5ffb640 10 client.26545 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcce5ffb640 1 -- 172.21.15.64:0/2111931585 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558e4548cd60 con 0x558e4553c2d0 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcce5ffb640 10 client.26545 handle_command_reply: tid=0 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 client.26545 shutdown 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 msgr2=0x558e45530330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e45530330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcce802aa00 tx=0x7fcce8004f40 comp rx=0 tx=0).stop 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 shutdown_connections 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558e4553c2d0 0x558e45522810 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e4552c890 0x558e455a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcce5ffb640 0 client.26545 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:49.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4551fc60 0x558e45530330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 --2- 172.21.15.64:0/2111931585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e4551f340 0x558e4552fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 >> 172.21.15.64:0/2111931585 conn(0x558e4551ba30 msgr2=0x558e45527c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 shutdown_connections 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 1 -- 172.21.15.64:0/2111931585 wait complete. 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.290+0000 7fcd0734d740 20 client.26545 trim_cache size 0 max 16384 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:14:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:14:49.304 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fccec0a3430 msgr2=0x7fccec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fccec0a3430 0x7fccec0a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccf809bc60 tx=0x7fccf806dce0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 msgr2=0x7fcd0019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd0019bae0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fccf800dac0 tx=0x7fccf800df80 comp rx=0 tx=0).stop 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 shutdown_connections 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fccec0a3430 0x7fccec0a58f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd00105ad0 0x7fcd0019bae0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd00101ba0 0x7fcd00197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 --2- 172.21.15.64:0/3535281572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd00101200 0x7fcd00197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 >> 172.21.15.64:0/3535281572 conn(0x7fcd00077210 msgr2=0x7fcd00103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:49.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 shutdown_connections 2024-09-19T05:14:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:49.291+0000 7fcd04f74640 1 -- 172.21.15.64:0/3535281572 wait complete. 2024-09-19T05:14:49.307 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:49 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:14:49.307 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:49 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:49.307 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:49 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:14:49.307 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:49 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:49.308 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:49 smithi064 ceph-mon[30960]: pgmap v274: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:49.308 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:49 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:14:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:49 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:14:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:49 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:49 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:14:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:49 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:49.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:49 smithi103 ceph-mon[34446]: pgmap v274: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:49.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:49 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:14:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:49 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:14:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:49 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:49 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:14:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:49 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:49 smithi174 ceph-mon[34488]: pgmap v274: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:49 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:14:50.133 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:14:50.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:14:50.392 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 -- 172.21.15.64:0/2174111045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c102790 msgr2=0x7f004c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 --2- 172.21.15.64:0/2174111045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c102790 0x7f004c1067e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0048009900 tx=0x7f004802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:14:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 -- 172.21.15.64:0/2174111045 shutdown_connections 2024-09-19T05:14:50.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 --2- 172.21.15.64:0/2174111045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f004c106f10 0x7f004c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 --2- 172.21.15.64:0/2174111045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c102790 0x7f004c1067e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 --2- 172.21.15.64:0/2174111045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c101de0 0x7f004c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 -- 172.21.15.64:0/2174111045 >> 172.21.15.64:0/2174111045 conn(0x7f004c0fd8c0 msgr2=0x7f004c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 -- 172.21.15.64:0/2174111045 shutdown_connections 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.863+0000 7f00532be640 1 -- 172.21.15.64:0/2174111045 wait complete. 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 Processor -- start 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 -- start start 2024-09-19T05:14:50.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 0x7f004c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f004c102790 0x7f004c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c106f10 0x7f004c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f004c111e40 con 0x7f004c101de0 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f004c111cc0 con 0x7f004c106f10 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f00532be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f004c111fc0 con 0x7f004c102790 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 0x7f004c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 0x7f004c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50810/0 (socket says 172.21.15.64:50810) 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 -- 172.21.15.64:0/239172672 learned_addr learned my addr 172.21.15.64:0/239172672 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f0050859640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f004c102790 0x7f004c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005185b640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c106f10 0x7f004c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 -- 172.21.15.64:0/239172672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f004c102790 msgr2=0x7f004c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:50.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f004c102790 0x7f004c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 -- 172.21.15.64:0/239172672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c106f10 msgr2=0x7f004c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c106f10 0x7f004c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.864+0000 7f005105a640 1 -- 172.21.15.64:0/239172672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c19d2d0 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.865+0000 7f005105a640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 0x7f004c198320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f004000c920 tx=0x7f004000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.865+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040007c70 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.865+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0040007e10 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.865+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c19d5c0 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.865+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040018820 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.865+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c1a4ea0 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.866+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c1021c0 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.867+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f00400189c0 con 0x7f004c101de0 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.868+0000 7f003e7fc640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f00380a3430 0x7f00380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:50.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.868+0000 7f0050859640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f00380a3430 0x7f00380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.868+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f00400cfed0 con 0x7f004c101de0 2024-09-19T05:14:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.868+0000 7f0050859640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f00380a3430 0x7f00380a58f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0048031440 tx=0x7f00480095f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:50.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:50.872+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0040014030 con 0x7f004c101de0 2024-09-19T05:14:51.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.086+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004c109b60 con 0x7f004c101de0 2024-09-19T05:14:51.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.087+0000 7f003e7fc640 1 -- 172.21.15.64:0/239172672 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f004000de20 con 0x7f004c101de0 2024-09-19T05:14:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:14:51.090 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:14:51.090 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:14:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f00380a3430 msgr2=0x7f00380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f00380a3430 0x7f00380a58f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0048031440 tx=0x7f00480095f0 comp rx=0 tx=0).stop 2024-09-19T05:14:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 msgr2=0x7f004c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 0x7f004c198320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f004000c920 tx=0x7f004000cde0 comp rx=0 tx=0).stop 2024-09-19T05:14:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 shutdown_connections 2024-09-19T05:14:51.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f00380a3430 0x7f00380a58f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:51.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c106f10 0x7f004c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:51.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f004c102790 0x7f004c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:51.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 --2- 172.21.15.64:0/239172672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f004c101de0 0x7f004c198320 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:51.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 >> 172.21.15.64:0/239172672 conn(0x7f004c0fd8c0 msgr2=0x7f004c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:51.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 shutdown_connections 2024-09-19T05:14:51.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:51.090+0000 7f00532be640 1 -- 172.21.15.64:0/239172672 wait complete. 2024-09-19T05:14:51.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:51 smithi064 ceph-mon[30960]: pgmap v275: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:14:51.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:51 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/239172672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:51.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:51 smithi103 ceph-mon[34446]: pgmap v275: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:14:51.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:51 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/239172672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:51.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:51 smithi174 ceph-mon[34488]: pgmap v275: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:14:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:51 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/239172672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:51.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:14:52.087 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:52.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 -- 172.21.15.64:0/3337224979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4106a80 msgr2=0x7f0fc4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 --2- 172.21.15.64:0/3337224979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4106a80 0x7f0fc4106e60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009930 tx=0x7f0fb402a420 comp rx=0 tx=0).stop 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 -- 172.21.15.64:0/3337224979 shutdown_connections 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 --2- 172.21.15.64:0/3337224979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fc4101420 0x7f0fc410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 --2- 172.21.15.64:0/3337224979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc4100a80 0x7f0fc4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 --2- 172.21.15.64:0/3337224979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4106a80 0x7f0fc4106e60 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 -- 172.21.15.64:0/3337224979 >> 172.21.15.64:0/3337224979 conn(0x7f0fc40fc7a0 msgr2=0x7f0fc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 -- 172.21.15.64:0/3337224979 shutdown_connections 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.633+0000 7f0fc981a640 1 -- 172.21.15.64:0/3337224979 wait complete. 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 Processor -- start 2024-09-19T05:14:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 -- start start 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fc4100a80 0x7f0fc4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 0x7f0fc4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc4106a80 0x7f0fc419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4110cf0 con 0x7f0fc4101420 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4110b70 con 0x7f0fc4106a80 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fc981a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4110e70 con 0x7f0fc4100a80 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 0x7f0fc4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 0x7f0fc4198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50838/0 (socket says 172.21.15.64:50838) 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.634+0000 7f0fbbfff640 1 -- 172.21.15.64:0/2111829740 learned_addr learned my addr 172.21.15.64:0/2111829740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fc4100a80 0x7f0fc4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc4106a80 0x7f0fc419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fbbfff640 1 -- 172.21.15.64:0/2111829740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fc4100a80 msgr2=0x7f0fc4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fc4100a80 0x7f0fc4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fbbfff640 1 -- 172.21.15.64:0/2111829740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc4106a80 msgr2=0x7f0fc419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc4106a80 0x7f0fc419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fbbfff640 1 -- 172.21.15.64:0/2111829740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc419d300 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 0x7f0fc4198890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e9f0 tx=0x7f0fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc8818640 1 -- 172.21.15.64:0/2111829740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00cd80 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc8818640 1 -- 172.21.15.64:0/2111829740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fac0108e0 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc419d5f0 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc8818640 1 -- 172.21.15.64:0/2111829740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac018920 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.635+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc41a4ed0 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.638+0000 7f0fc8818640 1 -- 172.21.15.64:0/2111829740 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0fac010a90 con 0x7f0fc4101420 2024-09-19T05:14:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.638+0000 7f0fcbbf3740 1 Processor -- start 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.638+0000 7f0fcbbf3740 1 -- start start 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a4cc202f0 con 0x564a4ccc5370 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a4cbc7cf0 con 0x564a4ccbbb40 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a4cc1c2b0 con 0x564a4ccba5d0 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccc7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50852/0 (socket says 172.21.15.64:50852) 2024-09-19T05:14:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 -- 172.21.15.64:0/323105890 learned_addr learned my addr 172.21.15.64:0/323105890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 -- 172.21.15.64:0/323105890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 msgr2=0x564a4ccbaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccbaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 -- 172.21.15.64:0/323105890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 msgr2=0x564a4ccba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 -- 172.21.15.64:0/323105890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4cbac3a0 con 0x564a4ccc5370 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fc8818640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0fb00a33b0 0x7f0fb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccc7760 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0cd6b0 tx=0x7f0fac0d3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cedc7aa09ea21a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0fb00a33b0 0x7f0fb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.639+0000 7f0fc8818640 1 -- 172.21.15.64:0/2111829740 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f0fac0d0020 con 0x7f0fc4101420 2024-09-19T05:14:52.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fb9ffb640 1 -- 172.21.15.64:0/323105890 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac098620 con 0x564a4ccc5370 2024-09-19T05:14:52.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fb9ffb640 1 -- 172.21.15.64:0/323105890 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fac010de0 con 0x564a4ccc5370 2024-09-19T05:14:52.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fb9ffb640 1 -- 172.21.15.64:0/323105890 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac09bce0 con 0x564a4ccc5370 2024-09-19T05:14:52.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/323105890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 msgr2=0x564a4ccc7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0fb00a33b0 0x7f0fb00a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005fd0 tx=0x7f0fbc009600 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccc7760 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0cd6b0 tx=0x7f0fac0d3f90 comp rx=0 tx=0).stop 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/323105890 shutdown_connections 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccc7760 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccbaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/323105890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.640+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/323105890 >> 172.21.15.64:0/323105890 conn(0x564a4cc83310 msgr2=0x564a4ccb6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/323105890 shutdown_connections 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/323105890 wait complete. 2024-09-19T05:14:52.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 -- start start 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 Processor -- start 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccba5d0 0x564a4ccc4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4ccc1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccc5370 0x564a4ccc25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a4ccc8070 con 0x564a4ccbbb40 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a4cbdb060 con 0x564a4ccba5d0 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a4ccc81f0 con 0x564a4ccc5370 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fcbbf3740 10 client.0 _ll_register_callbacks cb 0x564a4c92e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.641+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4ccc1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4ccc1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50862/0 (socket says 172.21.15.64:50862) 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 -- 172.21.15.64:0/451608959 learned_addr learned my addr 172.21.15.64:0/451608959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccba5d0 0x564a4ccc4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccc5370 0x564a4ccc25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 -- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccc5370 msgr2=0x564a4ccc25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccc5370 0x564a4ccc25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 -- 172.21.15.64:0/451608959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccba5d0 msgr2=0x564a4ccc4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccba5d0 0x564a4ccc4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 -- 172.21.15.64:0/451608959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4c9913c0 con 0x564a4ccbbb40 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4ccc1ff0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0cdd40 tx=0x7f0fac009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203b29989763b3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/451608959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac09bec0 con 0x564a4ccbbb40 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/451608959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f0fac09c5e0 con 0x564a4ccbbb40 2024-09-19T05:14:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a4cc2d860 con 0x564a4ccbbb40 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/451608959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fac09c150 con 0x564a4ccbbb40 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fcbbf3740 10 client.17541 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.642+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a4ccc8070 con 0x564a4ccbbb40 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/451608959 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564a4cc2d860 con 0x564a4ccbbb40 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/451608959 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0fac0a4ab0 con 0x564a4ccbbb40 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fcbbf3740 10 client.17541 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fcbbf3740 10 client.17541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fcbbf3740 20 client.17541 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4ccd4c60 0x564a4ccd7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fcbbf3740 4 client.17541 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a4cc27d70 con 0x564a4ccd4c60 2024-09-19T05:14:52.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4ccd4c60 0x564a4ccd7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fa2ffd640 10 client.17541 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.643+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4ccd4c60 0x564a4ccd7050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7ebec22a40b9917 server_cookie=b747da0f3f5dc02c in_seq=0 out_seq=0 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/451608959 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a4cc27d70 con 0x564a4ccd4c60 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fa2ffd640 10 client.17541 handle_command_reply: tid=0 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 client.17541 shutdown 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 msgr2=0x564a4ccc1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4ccc1ff0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0cdd40 tx=0x7f0fac009770 comp rx=0 tx=0).stop 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 shutdown_connections 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4ccd4c60 0x564a4ccd7050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccc5370 0x564a4ccc25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4ccc1ff0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/451608959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccba5d0 0x564a4ccc4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 >> 172.21.15.64:0/451608959 conn(0x564a4cc83310 msgr2=0x564a4ccbc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.644+0000 7f0fa2ffd640 0 client.17541 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.645+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 shutdown_connections 2024-09-19T05:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.645+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/451608959 wait complete. 2024-09-19T05:14:52.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.645+0000 7f0fcbbf3740 20 client.17541 trim_cache size 0 max 16384 2024-09-19T05:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 Processor -- start 2024-09-19T05:14:52.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 -- start start 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccd1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccdc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41b0270 con 0x564a4ccc5370 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41adb40 con 0x564a4ccbbb40 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ab9b0 con 0x564a4ccba5d0 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccdc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccdc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50868/0 (socket says 172.21.15.64:50868) 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 -- 172.21.15.64:0/3135643762 learned_addr learned my addr 172.21.15.64:0/3135643762 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccd1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 -- 172.21.15.64:0/3135643762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 msgr2=0x564a4ccd1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccd1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 -- 172.21.15.64:0/3135643762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 msgr2=0x564a4ccc0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccc0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fc37fe640 1 -- 172.21.15.64:0/3135643762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4ccb64f0 con 0x564a4ccc5370 2024-09-19T05:14:52.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.658+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccdc7e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0092f0 tx=0x7f0fbc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7b21311b955559 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fb97fa640 1 -- 172.21.15.64:0/3135643762 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc003b70 con 0x564a4ccc5370 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fb97fa640 1 -- 172.21.15.64:0/3135643762 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fbc003d10 con 0x564a4ccc5370 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fb97fa640 1 -- 172.21.15.64:0/3135643762 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc021a80 con 0x564a4ccc5370 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3135643762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 msgr2=0x564a4ccdc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccdc7e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0092f0 tx=0x7f0fbc016000 comp rx=0 tx=0).stop 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3135643762 shutdown_connections 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccc5370 0x564a4ccdc7e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccbbb40 0x564a4ccc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.659+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3135643762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4ccd1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3135643762 >> 172.21.15.64:0/3135643762 conn(0x564a4cc83310 msgr2=0x564a4ccc08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3135643762 shutdown_connections 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3135643762 wait complete. 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- start start 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 Processor -- start 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4cccd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4cccdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccc5370 0x564a4ccbe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ac000 con 0x564a4ccbbb40 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ad470 con 0x564a4ccc5370 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.660+0000 7f0fcbbf3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ac6d0 con 0x564a4ccba5d0 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fcbbf3740 10 client.0 _ll_register_callbacks cb 0x7f0fbc009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4cccdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4cccdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50882/0 (socket says 172.21.15.64:50882) 2024-09-19T05:14:52.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 -- 172.21.15.64:0/3786437896 learned_addr learned my addr 172.21.15.64:0/3786437896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4cccd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccc5370 0x564a4ccbe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 -- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 msgr2=0x564a4cccd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4cccd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 -- 172.21.15.64:0/3786437896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccc5370 msgr2=0x564a4ccbe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccc5370 0x564a4ccbe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 -- 172.21.15.64:0/3786437896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4ccbf070 con 0x564a4ccbbb40 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fc37fe640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccc5370 0x564a4ccbe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:52.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fbbfff640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4cccdba0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0fac09c310 tx=0x7f0fac0d3680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5a9815ae06bb4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/3786437896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac09bec0 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/3786437896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f0fac010630 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fbc003d10 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/3786437896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fac0a0680 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fcbbf3740 10 client.17547 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.661+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a4cd3f280 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/3786437896 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fbc003d10 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/3786437896 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0fac0bc0b0 con 0x564a4ccbbb40 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fcbbf3740 10 client.17547 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fcbbf3740 10 client.17547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:14:52.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fcbbf3740 20 client.17547 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4cd3f800 0x564a4cd41bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fcbbf3740 4 client.17547 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a4cba4170 con 0x564a4cd3f800 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.662+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4cd3f800 0x564a4cd41bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fc2ffd640 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4cd3f800 0x564a4cd41bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e8a661a86426a52 server_cookie=e0da58493a4e98e1 in_seq=0 out_seq=0 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fa2ffd640 10 client.17547 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fa2ffd640 1 -- 172.21.15.64:0/3786437896 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a4cba4170 con 0x564a4cd3f800 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fa2ffd640 10 client.17547 handle_command_reply: tid=0 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 client.17547 shutdown 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 msgr2=0x564a4cccdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4cccdba0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0fac09c310 tx=0x7f0fac0d3680 comp rx=0 tx=0).stop 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 shutdown_connections 2024-09-19T05:14:52.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564a4cd3f800 0x564a4cd41bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a4ccc5370 0x564a4ccbe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a4ccbbb40 0x564a4cccdba0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 --2- 172.21.15.64:0/3786437896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a4ccba5d0 0x564a4cccd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fa2ffd640 0 client.17547 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 >> 172.21.15.64:0/3786437896 conn(0x564a4cc83310 msgr2=0x564a4ccc10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 shutdown_connections 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.663+0000 7f0fcbbf3740 1 -- 172.21.15.64:0/3786437896 wait complete. 2024-09-19T05:14:52.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fcbbf3740 20 client.17547 trim_cache size 0 max 16384 2024-09-19T05:14:52.686 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0fb00a33b0 msgr2=0x7f0fb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0fb00a33b0 0x7f0fb00a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005fd0 tx=0x7f0fbc009600 comp rx=0 tx=0).stop 2024-09-19T05:14:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 msgr2=0x7f0fc4198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 0x7f0fc4198890 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e9f0 tx=0x7f0fac00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 shutdown_connections 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f0fb00a33b0 0x7f0fb00a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc4106a80 0x7f0fc419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fc4101420 0x7f0fc4198890 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 --2- 172.21.15.64:0/2111829740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fc4100a80 0x7f0fc4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 >> 172.21.15.64:0/2111829740 conn(0x7f0fc40fc7a0 msgr2=0x7f0fc406ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 shutdown_connections 2024-09-19T05:14:52.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:52.664+0000 7f0fc981a640 1 -- 172.21.15.64:0/2111829740 wait complete. 2024-09-19T05:14:53.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:53 smithi064 ceph-mon[30960]: pgmap v276: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:53.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:53 smithi103 ceph-mon[34446]: pgmap v276: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:53.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:14:53.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:53 smithi174 ceph-mon[34488]: pgmap v276: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:53.731 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:54.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 -- 172.21.15.64:0/3086677797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898106f10 msgr2=0x7f989810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 --2- 172.21.15.64:0/3086677797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898106f10 0x7f989810ea90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9890009ca0 tx=0x7f989002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 -- 172.21.15.64:0/3086677797 shutdown_connections 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 --2- 172.21.15.64:0/3086677797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898106f10 0x7f989810ea90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 --2- 172.21.15.64:0/3086677797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898102790 0x7f98981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 --2- 172.21.15.64:0/3086677797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9898101de0 0x7f98981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 -- 172.21.15.64:0/3086677797 >> 172.21.15.64:0/3086677797 conn(0x7f98980fd8c0 msgr2=0x7f98980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 -- 172.21.15.64:0/3086677797 shutdown_connections 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.366+0000 7f989e814640 1 -- 172.21.15.64:0/3086677797 wait complete. 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 Processor -- start 2024-09-19T05:14:54.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 -- start start 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 0x7f9898198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9898102790 0x7f9898198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898106f10 0x7f989819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9898111e20 con 0x7f9898101de0 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9898111ca0 con 0x7f9898106f10 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.367+0000 7f989e814640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9898111fa0 con 0x7f9898102790 2024-09-19T05:14:54.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 0x7f9898198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 0x7f9898198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46586/0 (socket says 172.21.15.64:46586) 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 -- 172.21.15.64:0/3727958289 learned_addr learned my addr 172.21.15.64:0/3727958289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f98977fe640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9898102790 0x7f9898198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f989cdb1640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898106f10 0x7f989819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 -- 172.21.15.64:0/3727958289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9898102790 msgr2=0x7f9898198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9898102790 0x7f9898198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 -- 172.21.15.64:0/3727958289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898106f10 msgr2=0x7f989819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898106f10 0x7f989819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 -- 172.21.15.64:0/3727958289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989819d1f0 con 0x7f9898101de0 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f989cdb1640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898106f10 0x7f989819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f9897fff640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 0x7f9898198240 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f988800c920 tx=0x7f988800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:54.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f98957fa640 1 -- 172.21.15.64:0/3727958289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888007c70 con 0x7f9898101de0 2024-09-19T05:14:54.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f98957fa640 1 -- 172.21.15.64:0/3727958289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9888007e10 con 0x7f9898101de0 2024-09-19T05:14:54.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989819d4e0 con 0x7f9898101de0 2024-09-19T05:14:54.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f98957fa640 1 -- 172.21.15.64:0/3727958289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888018820 con 0x7f9898101de0 2024-09-19T05:14:54.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.368+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981a4dc0 con 0x7f9898101de0 2024-09-19T05:14:54.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.371+0000 7f98957fa640 1 -- 172.21.15.64:0/3727958289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f98880189c0 con 0x7f9898101de0 2024-09-19T05:14:54.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.371+0000 7f98a0bed740 1 Processor -- start 2024-09-19T05:14:54.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.371+0000 7f98a0bed740 1 -- start start 2024-09-19T05:14:54.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.371+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264148f50 0x55c2641474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c264147e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 0x55c264154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98a0bed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c2640a92f0 con 0x55c264152780 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98a0bed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c264050cf0 con 0x55c2641479e0 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98a0bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2640a52b0 con 0x55c264148f50 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 0x55c264154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 0x55c264154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46602/0 (socket says 172.21.15.64:46602) 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 -- 172.21.15.64:0/1451204276 learned_addr learned my addr 172.21.15.64:0/1451204276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98957fa640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f98800a33b0 0x7f98800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f9897fff640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c264147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f989cdb1640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264148f50 0x55c2641474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f989cdb1640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f98800a33b0 0x7f98800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98957fa640 1 -- 172.21.15.64:0/3727958289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f98880cfee0 con 0x7f9898101de0 2024-09-19T05:14:54.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 -- 172.21.15.64:0/1451204276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264148f50 msgr2=0x55c2641474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264148f50 0x55c2641474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 -- 172.21.15.64:0/1451204276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 msgr2=0x55c264147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c264147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f98977fe640 1 -- 172.21.15.64:0/1451204276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2640353a0 con 0x55c264152780 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f989cdb1640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264148f50 0x55c2641474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.372+0000 7f9897fff640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c264147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98977fe640 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 0x55c264154b70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f988c00bc30 tx=0x7f988c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e27f417ab0e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f989cdb1640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f98800a33b0 0x7f98800a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9890031040 tx=0x7f98900095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f987e7fc640 1 -- 172.21.15.64:0/1451204276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c01c480 con 0x55c264152780 2024-09-19T05:14:54.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f987e7fc640 1 -- 172.21.15.64:0/1451204276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c01cad0 con 0x55c264152780 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f987e7fc640 1 -- 172.21.15.64:0/1451204276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c011a40 con 0x55c264152780 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98a0bed740 1 -- 172.21.15.64:0/1451204276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 msgr2=0x55c264154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98a0bed740 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 0x55c264154b70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f988c00bc30 tx=0x7f988c0107a0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98a0bed740 1 -- 172.21.15.64:0/1451204276 shutdown_connections 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98a0bed740 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264152780 0x55c264154b70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98a0bed740 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c264147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.373+0000 7f98a0bed740 1 --2- 172.21.15.64:0/1451204276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264148f50 0x55c2641474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- 172.21.15.64:0/1451204276 >> 172.21.15.64:0/1451204276 conn(0x55c26410c310 msgr2=0x55c264144070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- 172.21.15.64:0/1451204276 shutdown_connections 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- 172.21.15.64:0/1451204276 wait complete. 2024-09-19T05:14:54.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- start start 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 Processor -- start 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2641479e0 0x55c264152160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c26414f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c264152780 0x55c26414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c264155480 con 0x55c264148f50 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c264064060 con 0x55c264152780 2024-09-19T05:14:54.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.374+0000 7f98a0bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c264155600 con 0x55c2641479e0 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98a0bed740 10 client.0 _ll_register_callbacks cb 0x55c263db78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c26414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c26414f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46606/0 (socket says 172.21.15.64:46606) 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 -- 172.21.15.64:0/57078580 learned_addr learned my addr 172.21.15.64:0/57078580 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f989cdb1640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c264152780 0x55c26414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f9897fff640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2641479e0 0x55c264152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 -- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2641479e0 msgr2=0x55c264152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2641479e0 0x55c264152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 -- 172.21.15.64:0/57078580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c264152780 msgr2=0x55c26414f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c264152780 0x55c26414f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 -- 172.21.15.64:0/57078580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c263e1a3c0 con 0x55c264148f50 2024-09-19T05:14:54.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98977fe640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c26414f410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b5f0 tx=0x7f988c01ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835a917e50a3dab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98777fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98777fe640 1 -- 172.21.15.64:0/57078580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c02b060 con 0x55c264148f50 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98777fe640 1 -- 172.21.15.64:0/57078580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f988c01c480 con 0x55c264148f50 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.375+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c01cad0 con 0x55c264148f50 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98777fe640 1 -- 172.21.15.64:0/57078580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f988c01c680 con 0x55c264148f50 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 10 client.17559 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c264160f00 con 0x55c264148f50 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98777fe640 1 -- 172.21.15.64:0/57078580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c01cad0 con 0x55c264148f50 2024-09-19T05:14:54.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98777fe640 1 -- 172.21.15.64:0/57078580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f988c011940 con 0x55c264148f50 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 10 client.17559 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 10 client.17559 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 20 client.17559 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c264162300 0x55c2641646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 4 client.17559 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2640b6860 con 0x55c264162300 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.376+0000 7f9897fff640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c264162300 0x55c2641646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.377+0000 7f9897fff640 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c264162300 0x55c2641646f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5747712cc9c8fef server_cookie=542cba9c386d63ae in_seq=0 out_seq=0 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.377+0000 7f98777fe640 10 client.17559 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.377+0000 7f98777fe640 1 -- 172.21.15.64:0/57078580 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2640b6860 con 0x55c264162300 2024-09-19T05:14:54.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.377+0000 7f98777fe640 10 client.17559 handle_command_reply: tid=0 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.377+0000 7f98a0bed740 1 client.17559 shutdown 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 msgr2=0x55c26414f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c26414f410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b5f0 tx=0x7f988c01ceb0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 shutdown_connections 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c264152780 0x55c26414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c264162300 0x55c2641646f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c26414f410 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 --2- 172.21.15.64:0/57078580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2641479e0 0x55c264152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 >> 172.21.15.64:0/57078580 conn(0x55c26410c310 msgr2=0x55c264149d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98777fe640 0 client.17559 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 shutdown_connections 2024-09-19T05:14:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 1 -- 172.21.15.64:0/57078580 wait complete. 2024-09-19T05:14:54.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.378+0000 7f98a0bed740 20 client.17559 trim_cache size 0 max 16384 2024-09-19T05:14:54.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 Processor -- start 2024-09-19T05:14:54.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 -- start start 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c26416a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98981adf20 con 0x55c264148f50 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98981ab7f0 con 0x55c2641479e0 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98a0bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98981a9660 con 0x55c264152780 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46622/0 (socket says 172.21.15.64:46622) 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 -- 172.21.15.64:0/2213813158 learned_addr learned my addr 172.21.15.64:0/2213813158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f989cdb1640 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c26416a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f9897fff640 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 -- 172.21.15.64:0/2213813158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 msgr2=0x55c26416a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c26416a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 -- 172.21.15.64:0/2213813158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 msgr2=0x55c26415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f98977fe640 1 -- 172.21.15.64:0/2213813158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2641442e0 con 0x55c264148f50 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.391+0000 7f989cdb1640 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c26416a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98977fe640 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264169cc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f988c010ca0 tx=0x7f988c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3920daabd365f79e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f987dffb640 1 -- 172.21.15.64:0/2213813158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c02b070 con 0x55c264148f50 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f987dffb640 1 -- 172.21.15.64:0/2213813158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c005a70 con 0x55c264148f50 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f987dffb640 1 -- 172.21.15.64:0/2213813158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c004020 con 0x55c264148f50 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 -- 172.21.15.64:0/2213813158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 msgr2=0x55c264169cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264169cc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f988c010ca0 tx=0x7f988c0107a0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 -- 172.21.15.64:0/2213813158 shutdown_connections 2024-09-19T05:14:54.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c26416a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264169cc0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 --2- 172.21.15.64:0/2213813158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 -- 172.21.15.64:0/2213813158 >> 172.21.15.64:0/2213813158 conn(0x55c26410c310 msgr2=0x55c26414e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 -- 172.21.15.64:0/2213813158 shutdown_connections 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.392+0000 7f98a0bed740 1 -- 172.21.15.64:0/2213813158 wait complete. 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 -- start start 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 Processor -- start 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264156090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c2641565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98981a9cb0 con 0x55c264148f50 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98981ab120 con 0x55c2641479e0 2024-09-19T05:14:54.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98981aa380 con 0x55c264152780 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98a0bed740 10 client.0 _ll_register_callbacks cb 0x7f988809bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46638/0 (socket says 172.21.15.64:46638) 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.393+0000 7f98977fe640 1 -- 172.21.15.64:0/3141618544 learned_addr learned my addr 172.21.15.64:0/3141618544 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f9897fff640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f989cdb1640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c2641565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98977fe640 1 -- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 msgr2=0x55c2641565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98977fe640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c2641565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98977fe640 1 -- 172.21.15.64:0/3141618544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 msgr2=0x55c26415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98977fe640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98977fe640 1 -- 172.21.15.64:0/3141618544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c264156b10 con 0x55c264148f50 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f989cdb1640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c2641565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:54.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98977fe640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264156090 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f988c011010 tx=0x7f988c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4726825b46159dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98777fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98777fe640 1 -- 172.21.15.64:0/3141618544 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c03c070 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98777fe640 1 -- 172.21.15.64:0/3141618544 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f988c01c480 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c005a70 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98777fe640 1 -- 172.21.15.64:0/3141618544 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f988c048050 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98a0bed740 10 client.17565 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.394+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2641cc9c0 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98777fe640 1 -- 172.21.15.64:0/3141618544 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c005a70 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98777fe640 1 -- 172.21.15.64:0/3141618544 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f988c003880 con 0x55c264148f50 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98a0bed740 10 client.17565 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98a0bed740 10 client.17565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98a0bed740 20 client.17565 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98a0bed740 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c2641cd000 0x55c2641cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:54.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98a0bed740 4 client.17565 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:54.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2640b0d70 con 0x55c2641cd000 2024-09-19T05:14:54.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f9897fff640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c2641cd000 0x55c2641cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f9897fff640 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c2641cd000 0x55c2641cf420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b1e34d7abc7e709 server_cookie=7e3483ebdab1afeb in_seq=0 out_seq=0 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.395+0000 7f98777fe640 10 client.17565 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98777fe640 1 -- 172.21.15.64:0/3141618544 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2640b0d70 con 0x55c2641cd000 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98777fe640 10 client.17565 handle_command_reply: tid=0 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 client.17565 shutdown 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 msgr2=0x55c264156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264156090 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f988c011010 tx=0x7f988c0119a0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 shutdown_connections 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c264152780 0x55c2641565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x55c2641cd000 0x55c2641cf420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c264148f50 0x55c264156090 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 --2- 172.21.15.64:0/3141618544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2641479e0 0x55c26415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98777fe640 0 client.17565 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 >> 172.21.15.64:0/3141618544 conn(0x55c26410c310 msgr2=0x55c26413e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 shutdown_connections 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 1 -- 172.21.15.64:0/3141618544 wait complete. 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.396+0000 7f98a0bed740 20 client.17565 trim_cache size 0 max 16384 2024-09-19T05:14:54.404 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f98800a33b0 msgr2=0x7f98800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f98800a33b0 0x7f98800a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9890031040 tx=0x7f98900095c0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 msgr2=0x7f9898198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 0x7f9898198240 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f988800c920 tx=0x7f988800cde0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 shutdown_connections 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f98800a33b0 0x7f98800a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9898106f10 0x7f989819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9898102790 0x7f9898198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 --2- 172.21.15.64:0/3727958289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898101de0 0x7f9898198240 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:54.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 >> 172.21.15.64:0/3727958289 conn(0x7f98980fd8c0 msgr2=0x7f98980fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:54.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 shutdown_connections 2024-09-19T05:14:54.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:54.397+0000 7f989e814640 1 -- 172.21.15.64:0/3727958289 wait complete. 2024-09-19T05:14:55.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:14:55.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:55 smithi064 ceph-mon[30960]: pgmap v277: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:55.413 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:55.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:55 smithi103 ceph-mon[34446]: pgmap v277: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:55.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:55 smithi174 ceph-mon[34488]: pgmap v277: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 -- 172.21.15.64:0/660055951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 msgr2=0x7f79f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/660055951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f810d950 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009cd0 tx=0x7f79f002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 -- 172.21.15.64:0/660055951 shutdown_connections 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/660055951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f810d950 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/660055951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f8100a80 0x7f79f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/660055951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f8106a80 0x7f79f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 -- 172.21.15.64:0/660055951 >> 172.21.15.64:0/660055951 conn(0x7f79f80fc7a0 msgr2=0x7f79f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 -- 172.21.15.64:0/660055951 shutdown_connections 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.890+0000 7f79ffdb7640 1 -- 172.21.15.64:0/660055951 wait complete. 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 Processor -- start 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 -- start start 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f8100a80 0x7f79f8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f8106a80 0x7f79f81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8110d30 con 0x7f79f8101420 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8110bb0 con 0x7f79f8106a80 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79ffdb7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8110eb0 con 0x7f79f8100a80 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79fd352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79fd352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f8102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46654/0 (socket says 172.21.15.64:46654) 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79fd352640 1 -- 172.21.15.64:0/1646043517 learned_addr learned my addr 172.21.15.64:0/1646043517 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.891+0000 7f79fe354640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f8106a80 0x7f79f81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fdb53640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f8100a80 0x7f79f8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fd352640 1 -- 172.21.15.64:0/1646043517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f8100a80 msgr2=0x7f79f8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fd352640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f8100a80 0x7f79f8102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fd352640 1 -- 172.21.15.64:0/1646043517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f8106a80 msgr2=0x7f79f81030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fd352640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f8106a80 0x7f79f81030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fd352640 1 -- 172.21.15.64:0/1646043517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f81a67e0 con 0x7f79f8101420 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79fd352640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f8102b90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f79e800d950 tx=0x7f79e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79eeffd640 1 -- 172.21.15.64:0/1646043517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8017070 con 0x7f79f8101420 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f81a6ad0 con 0x7f79f8101420 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79eeffd640 1 -- 172.21.15.64:0/1646043517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79e8009c30 con 0x7f79f8101420 2024-09-19T05:14:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79eeffd640 1 -- 172.21.15.64:0/1646043517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e800f620 con 0x7f79f8101420 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.892+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f81a7010 con 0x7f79f8101420 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.894+0000 7f79eeffd640 1 -- 172.21.15.64:0/1646043517 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f79e80109a0 con 0x7f79f8101420 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 Processor -- start 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 -- start start 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730b1370 0x5590730b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55907300c2f0 con 0x5590730a65d0 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559072fb3cf0 con 0x5590730b1370 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f7a02190740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590730082b0 con 0x5590730a7b40 2024-09-19T05:14:55.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f79fdb53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f79fdb53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46670/0 (socket says 172.21.15.64:46670) 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f79fdb53640 1 -- 172.21.15.64:0/3508110617 learned_addr learned my addr 172.21.15.64:0/3508110617 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f79fd352640 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730b1370 0x5590730b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.895+0000 7f79eeffd640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79e40a33b0 0x7f79e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fe354640 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fdb53640 1 -- 172.21.15.64:0/3508110617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 msgr2=0x5590730a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fdb53640 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fe354640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79e40a33b0 0x7f79e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fdb53640 1 -- 172.21.15.64:0/3508110617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730b1370 msgr2=0x5590730b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fdb53640 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730b1370 0x5590730b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fdb53640 1 -- 172.21.15.64:0/3508110617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559072f983a0 con 0x5590730a65d0 2024-09-19T05:14:55.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79eeffd640 1 -- 172.21.15.64:0/1646043517 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f79e80cf980 con 0x7f79f8101420 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fdb53640 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730a6a10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009bd0 tx=0x7f79f4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6fc91ae23cc6da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79e3fff640 1 -- 172.21.15.64:0/3508110617 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4004020 con 0x5590730a65d0 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79fe354640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79e40a33b0 0x7f79e40a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79f0005e10 tx=0x7f79f0005da0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79e3fff640 1 -- 172.21.15.64:0/3508110617 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f40041c0 con 0x5590730a65d0 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.896+0000 7f79e3fff640 1 -- 172.21.15.64:0/3508110617 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f40126b0 con 0x5590730a65d0 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 -- 172.21.15.64:0/3508110617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 msgr2=0x5590730a6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730a6a10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009bd0 tx=0x7f79f4010740 comp rx=0 tx=0).stop 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 -- 172.21.15.64:0/3508110617 shutdown_connections 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730b1370 0x5590730b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730a6a10 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 --2- 172.21.15.64:0/3508110617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 -- 172.21.15.64:0/3508110617 >> 172.21.15.64:0/3508110617 conn(0x55907306f310 msgr2=0x5590730a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 -- 172.21.15.64:0/3508110617 shutdown_connections 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.897+0000 7f7a02190740 1 -- 172.21.15.64:0/3508110617 wait complete. 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 -- start start 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 Processor -- start 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 0x5590730ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5590730b4070 con 0x5590730b1370 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559072fc7060 con 0x5590730a65d0 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590730b41f0 con 0x5590730a7b40 2024-09-19T05:14:55.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f7a02190740 10 client.0 _ll_register_callbacks cb 0x559072d1a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f79fe354640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 0x5590730ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f79fe354640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 0x5590730ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46682/0 (socket says 172.21.15.64:46682) 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f79fe354640 1 -- 172.21.15.64:0/3168885503 learned_addr learned my addr 172.21.15.64:0/3168885503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f79fd352640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.898+0000 7f79fdb53640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79fe354640 1 -- 172.21.15.64:0/3168885503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 msgr2=0x5590730ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79fe354640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79fe354640 1 -- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 msgr2=0x5590730b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79fe354640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79fe354640 1 -- 172.21.15.64:0/3168885503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559072d7d3c0 con 0x5590730b1370 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79fe354640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 0x5590730ae640 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f79f0035000 tx=0x7f79f002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971ef6e0874cb23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79e17fa640 1 -- 172.21.15.64:0/3168885503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0056060 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79e17fa640 1 -- 172.21.15.64:0/3168885503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f79f00305a0 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f40041c0 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79e17fa640 1 -- 172.21.15.64:0/3168885503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79f00307a0 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f7a02190740 10 client.17577 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590730bfb40 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.899+0000 7f79e17fa640 1 -- 172.21.15.64:0/3168885503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f40041c0 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f79e17fa640 1 -- 172.21.15.64:0/3168885503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79f0050030 con 0x5590730b1370 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f7a02190740 10 client.17577 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f7a02190740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f7a02190740 20 client.17577 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f7a02190740 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590730c0fa0 0x5590730c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f7a02190740 4 client.17577 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559073019860 con 0x5590730c0fa0 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f79fdb53640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590730c0fa0 0x5590730c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f79fdb53640 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590730c0fa0 0x5590730c3440 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711b76bcbfb4c5e6 server_cookie=ea987e2458a4e6d3 in_seq=0 out_seq=0 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.900+0000 7f79e17fa640 10 client.17577 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f79e17fa640 1 -- 172.21.15.64:0/3168885503 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559073019860 con 0x5590730c0fa0 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f79e17fa640 10 client.17577 handle_command_reply: tid=0 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 client.17577 shutdown 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 msgr2=0x5590730ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 0x5590730ae640 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f79f0035000 tx=0x7f79f002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 shutdown_connections 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5590730c0fa0 0x5590730c3440 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730b1370 0x5590730ae640 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730a7b40 0x5590730ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 --2- 172.21.15.64:0/3168885503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f79e17fa640 0 client.17577 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 >> 172.21.15.64:0/3168885503 conn(0x55907306f310 msgr2=0x5590730a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 shutdown_connections 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 1 -- 172.21.15.64:0/3168885503 wait complete. 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.901+0000 7f7a02190740 20 client.17577 trim_cache size 0 max 16384 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 Processor -- start 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 -- start start 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a7b40 0x5590730a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81b23d0 con 0x5590730a65d0 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81afca0 con 0x5590730a7b40 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f7a02190740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81adb10 con 0x5590730b1370 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f79fdb53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f79fdb53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46686/0 (socket says 172.21.15.64:46686) 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.915+0000 7f79fdb53640 1 -- 172.21.15.64:0/390469200 learned_addr learned my addr 172.21.15.64:0/390469200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fd352640 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a7b40 0x5590730a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fe354640 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fdb53640 1 -- 172.21.15.64:0/390469200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 msgr2=0x5590730a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fdb53640 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fdb53640 1 -- 172.21.15.64:0/390469200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a7b40 msgr2=0x5590730a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fdb53640 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a7b40 0x5590730a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fdb53640 1 -- 172.21.15.64:0/390469200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590730acfe0 con 0x5590730a65d0 2024-09-19T05:14:55.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fd352640 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a7b40 0x5590730a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79fdb53640 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730ad8f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f79f4010c40 tx=0x7f79f40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7387b4ba56b76c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79e37fe640 1 -- 172.21.15.64:0/390469200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f40035f0 con 0x5590730a65d0 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79e37fe640 1 -- 172.21.15.64:0/390469200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f40054e0 con 0x5590730a65d0 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.916+0000 7f79e37fe640 1 -- 172.21.15.64:0/390469200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4004a20 con 0x5590730a65d0 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 -- 172.21.15.64:0/390469200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 msgr2=0x5590730ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730ad8f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f79f4010c40 tx=0x7f79f40096a0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 -- 172.21.15.64:0/390469200 shutdown_connections 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a7b40 0x5590730a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 --2- 172.21.15.64:0/390469200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a65d0 0x5590730ad8f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 -- 172.21.15.64:0/390469200 >> 172.21.15.64:0/390469200 conn(0x55907306f310 msgr2=0x5590730a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:55.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 -- 172.21.15.64:0/390469200 shutdown_connections 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 -- 172.21.15.64:0/390469200 wait complete. 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 -- start start 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.917+0000 7f7a02190740 1 Processor -- start 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 0x5590730b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81ae160 con 0x5590730a7b40 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81af5d0 con 0x5590730a65d0 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81ae830 con 0x5590730b1370 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f7a02190740 10 client.0 _ll_register_callbacks cb 0x7f79f0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 0x5590730b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 0x5590730b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46688/0 (socket says 172.21.15.64:46688) 2024-09-19T05:14:55.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 -- 172.21.15.64:0/1922449398 learned_addr learned my addr 172.21.15.64:0/1922449398 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fe354640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fdb53640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 -- 172.21.15.64:0/1922449398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 msgr2=0x5590730b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 -- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 msgr2=0x5590730b9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fd352640 1 -- 172.21.15.64:0/1922449398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590730b5780 con 0x5590730a7b40 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.918+0000 7f79fdb53640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79fd352640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 0x5590730b4c80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f79e80cd100 tx=0x7f79e8097db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d9f1d9e3c4eaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79e17fa640 1 -- 172.21.15.64:0/1922449398 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8020040 con 0x5590730a7b40 2024-09-19T05:14:55.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79e17fa640 1 -- 172.21.15.64:0/1922449398 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f79e8097980 con 0x5590730a7b40 2024-09-19T05:14:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f40054e0 con 0x5590730a7b40 2024-09-19T05:14:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79e17fa640 1 -- 172.21.15.64:0/1922449398 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79e809bc30 con 0x5590730a7b40 2024-09-19T05:14:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 10 client.17583 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55907312ba50 con 0x5590730a7b40 2024-09-19T05:14:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79e17fa640 1 -- 172.21.15.64:0/1922449398 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f40054e0 con 0x5590730a7b40 2024-09-19T05:14:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f79e17fa640 1 -- 172.21.15.64:0/1922449398 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79e807f0c0 con 0x5590730a7b40 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 10 client.17583 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 20 client.17583 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55907312c250 0x55907312e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 4 client.17583 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.919+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559073013d70 con 0x55907312c250 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f79fdb53640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55907312c250 0x55907312e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f79fdb53640 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55907312c250 0x55907312e670 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d6da8dcbf49fc9 server_cookie=b1502b0447327fdf in_seq=0 out_seq=0 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f79e17fa640 10 client.17583 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f79e17fa640 1 -- 172.21.15.64:0/1922449398 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559073013d70 con 0x55907312c250 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f79e17fa640 10 client.17583 handle_command_reply: tid=0 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 client.17583 shutdown 2024-09-19T05:14:55.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 msgr2=0x5590730b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 0x5590730b4c80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f79e80cd100 tx=0x7f79e8097db0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 shutdown_connections 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590730b1370 0x5590730b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55907312c250 0x55907312e670 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590730a7b40 0x5590730b4c80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 --2- 172.21.15.64:0/1922449398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590730a65d0 0x5590730b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f79e17fa640 0 client.17583 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.920+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 >> 172.21.15.64:0/1922449398 conn(0x55907306f310 msgr2=0x5590730b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 shutdown_connections 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f7a02190740 1 -- 172.21.15.64:0/1922449398 wait complete. 2024-09-19T05:14:55.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f7a02190740 20 client.17583 trim_cache size 0 max 16384 2024-09-19T05:14:55.943 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:14:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79e40a33b0 msgr2=0x7f79e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79e40a33b0 0x7f79e40a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79f0005e10 tx=0x7f79f0005da0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 msgr2=0x7f79f8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f8102b90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f79e800d950 tx=0x7f79e800de10 comp rx=0 tx=0).stop 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 shutdown_connections 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f79e40a33b0 0x7f79e40a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79f8106a80 0x7f79f81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79f8101420 0x7f79f8102b90 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 --2- 172.21.15.64:0/1646043517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79f8100a80 0x7f79f8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 >> 172.21.15.64:0/1646043517 conn(0x7f79f80fc7a0 msgr2=0x7f79f806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 shutdown_connections 2024-09-19T05:14:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:55.921+0000 7f79ffdb7640 1 -- 172.21.15.64:0/1646043517 wait complete. 2024-09-19T05:14:57.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:57 smithi064 ceph-mon[30960]: pgmap v278: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:57.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:57 smithi103 ceph-mon[34446]: pgmap v278: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:57.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:57 smithi174 ceph-mon[34488]: pgmap v278: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:14:57.660 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:14:57.660 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:14:57.932 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.374+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/717515617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 msgr2=0x7fbe9410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.374+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/717515617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe9410d950 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009930 tx=0x7fbe7c02a420 comp rx=0 tx=0).stop 2024-09-19T05:14:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.374+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/717515617 shutdown_connections 2024-09-19T05:14:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/717515617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe9410d950 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/717515617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe94100a80 0x7fbe94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/717515617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe94106a80 0x7fbe94106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/717515617 >> 172.21.15.64:0/717515617 conn(0x7fbe940fc7a0 msgr2=0x7fbe940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/717515617 shutdown_connections 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/717515617 wait complete. 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 Processor -- start 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 -- start start 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.375+0000 7fbe9ad8e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe94100a80 0x7fbe94198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe9ad8e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe941988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe9ad8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe94106a80 0x7fbe9419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe941988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe941988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46692/0 (socket says 172.21.15.64:46692) 2024-09-19T05:14:58.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 -- 172.21.15.64:0/970753709 learned_addr learned my addr 172.21.15.64:0/970753709 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94111e50 con 0x7fbe94101420 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94111cd0 con 0x7fbe94106a80 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94111fd0 con 0x7fbe94100a80 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe94100a80 0x7fbe94198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe9932b640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe94106a80 0x7fbe9419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 -- 172.21.15.64:0/970753709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe94100a80 msgr2=0x7fbe94198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe94100a80 0x7fbe94198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 -- 172.21.15.64:0/970753709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe94106a80 msgr2=0x7fbe9419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe94106a80 0x7fbe9419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 -- 172.21.15.64:0/970753709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9419d320 con 0x7fbe94101420 2024-09-19T05:14:58.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.376+0000 7fbe93fff640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe941988b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d950 tx=0x7fbe8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.377+0000 7fbe91ffb640 1 -- 172.21.15.64:0/970753709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84017070 con 0x7fbe94101420 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.377+0000 7fbe91ffb640 1 -- 172.21.15.64:0/970753709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe84009c30 con 0x7fbe94101420 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.377+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9419d610 con 0x7fbe94101420 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.377+0000 7fbe91ffb640 1 -- 172.21.15.64:0/970753709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400f620 con 0x7fbe94101420 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.377+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941a4ef0 con 0x7fbe94101420 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.379+0000 7fbe9d167740 1 Processor -- start 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 -- start start 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 0x561795847f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795852800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795855130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5617957ad280 con 0x561795847b60 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561795754b10 con 0x561795852d40 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9d167740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617957a9240 con 0x561795848480 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 0x561795847f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 0x561795847f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46704/0 (socket says 172.21.15.64:46704) 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 -- 172.21.15.64:0/2595620872 learned_addr learned my addr 172.21.15.64:0/2595620872 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe91ffb640 1 -- 172.21.15.64:0/970753709 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fbe840109a0 con 0x7fbe94101420 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795852800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe93fff640 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795855130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 -- 172.21.15.64:0/2595620872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 msgr2=0x561795852800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795852800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 -- 172.21.15.64:0/2595620872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 msgr2=0x561795855130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795855130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe9932b640 1 -- 172.21.15.64:0/2595620872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617957611f0 con 0x561795847b60 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.380+0000 7fbe93fff640 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795855130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe9932b640 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 0x561795847f40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009230 tx=0x7fbe7c032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6258449f98d7721 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe8affd640 1 -- 172.21.15.64:0/2595620872 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c03f070 con 0x561795847b60 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe8affd640 1 -- 172.21.15.64:0/2595620872 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe7c0316a0 con 0x561795847b60 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe91ffb640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbe800a33b0 0x7fbe800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe8affd640 1 -- 172.21.15.64:0/2595620872 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c035070 con 0x561795847b60 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbe800a33b0 0x7fbe800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.381+0000 7fbe91ffb640 1 -- 172.21.15.64:0/970753709 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fbe840cf9a0 con 0x7fbe94101420 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 -- 172.21.15.64:0/2595620872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 msgr2=0x561795847f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbe800a33b0 0x7fbe800a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x561795844f50 tx=0x7fbe8c00d040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 0x561795847f40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009230 tx=0x7fbe7c032e50 comp rx=0 tx=0).stop 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 -- 172.21.15.64:0/2595620872 shutdown_connections 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795855130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795852800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 --2- 172.21.15.64:0/2595620872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795847b60 0x561795847f40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 -- 172.21.15.64:0/2595620872 >> 172.21.15.64:0/2595620872 conn(0x5617958440f0 msgr2=0x5617958444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 -- 172.21.15.64:0/2595620872 shutdown_connections 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 -- 172.21.15.64:0/2595620872 wait complete. 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.382+0000 7fbe9d167740 1 -- start start 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 Processor -- start 2024-09-19T05:14:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x56179584fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795850170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561795855910 con 0x561795852d40 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561795767f10 con 0x561795847b60 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561795855a90 con 0x561795848480 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9d167740 10 client.0 _ll_register_callbacks cb 0x561795730a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9932b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9932b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46718/0 (socket says 172.21.15.64:46718) 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe9932b640 1 -- 172.21.15.64:0/3758270691 learned_addr learned my addr 172.21.15.64:0/3758270691 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe93fff640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795850170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.383+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x56179584fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9932b640 1 -- 172.21.15.64:0/3758270691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 msgr2=0x561795850170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795850170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9932b640 1 -- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 msgr2=0x56179584fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x56179584fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9932b640 1 -- 172.21.15.64:0/3758270691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179551e6a0 con 0x561795852d40 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x56179584fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958506b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c031a60 tx=0x7fbe7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93775124af3ce8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe73fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe73fff640 1 -- 172.21.15.64:0/3758270691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c03f070 con 0x561795852d40 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe73fff640 1 -- 172.21.15.64:0/3758270691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fbe7c004440 con 0x561795852d40 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe7c0316a0 con 0x561795852d40 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe73fff640 1 -- 172.21.15.64:0/3758270691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe7c054050 con 0x561795852d40 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9d167740 10 client.17595 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561795861490 con 0x561795852d40 2024-09-19T05:14:58.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.384+0000 7fbe73fff640 1 -- 172.21.15.64:0/3758270691 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe7c0316a0 con 0x561795852d40 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe73fff640 1 -- 172.21.15.64:0/3758270691 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe7c004a20 con 0x561795852d40 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe9d167740 10 client.17595 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe9d167740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe9d167740 20 client.17595 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958627e0 0x561795864bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe9d167740 4 client.17595 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617957ba7f0 con 0x5617958627e0 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958627e0 0x561795864bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958627e0 0x561795864bd0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c0c2c9636937f0 server_cookie=e2ec40827d512820 in_seq=0 out_seq=0 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.385+0000 7fbe73fff640 10 client.17595 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe73fff640 1 -- 172.21.15.64:0/3758270691 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617957ba7f0 con 0x5617958627e0 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe73fff640 10 client.17595 handle_command_reply: tid=0 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 client.17595 shutdown 2024-09-19T05:14:58.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 msgr2=0x5617958506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958506b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c031a60 tx=0x7fbe7c009260 comp rx=0 tx=0).stop 2024-09-19T05:14:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 shutdown_connections 2024-09-19T05:14:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958627e0 0x561795864bd0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958506b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795850170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3758270691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x56179584fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe73fff640 0 client.17595 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:58.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.386+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 >> 172.21.15.64:0/3758270691 conn(0x5617958440f0 msgr2=0x561795843270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:58.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.387+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 shutdown_connections 2024-09-19T05:14:58.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.387+0000 7fbe9d167740 1 -- 172.21.15.64:0/3758270691 wait complete. 2024-09-19T05:14:58.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.387+0000 7fbe9d167740 20 client.17595 trim_cache size 0 max 16384 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 Processor -- start 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 -- start start 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795847b60 0x56179584eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 0x561795867bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795868130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941b0200 con 0x561795848480 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941adad0 con 0x561795852d40 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.410+0000 7fbe9d167740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941ab940 con 0x561795847b60 2024-09-19T05:14:58.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 0x561795867bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 0x561795867bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46724/0 (socket says 172.21.15.64:46724) 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 -- 172.21.15.64:0/3941863503 learned_addr learned my addr 172.21.15.64:0/3941863503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795868130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795847b60 0x56179584eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 -- 172.21.15.64:0/3941863503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795847b60 msgr2=0x56179584eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795847b60 0x56179584eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 -- 172.21.15.64:0/3941863503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 msgr2=0x561795868130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795868130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 -- 172.21.15.64:0/3941863503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561795852210 con 0x561795848480 2024-09-19T05:14:58.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795847b60 0x56179584eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795868130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe93fff640 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 0x561795867bf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbe840cd6c0 tx=0x7fbe84097ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80de4a7700618d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe8a7fc640 1 -- 172.21.15.64:0/3941863503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840b1070 con 0x561795848480 2024-09-19T05:14:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe8a7fc640 1 -- 172.21.15.64:0/3941863503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe840cfe50 con 0x561795848480 2024-09-19T05:14:58.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.411+0000 7fbe8a7fc640 1 -- 172.21.15.64:0/3941863503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84020070 con 0x561795848480 2024-09-19T05:14:58.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 -- 172.21.15.64:0/3941863503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 msgr2=0x561795867bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 0x561795867bf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbe840cd6c0 tx=0x7fbe84097ef0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 -- 172.21.15.64:0/3941863503 shutdown_connections 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795852d40 0x561795868130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795848480 0x561795867bf0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3941863503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795847b60 0x56179584eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 -- 172.21.15.64:0/3941863503 >> 172.21.15.64:0/3941863503 conn(0x5617958440f0 msgr2=0x56179584e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 -- 172.21.15.64:0/3941863503 shutdown_connections 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.412+0000 7fbe9d167740 1 -- 172.21.15.64:0/3941863503 wait complete. 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 -- start start 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 Processor -- start 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x561795865a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795865fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941abf90 con 0x561795852d40 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941ad400 con 0x561795847b60 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941ac660 con 0x561795848480 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9d167740 10 client.0 _ll_register_callbacks cb 0x7fbe7c032d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9932b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9932b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46728/0 (socket says 172.21.15.64:46728) 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe9932b640 1 -- 172.21.15.64:0/3193368571 learned_addr learned my addr 172.21.15.64:0/3193368571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x561795865a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.413+0000 7fbe93fff640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795865fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9932b640 1 -- 172.21.15.64:0/3193368571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 msgr2=0x561795865fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795865fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9932b640 1 -- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 msgr2=0x561795865a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x561795865a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9932b640 1 -- 172.21.15.64:0/3193368571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617958d0630 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9932b640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958664e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c005780 tx=0x7fbe7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97c12e35b8b4fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe73fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe73fff640 1 -- 172.21.15.64:0/3193368571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c03f040 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe73fff640 1 -- 172.21.15.64:0/3193368571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fbe7c004240 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe840cfe50 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe73fff640 1 -- 172.21.15.64:0/3193368571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe7c066050 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9d167740 10 client.17601 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.414+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617958d18a0 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe73fff640 1 -- 172.21.15.64:0/3193368571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe840cfe50 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe73fff640 1 -- 172.21.15.64:0/3193368571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe7c004a90 con 0x561795852d40 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe9d167740 10 client.17601 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:58.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe9d167740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe9d167740 20 client.17601 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958d1fb0 0x5617958d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe9d167740 4 client.17601 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5617957b4d00 con 0x5617958d1fb0 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958d1fb0 0x5617958d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe98b2a640 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958d1fb0 0x5617958d43d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916d07395438e3d6 server_cookie=9107a31954ad922e in_seq=0 out_seq=0 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.415+0000 7fbe73fff640 10 client.17601 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.416+0000 7fbe73fff640 1 -- 172.21.15.64:0/3193368571 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5617957b4d00 con 0x5617958d1fb0 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.416+0000 7fbe73fff640 10 client.17601 handle_command_reply: tid=0 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 client.17601 shutdown 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 msgr2=0x5617958664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958664e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c005780 tx=0x7fbe7c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 shutdown_connections 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5617958d1fb0 0x5617958d43d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe73fff640 0 client.17601 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561795852d40 0x5617958664e0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795848480 0x561795865fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 --2- 172.21.15.64:0/3193368571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561795847b60 0x561795865a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 >> 172.21.15.64:0/3193368571 conn(0x5617958440f0 msgr2=0x56179585e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 shutdown_connections 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 1 -- 172.21.15.64:0/3193368571 wait complete. 2024-09-19T05:14:58.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.417+0000 7fbe9d167740 20 client.17601 trim_cache size 0 max 16384 2024-09-19T05:14:58.427 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:14:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "fe36a8f8-6f42-48fc-b545-14c984406b4d", 2024-09-19T05:14:58.427 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:14:58.427 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbe800a33b0 msgr2=0x7fbe800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbe800a33b0 0x7fbe800a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x561795844f50 tx=0x7fbe8c00d040 comp rx=0 tx=0).stop 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 msgr2=0x7fbe941988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe941988b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d950 tx=0x7fbe8400de10 comp rx=0 tx=0).stop 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 shutdown_connections 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbe800a33b0 0x7fbe800a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe94106a80 0x7fbe9419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe94101420 0x7fbe941988b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 --2- 172.21.15.64:0/970753709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe94100a80 0x7fbe94198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:58.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 >> 172.21.15.64:0/970753709 conn(0x7fbe940fc7a0 msgr2=0x7fbe940fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:58.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 shutdown_connections 2024-09-19T05:14:58.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:58.418+0000 7fbe9ad8e640 1 -- 172.21.15.64:0/970753709 wait complete. 2024-09-19T05:14:59.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:59.350 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:59.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:59 smithi064 ceph-mon[30960]: pgmap v279: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:59 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:14:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:59 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:59 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:14:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:59 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:59.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:14:59 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:14:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:59 smithi103 ceph-mon[34446]: pgmap v279: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:59 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:14:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:59 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:59 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:14:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:59 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:59.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:14:59 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:14:59.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:59 smithi174 ceph-mon[34488]: pgmap v279: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:14:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:59 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:14:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:59 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:59 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:14:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:59 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:59.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:14:59 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 -- 172.21.15.64:0/363009321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 msgr2=0x7f6ea810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 --2- 172.21.15.64:0/363009321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea810ea90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009ca0 tx=0x7f6ea002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 -- 172.21.15.64:0/363009321 shutdown_connections 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 --2- 172.21.15.64:0/363009321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea810ea90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 --2- 172.21.15.64:0/363009321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8102790 0x7f6ea81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 --2- 172.21.15.64:0/363009321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8101de0 0x7f6ea81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 -- 172.21.15.64:0/363009321 >> 172.21.15.64:0/363009321 conn(0x7f6ea80fd8c0 msgr2=0x7f6ea80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 -- 172.21.15.64:0/363009321 shutdown_connections 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.949+0000 7f6eafb26640 1 -- 172.21.15.64:0/363009321 wait complete. 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 Processor -- start 2024-09-19T05:14:59.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 -- start start 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8101de0 0x7f6ea80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102790 0x7f6ea8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8111e40 con 0x7f6ea8106f10 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8111cc0 con 0x7f6ea8101de0 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.950+0000 7f6eafb26640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8111fc0 con 0x7f6ea8102790 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46740/0 (socket says 172.21.15.64:46740) 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 -- 172.21.15.64:0/806446109 learned_addr learned my addr 172.21.15.64:0/806446109 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8101de0 0x7f6ea80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 -- 172.21.15.64:0/806446109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102790 msgr2=0x7f6ea8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102790 0x7f6ea8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102790 0x7f6ea8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 -- 172.21.15.64:0/806446109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8101de0 msgr2=0x7f6ea80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8101de0 0x7f6ea80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 -- 172.21.15.64:0/806446109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81a6820 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102790 0x7f6ea8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea8074f70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009dc0 tx=0x7f6ea0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6e9affd640 1 -- 172.21.15.64:0/806446109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0004040 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6e9affd640 1 -- 172.21.15.64:0/806446109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea0004690 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81a6ab0 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6e9affd640 1 -- 172.21.15.64:0/806446109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0030c90 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.951+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a6ff0 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6e9affd640 1 -- 172.21.15.64:0/806446109 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f6ea00041e0 con 0x7f6ea8106f10 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 Processor -- start 2024-09-19T05:14:59.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 -- start start 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d8101140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c2d805c2e0 con 0x55c2d80f64a0 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8003ce0 con 0x55c2d8101680 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.954+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d80582a0 con 0x55c2d80f6dc0 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46744/0 (socket says 172.21.15.64:46744) 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 -- 172.21.15.64:0/321186511 learned_addr learned my addr 172.21.15.64:0/321186511 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d8101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 -- 172.21.15.64:0/321186511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 msgr2=0x55c2d8101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d8101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 -- 172.21.15.64:0/321186511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 msgr2=0x55c2d8103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d8103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6e9affd640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6e940a33b0 0x7f6e940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 -- 172.21.15.64:0/321186511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d7fe8440 con 0x55c2d80f64a0 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6e940a33b0 0x7f6e940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80f6880 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c014540 tx=0x7f6e9c00b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3f4d4e22dceb77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6e9affd640 1 -- 172.21.15.64:0/806446109 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f6ea00f1460 con 0x7f6ea8106f10 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6e93fff640 1 -- 172.21.15.64:0/321186511 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c012710 con 0x55c2d80f64a0 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6e93fff640 1 -- 172.21.15.64:0/321186511 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9c01a070 con 0x55c2d80f64a0 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.955+0000 7f6e93fff640 1 -- 172.21.15.64:0/321186511 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00eab0 con 0x55c2d80f64a0 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 -- 172.21.15.64:0/321186511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 msgr2=0x55c2d80f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80f6880 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c014540 tx=0x7f6e9c00b530 comp rx=0 tx=0).stop 2024-09-19T05:14:59.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6e940a33b0 0x7f6e940a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55c2d80f3890 tx=0x7f6ea4008040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 -- 172.21.15.64:0/321186511 shutdown_connections 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d8103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/321186511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80f6880 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 -- 172.21.15.64:0/321186511 >> 172.21.15.64:0/321186511 conn(0x55c2d80f2a30 msgr2=0x55c2d80f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 -- 172.21.15.64:0/321186511 shutdown_connections 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.956+0000 7f6eb1eff740 1 -- 172.21.15.64:0/321186511 wait complete. 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 -- start start 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 Processor -- start 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8101020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d80fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d80fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8104250 con 0x55c2d80f64a0 2024-09-19T05:14:59.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8017050 con 0x55c2d8101680 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d81043d0 con 0x55c2d80f6dc0 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eb1eff740 10 client.0 _ll_register_callbacks cb 0x55c2d8057160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6ead8c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6ead8c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8101020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46760/0 (socket says 172.21.15.64:46760) 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6ead8c2640 1 -- 172.21.15.64:0/105603786 learned_addr learned my addr 172.21.15.64:0/105603786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.957+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d80fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead8c2640 1 -- 172.21.15.64:0/105603786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 msgr2=0x55c2d80fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d80fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d80fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead8c2640 1 -- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 msgr2=0x55c2d80fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d80fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead8c2640 1 -- 172.21.15.64:0/105603786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d7dcd3c0 con 0x55c2d80f64a0 2024-09-19T05:14:59.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d80fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8101020 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4015710 tx=0x7f6ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca1582996c3c8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6e917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6e917fa640 1 -- 172.21.15.64:0/105603786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea401c050 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6e917fa640 1 -- 172.21.15.64:0/105603786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f6ea40169f0 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9c01a070 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6e917fa640 1 -- 172.21.15.64:0/105603786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ea402f050 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6eb1eff740 10 client.17613 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2d810fcf0 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.958+0000 7f6e917fa640 1 -- 172.21.15.64:0/105603786 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9c01a070 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6e917fa640 1 -- 172.21.15.64:0/105603786 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ea40143d0 con 0x55c2d80f64a0 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6eb1eff740 10 client.17613 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6eb1eff740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:59.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6eb1eff740 20 client.17613 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d8111670 0x55c2d8113af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6eb1eff740 4 client.17613 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2d8069850 con 0x55c2d8111670 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d8111670 0x55c2d8113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d8111670 0x55c2d8113af0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275cd6fd62da8974 server_cookie=386196ad3c5113c9 in_seq=0 out_seq=0 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.959+0000 7f6e917fa640 10 client.17613 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6e917fa640 1 -- 172.21.15.64:0/105603786 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2d8069850 con 0x55c2d8111670 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6e917fa640 10 client.17613 handle_command_reply: tid=0 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 client.17613 shutdown 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 msgr2=0x55c2d8101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8101020 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4015710 tx=0x7f6ea40046c0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 shutdown_connections 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d8101680 0x55c2d80fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d80f6dc0 0x55c2d80fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d8111670 0x55c2d8113af0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/105603786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8101020 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6e917fa640 0 client.17613 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.960+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 >> 172.21.15.64:0/105603786 conn(0x55c2d80f2a30 msgr2=0x55c2d80f8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.961+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 shutdown_connections 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.961+0000 7f6eb1eff740 1 -- 172.21.15.64:0/105603786 wait complete. 2024-09-19T05:14:59.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.961+0000 7f6eb1eff740 20 client.17613 trim_cache size 0 max 16384 2024-09-19T05:14:59.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.978+0000 7f6eb1eff740 1 Processor -- start 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.978+0000 7f6eb1eff740 1 -- start start 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d80f1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d80f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81b2370 con 0x55c2d80f64a0 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81afc40 con 0x55c2d80f6dc0 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81adab0 con 0x55c2d8101680 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46774/0 (socket says 172.21.15.64:46774) 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 -- 172.21.15.64:0/503229934 learned_addr learned my addr 172.21.15.64:0/503229934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:59.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d80f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 -- 172.21.15.64:0/503229934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 msgr2=0x55c2d80f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d80f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d80f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 -- 172.21.15.64:0/503229934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 msgr2=0x55c2d80f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d80f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 -- 172.21.15.64:0/503229934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d80fd280 con 0x55c2d80f64a0 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d80f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80fcba0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4011010 tx=0x7f6ea4009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8585752fe38d82bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.979+0000 7f6e937fe640 1 -- 172.21.15.64:0/503229934 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea401c040 con 0x55c2d80f64a0 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6e937fe640 1 -- 172.21.15.64:0/503229934 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea4026070 con 0x55c2d80f64a0 2024-09-19T05:14:59.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6e937fe640 1 -- 172.21.15.64:0/503229934 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400f040 con 0x55c2d80f64a0 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 -- 172.21.15.64:0/503229934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 msgr2=0x55c2d80fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80fcba0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4011010 tx=0x7f6ea4009110 comp rx=0 tx=0).stop 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 -- 172.21.15.64:0/503229934 shutdown_connections 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d80f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d80f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/503229934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d80fcba0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 -- 172.21.15.64:0/503229934 >> 172.21.15.64:0/503229934 conn(0x55c2d80f2a30 msgr2=0x55c2d80f1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 -- 172.21.15.64:0/503229934 shutdown_connections 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.980+0000 7f6eb1eff740 1 -- 172.21.15.64:0/503229934 wait complete. 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 -- start start 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 Processor -- start 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8109c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d8106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d81074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae100 con 0x55c2d80f64a0 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81af570 con 0x55c2d80f6dc0 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae7d0 con 0x55c2d8101680 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eb1eff740 10 client.0 _ll_register_callbacks cb 0x7f6ea40096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6ead8c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6ead8c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46782/0 (socket says 172.21.15.64:46782) 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6ead8c2640 1 -- 172.21.15.64:0/1326731304 learned_addr learned my addr 172.21.15.64:0/1326731304 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6eae0c3640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d81074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.981+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d8106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6ead8c2640 1 -- 172.21.15.64:0/1326731304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 msgr2=0x55c2d81074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d81074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6ead8c2640 1 -- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 msgr2=0x55c2d8106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d8106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6ead8c2640 1 -- 172.21.15.64:0/1326731304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d80f6bf0 con 0x55c2d80f64a0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6ead8c2640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8109c50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4015040 tx=0x7f6ea4006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecca98161ea1b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6e917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6e917fa640 1 -- 172.21.15.64:0/1326731304 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4043020 con 0x55c2d80f64a0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea4026070 con 0x55c2d80f64a0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6e917fa640 1 -- 172.21.15.64:0/1326731304 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f6ea4014230 con 0x55c2d80f64a0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6e917fa640 1 -- 172.21.15.64:0/1326731304 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ea404e050 con 0x55c2d80f64a0 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6eb1eff740 10 client.17619 fetch_fsmap learned FSMap version 25 2024-09-19T05:14:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.982+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2d8101390 con 0x55c2d80f64a0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6e917fa640 1 -- 172.21.15.64:0/1326731304 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea4026070 con 0x55c2d80f64a0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6e917fa640 1 -- 172.21.15.64:0/1326731304 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ea403f500 con 0x55c2d80f64a0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6eb1eff740 10 client.17619 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6eb1eff740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6eb1eff740 20 client.17619 populate_metadata read hostname 'smithi064' 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d817c4c0 0x55c2d817e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6eb1eff740 4 client.17619 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2d8063d60 con 0x55c2d817c4c0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d817c4c0 0x55c2d817e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6ead0c1640 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d817c4c0 0x55c2d817e8b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0d795a0d01e43b server_cookie=6d4d9092c9687a35 in_seq=0 out_seq=0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.983+0000 7f6e917fa640 10 client.17619 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6e917fa640 1 -- 172.21.15.64:0/1326731304 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c2d8063d60 con 0x55c2d817c4c0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6e917fa640 10 client.17619 handle_command_reply: tid=0 2024-09-19T05:14:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 client.17619 shutdown 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 msgr2=0x55c2d8109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8109c50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4015040 tx=0x7f6ea4006c30 comp rx=0 tx=0).stop 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 shutdown_connections 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8101680 0x55c2d81074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2d80f6dc0 0x55c2d8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55c2d817c4c0 0x55c2d817e8b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 --2- 172.21.15.64:0/1326731304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c2d80f64a0 0x55c2d8109c50 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6e917fa640 0 client.17619 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 >> 172.21.15.64:0/1326731304 conn(0x55c2d80f2a30 msgr2=0x55c2d80f1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 shutdown_connections 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 1 -- 172.21.15.64:0/1326731304 wait complete. 2024-09-19T05:14:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.984+0000 7f6eb1eff740 20 client.17619 trim_cache size 0 max 16384 2024-09-19T05:14:59.996 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:14:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:14:59.996 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:14:59.996 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6e940a33b0 msgr2=0x7f6e940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6e940a33b0 0x7f6e940a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55c2d80f3890 tx=0x7f6ea4008040 comp rx=0 tx=0).stop 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 msgr2=0x7f6ea8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea8074f70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009dc0 tx=0x7f6ea0003ba0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 shutdown_connections 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f6e940a33b0 0x7f6e940a5870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ea8106f10 0x7f6ea8074f70 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea8102790 0x7f6ea8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 --2- 172.21.15.64:0/806446109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ea8101de0 0x7f6ea80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 >> 172.21.15.64:0/806446109 conn(0x7f6ea80fd8c0 msgr2=0x7f6ea80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 shutdown_connections 2024-09-19T05:14:59.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:14:59.985+0000 7f6eafb26640 1 -- 172.21.15.64:0/806446109 wait complete. 2024-09-19T05:15:00.741 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:15:00.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:15:01.006 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:01.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:01 smithi064 ceph-mon[30960]: pgmap v280: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:15:01.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:01 smithi103 ceph-mon[34446]: pgmap v280: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:15:01.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:01 smithi174 ceph-mon[34488]: pgmap v280: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2024-09-19T05:15:01.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 -- 172.21.15.64:0/2721889381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0102d40 msgr2=0x7fdff0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 --2- 172.21.15.64:0/2721889381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0102d40 0x7fdff0103120 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009900 tx=0x7fdfe002a480 comp rx=0 tx=0).stop 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 -- 172.21.15.64:0/2721889381 shutdown_connections 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 --2- 172.21.15.64:0/2721889381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff010c170 0x7fdff010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 --2- 172.21.15.64:0/2721889381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdff0103660 0x7fdff010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 --2- 172.21.15.64:0/2721889381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0102d40 0x7fdff0103120 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 -- 172.21.15.64:0/2721889381 >> 172.21.15.64:0/2721889381 conn(0x7fdff00fc7a0 msgr2=0x7fdff00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 -- 172.21.15.64:0/2721889381 shutdown_connections 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.537+0000 7fdff62e5640 1 -- 172.21.15.64:0/2721889381 wait complete. 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 Processor -- start 2024-09-19T05:15:01.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 -- start start 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff0102d40 0x7fdff0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 0x7fdff01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdff010c170 0x7fdff019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0111e20 con 0x7fdff0103660 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0111ca0 con 0x7fdff010c170 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.538+0000 7fdff62e5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0111fa0 con 0x7fdff0102d40 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 0x7fdff01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 0x7fdff01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46798/0 (socket says 172.21.15.64:46798) 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 -- 172.21.15.64:0/3098665549 learned_addr learned my addr 172.21.15.64:0/3098665549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdff4882640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdff010c170 0x7fdff019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:01.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfeffff640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff0102d40 0x7fdff0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 -- 172.21.15.64:0/3098665549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff0102d40 msgr2=0x7fdff0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff0102d40 0x7fdff0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 -- 172.21.15.64:0/3098665549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdff010c170 msgr2=0x7fdff019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdff010c170 0x7fdff019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 -- 172.21.15.64:0/3098665549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff019d330 con 0x7fdff0103660 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfef7fe640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 0x7fdff01988c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400d740 tx=0x7fdfe400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4017070 con 0x7fdff0103660 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff019d620 con 0x7fdff0103660 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfe400de50 con 0x7fdff0103660 2024-09-19T05:15:01.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400f620 con 0x7fdff0103660 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.539+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01a4f00 con 0x7fdff0103660 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.540+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff0103120 con 0x7fdff0103660 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.542+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fdfe4009c30 con 0x7fdff0103660 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.542+0000 7fdfed7fa640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdfd80a33b0 0x7fdfd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.542+0000 7fdfeffff640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdfd80a33b0 0x7fdfd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.542+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7fdfe40cf970 con 0x7fdff0103660 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.543+0000 7fdfeffff640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdfd80a33b0 0x7fdfd80a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006010 tx=0x7fdfe0000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:01.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.547+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdfe409c2c0 con 0x7fdff0103660 2024-09-19T05:15:01.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.757+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdff0107210 con 0x7fdff0103660 2024-09-19T05:15:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.758+0000 7fdfed7fa640 1 -- 172.21.15.64:0/3098665549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fdfe409bbc0 con 0x7fdff0103660 2024-09-19T05:15:01.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:15:01.761 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:15:01.761 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:15:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.760+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdfd80a33b0 msgr2=0x7fdfd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.760+0000 7fdff62e5640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdfd80a33b0 0x7fdfd80a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006010 tx=0x7fdfe0000bc0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.760+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 msgr2=0x7fdff01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.760+0000 7fdff62e5640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 0x7fdff01988c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400d740 tx=0x7fdfe400dc00 comp rx=0 tx=0).stop 2024-09-19T05:15:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 shutdown_connections 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fdfd80a33b0 0x7fdfd80a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdff010c170 0x7fdff019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdff0103660 0x7fdff01988c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 --2- 172.21.15.64:0/3098665549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff0102d40 0x7fdff0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 >> 172.21.15.64:0/3098665549 conn(0x7fdff00fc7a0 msgr2=0x7fdff00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 shutdown_connections 2024-09-19T05:15:01.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:01.761+0000 7fdff62e5640 1 -- 172.21.15.64:0/3098665549 wait complete. 2024-09-19T05:15:02.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:15:02.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:02 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3098665549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:02.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:02 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3098665549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:02.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:02 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3098665549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:02.639 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:03.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 -- 172.21.15.64:0/3371051023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc8103660 msgr2=0x7febc810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 --2- 172.21.15.64:0/3371051023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc8103660 0x7febc810bb50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7febb4009900 tx=0x7febb402a480 comp rx=0 tx=0).stop 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 -- 172.21.15.64:0/3371051023 shutdown_connections 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 --2- 172.21.15.64:0/3371051023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc810c170 0x7febc810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 --2- 172.21.15.64:0/3371051023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc8103660 0x7febc810bb50 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 --2- 172.21.15.64:0/3371051023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8102d40 0x7febc8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 -- 172.21.15.64:0/3371051023 >> 172.21.15.64:0/3371051023 conn(0x7febc80fc7a0 msgr2=0x7febc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 -- 172.21.15.64:0/3371051023 shutdown_connections 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.113+0000 7febceb31640 1 -- 172.21.15.64:0/3371051023 wait complete. 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 Processor -- start 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 -- start start 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8102d40 0x7febc81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc8103660 0x7febc8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 0x7febc819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febc8111e60 con 0x7febc810c170 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febc8111ce0 con 0x7febc8102d40 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febceb31640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febc8111fe0 con 0x7febc8103660 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febcd0ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 0x7febc819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febcd0ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 0x7febc819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46828/0 (socket says 172.21.15.64:46828) 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.114+0000 7febcd0ce640 1 -- 172.21.15.64:0/76732897 learned_addr learned my addr 172.21.15.64:0/76732897 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febc7fff640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc8103660 0x7febc8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcc8cd640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8102d40 0x7febc81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcd0ce640 1 -- 172.21.15.64:0/76732897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc8103660 msgr2=0x7febc8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcd0ce640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc8103660 0x7febc8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcd0ce640 1 -- 172.21.15.64:0/76732897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8102d40 msgr2=0x7febc81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcd0ce640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8102d40 0x7febc81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcd0ce640 1 -- 172.21.15.64:0/76732897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc819d3a0 con 0x7febc810c170 2024-09-19T05:15:03.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febcd0ce640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 0x7febc819ccc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7febc000b970 tx=0x7febc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febc5ffb640 1 -- 172.21.15.64:0/76732897 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0019070 con 0x7febc810c170 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febc5ffb640 1 -- 172.21.15.64:0/76732897 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc0014440 con 0x7febc810c170 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc819d690 con 0x7febc810c170 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febc5ffb640 1 -- 172.21.15.64:0/76732897 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00075d0 con 0x7febc810c170 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.115+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc81a4f70 con 0x7febc810c170 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febc5ffb640 1 -- 172.21.15.64:0/76732897 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7febc001c070 con 0x7febc810c170 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 Processor -- start 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 -- start start 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febc5ffb640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d9e9852f0 con 0x563d9ea2a370 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d9e92ccf0 con 0x563d9ea1f5d0 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febd0f0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d9e9812b0 con 0x563d9ea20b40 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febc7fff640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.118+0000 7febc5ffb640 1 -- 172.21.15.64:0/76732897 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7febc00cf270 con 0x7febc810c170 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcc8cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea2c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46832/0 (socket says 172.21.15.64:46832) 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 -- 172.21.15.64:0/3462223735 learned_addr learned my addr 172.21.15.64:0/3462223735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcc8cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea1f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.64:42688/0 (socket says 172.21.15.64:42688) 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febc7fff640 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 -- 172.21.15.64:0/3462223735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 msgr2=0x563d9ea1f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea1f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 -- 172.21.15.64:0/3462223735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 msgr2=0x563d9ea1fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea1fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 -- 172.21.15.64:0/3462223735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9e9113a0 con 0x563d9ea2a370 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcc8cd640 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea1f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febc7fff640 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febc7fff640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febb4031440 tx=0x7febb4009560 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febcd0ce640 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea2c760 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7febc00a3d10 tx=0x7febc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4320bf1aa943189d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febb2ffd640 1 -- 172.21.15.64:0/3462223735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc009f980 con 0x563d9ea2a370 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febb2ffd640 1 -- 172.21.15.64:0/3462223735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc00a4240 con 0x563d9ea2a370 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.119+0000 7febb2ffd640 1 -- 172.21.15.64:0/3462223735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00a3db0 con 0x563d9ea2a370 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 -- 172.21.15.64:0/3462223735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 msgr2=0x563d9ea2c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea2c760 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7febc00a3d10 tx=0x7febc0004a40 comp rx=0 tx=0).stop 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 -- 172.21.15.64:0/3462223735 shutdown_connections 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea2c760 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 --2- 172.21.15.64:0/3462223735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea1f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 -- 172.21.15.64:0/3462223735 >> 172.21.15.64:0/3462223735 conn(0x563d9e9e8310 msgr2=0x563d9ea1bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 -- 172.21.15.64:0/3462223735 shutdown_connections 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 -- 172.21.15.64:0/3462223735 wait complete. 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.120+0000 7febd0f0a740 1 -- start start 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 Processor -- start 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 0x563d9ea29de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d9ea2d070 con 0x563d9ea1f5d0 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d9e940060 con 0x563d9ea2a370 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d9ea2d1f0 con 0x563d9ea20b40 2024-09-19T05:15:03.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febd0f0a740 10 client.0 _ll_register_callbacks cb 0x563d9e6938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 0x563d9ea29de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 0x563d9ea29de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46838/0 (socket says 172.21.15.64:46838) 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 -- 172.21.15.64:0/2758991847 learned_addr learned my addr 172.21.15.64:0/2758991847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcd0ce640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febc7fff640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 -- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 msgr2=0x563d9ea27090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea27090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 -- 172.21.15.64:0/2758991847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 msgr2=0x563d9ea277d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea277d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.121+0000 7febcc8cd640 1 -- 172.21.15.64:0/2758991847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9e6f63c0 con 0x563d9ea1f5d0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7febcc8cd640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 0x563d9ea29de0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7febbc00baa0 tx=0x7febbc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533ceb663c12db1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7feba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7feba7fff640 1 -- 172.21.15.64:0/2758991847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc003f10 con 0x563d9ea1f5d0 2024-09-19T05:15:03.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7feba7fff640 1 -- 172.21.15.64:0/2758991847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7febbc011440 con 0x563d9ea1f5d0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc00a4240 con 0x563d9ea1f5d0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7feba7fff640 1 -- 172.21.15.64:0/2758991847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febbc011660 con 0x563d9ea1f5d0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7febd0f0a740 10 client.17637 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d9ea38c00 con 0x563d9ea1f5d0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7feba7fff640 1 -- 172.21.15.64:0/2758991847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc00a4240 con 0x563d9ea1f5d0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7feba7fff640 1 -- 172.21.15.64:0/2758991847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febbc0239a0 con 0x563d9ea1f5d0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7febd0f0a740 10 client.17637 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.122+0000 7febd0f0a740 10 client.17637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7febd0f0a740 20 client.17637 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7febd0f0a740 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9ea3c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7febd0f0a740 4 client.17637 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9e992860 con 0x563d9ea3a0c0 2024-09-19T05:15:03.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7febc7fff640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9ea3c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7febc7fff640 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9ea3c4f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71054ac360cb6330 server_cookie=f0528ccdb5eda48c in_seq=0 out_seq=0 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.123+0000 7feba7fff640 10 client.17637 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7feba7fff640 1 -- 172.21.15.64:0/2758991847 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d9e992860 con 0x563d9ea3a0c0 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7feba7fff640 10 client.17637 handle_command_reply: tid=0 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 client.17637 shutdown 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 msgr2=0x563d9ea29de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 0x563d9ea29de0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7febbc00baa0 tx=0x7febbc00bf60 comp rx=0 tx=0).stop 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 shutdown_connections 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9ea3c4f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea277d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7feba7fff640 0 client.17637 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 --2- 172.21.15.64:0/2758991847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea1f5d0 0x563d9ea29de0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 >> 172.21.15.64:0/2758991847 conn(0x563d9e9e8310 msgr2=0x563d9ea21990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 shutdown_connections 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.124+0000 7febd0f0a740 1 -- 172.21.15.64:0/2758991847 wait complete. 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.125+0000 7febd0f0a740 20 client.17637 trim_cache size 0 max 16384 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 Processor -- start 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 -- start start 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea41e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febc81b0250 con 0x563d9ea2a370 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febc81adb20 con 0x563d9ea1f5d0 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febd0f0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febc81ab990 con 0x563d9ea20b40 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea41e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea41e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46842/0 (socket says 172.21.15.64:46842) 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 -- 172.21.15.64:0/995323302 learned_addr learned my addr 172.21.15.64:0/995323302 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febc7fff640 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcc8cd640 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 -- 172.21.15.64:0/995323302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 msgr2=0x563d9ea226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 -- 172.21.15.64:0/995323302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 msgr2=0x563d9ea22180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea22180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.138+0000 7febcd0ce640 1 -- 172.21.15.64:0/995323302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9ea27b40 con 0x563d9ea2a370 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febcd0ce640 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea41e10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7febc009b610 tx=0x7febc0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb155567feeaf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febb27fc640 1 -- 172.21.15.64:0/995323302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc009f980 con 0x563d9ea2a370 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febb27fc640 1 -- 172.21.15.64:0/995323302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc005c030 con 0x563d9ea2a370 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febb27fc640 1 -- 172.21.15.64:0/995323302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00a3db0 con 0x563d9ea2a370 2024-09-19T05:15:03.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 -- 172.21.15.64:0/995323302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 msgr2=0x563d9ea41e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea41e10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7febc009b610 tx=0x7febc0004b50 comp rx=0 tx=0).stop 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 -- 172.21.15.64:0/995323302 shutdown_connections 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea2a370 0x563d9ea41e10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea20b40 0x563d9ea226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 --2- 172.21.15.64:0/995323302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea1f5d0 0x563d9ea22180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.139+0000 7febd0f0a740 1 -- 172.21.15.64:0/995323302 >> 172.21.15.64:0/995323302 conn(0x563d9e9e8310 msgr2=0x563d9ea1ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 -- 172.21.15.64:0/995323302 shutdown_connections 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 -- 172.21.15.64:0/995323302 wait complete. 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 -- start start 2024-09-19T05:15:03.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 Processor -- start 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea1f5d0 0x563d9ea32c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 0x563d9ea2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea2e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febc81abfe0 con 0x563d9ea20b40 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febc81ad450 con 0x563d9ea2a370 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febc81ac6b0 con 0x563d9ea1f5d0 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febd0f0a740 10 client.0 _ll_register_callbacks cb 0x7febbc0040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.140+0000 7febc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 0x563d9ea2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 0x563d9ea2dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46858/0 (socket says 172.21.15.64:46858) 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 -- 172.21.15.64:0/62531279 learned_addr learned my addr 172.21.15.64:0/62531279 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febcc8cd640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea1f5d0 0x563d9ea32c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febcd0ce640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea2e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 -- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea1f5d0 msgr2=0x563d9ea32c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea1f5d0 0x563d9ea32c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 -- 172.21.15.64:0/62531279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 msgr2=0x563d9ea2e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea2e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 -- 172.21.15.64:0/62531279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9ea2ed60 con 0x563d9ea20b40 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febcc8cd640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea1f5d0 0x563d9ea32c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febc7fff640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 0x563d9ea2dfb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7febb4032fd0 tx=0x7febb402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d533d5fd27d7f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7feba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7feba7fff640 1 -- 172.21.15.64:0/62531279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4059060 con 0x563d9ea20b40 2024-09-19T05:15:03.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7feba7fff640 1 -- 172.21.15.64:0/62531279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7febb4005240 con 0x563d9ea20b40 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc005c030 con 0x563d9ea20b40 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7feba7fff640 1 -- 172.21.15.64:0/62531279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febb4035040 con 0x563d9ea20b40 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febd0f0a740 10 client.17643 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.141+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d9eaa4eb0 con 0x563d9ea20b40 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7feba7fff640 1 -- 172.21.15.64:0/62531279 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc005c030 con 0x563d9ea20b40 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7feba7fff640 1 -- 172.21.15.64:0/62531279 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7febb40035f0 con 0x563d9ea20b40 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febd0f0a740 10 client.17643 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febd0f0a740 10 client.17643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febd0f0a740 20 client.17643 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febd0f0a740 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9eaa7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febd0f0a740 4 client.17643 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d9e98cd70 con 0x563d9ea3a0c0 2024-09-19T05:15:03.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.142+0000 7febcc8cd640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9eaa7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febcc8cd640 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9eaa7650 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=336663a140952935 server_cookie=4d028b188feb142c in_seq=0 out_seq=0 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7feba7fff640 10 client.17643 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7feba7fff640 1 -- 172.21.15.64:0/62531279 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d9e98cd70 con 0x563d9ea3a0c0 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7feba7fff640 10 client.17643 handle_command_reply: tid=0 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 client.17643 shutdown 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 msgr2=0x563d9ea2dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 0x563d9ea2dfb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7febb4032fd0 tx=0x7febb402a480 comp rx=0 tx=0).stop 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 shutdown_connections 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x563d9ea3a0c0 0x563d9eaa7650 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9ea2a370 0x563d9ea2e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d9ea20b40 0x563d9ea2dfb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 --2- 172.21.15.64:0/62531279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9ea1f5d0 0x563d9ea32c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7feba7fff640 0 client.17643 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 >> 172.21.15.64:0/62531279 conn(0x563d9e9e8310 msgr2=0x563d9ea288f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 shutdown_connections 2024-09-19T05:15:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 1 -- 172.21.15.64:0/62531279 wait complete. 2024-09-19T05:15:03.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.143+0000 7febd0f0a740 20 client.17643 trim_cache size 0 max 16384 2024-09-19T05:15:03.161 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 msgr2=0x7febb80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febb4031440 tx=0x7febb4009560 comp rx=0 tx=0).stop 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 msgr2=0x7febc819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 0x7febc819ccc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7febc000b970 tx=0x7febc000be30 comp rx=0 tx=0).stop 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 shutdown_connections 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7febb80a33b0 0x7febb80a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febc810c170 0x7febc819ccc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc8103660 0x7febc8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 --2- 172.21.15.64:0/76732897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8102d40 0x7febc81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 >> 172.21.15.64:0/76732897 conn(0x7febc80fc7a0 msgr2=0x7febc80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 shutdown_connections 2024-09-19T05:15:03.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:03.144+0000 7febceb31640 1 -- 172.21.15.64:0/76732897 wait complete. 2024-09-19T05:15:03.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:03 smithi064 ceph-mon[30960]: pgmap v281: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:03.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:03 smithi103 ceph-mon[34446]: pgmap v281: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:03.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:03 smithi174 ceph-mon[34488]: pgmap v281: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:03.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:15:04.112 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.580+0000 7f0715d55640 1 -- 172.21.15.64:0/1132924868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710102d40 msgr2=0x7f0710103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.580+0000 7f0715d55640 1 --2- 172.21.15.64:0/1132924868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710102d40 0x7f0710103120 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0700009900 tx=0x7f070002a480 comp rx=0 tx=0).stop 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.580+0000 7f0715d55640 1 -- 172.21.15.64:0/1132924868 shutdown_connections 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 --2- 172.21.15.64:0/1132924868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f071010c170 0x7f071010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 --2- 172.21.15.64:0/1132924868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0710103660 0x7f071010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 --2- 172.21.15.64:0/1132924868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0710102d40 0x7f0710103120 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 -- 172.21.15.64:0/1132924868 >> 172.21.15.64:0/1132924868 conn(0x7f07100fc7a0 msgr2=0x7f07100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:04.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 -- 172.21.15.64:0/1132924868 shutdown_connections 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 -- 172.21.15.64:0/1132924868 wait complete. 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 Processor -- start 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.581+0000 7f0715d55640 1 -- start start 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f0715d55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0710102d40 0x7f07101009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f0715d55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0710103660 0x7f0710100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f0715d55640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 0x7f0710101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f0715d55640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0710111e20 con 0x7f071010c170 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f0715d55640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0710111ca0 con 0x7f0710102d40 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f0715d55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0710111fa0 con 0x7f0710103660 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 0x7f0710101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 0x7f0710101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44390/0 (socket says 172.21.15.64:44390) 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 -- 172.21.15.64:0/3379764868 learned_addr learned my addr 172.21.15.64:0/3379764868 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070effd640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0710103660 0x7f0710100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070f7fe640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0710102d40 0x7f07101009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 -- 172.21.15.64:0/3379764868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0710103660 msgr2=0x7f0710100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0710103660 0x7f0710100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 -- 172.21.15.64:0/3379764868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0710102d40 msgr2=0x7f07101009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0710102d40 0x7f07101009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 -- 172.21.15.64:0/3379764868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07101a6910 con 0x7f071010c170 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070f7fe640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0710102d40 0x7f07101009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070ffff640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 0x7f0710101420 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f070800eb10 tx=0x7f070800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070cff9640 1 -- 172.21.15.64:0/3379764868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708004b20 con 0x7f071010c170 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070cff9640 1 -- 172.21.15.64:0/3379764868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0708004cc0 con 0x7f071010c170 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.582+0000 7f070cff9640 1 -- 172.21.15.64:0/3379764868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708010690 con 0x7f071010c170 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.583+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07101a6c00 con 0x7f071010c170 2024-09-19T05:15:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.583+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07101a70f0 con 0x7f071010c170 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f070cff9640 1 -- 172.21.15.64:0/3379764868 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f0708005020 con 0x7f071010c170 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 Processor -- start 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 -- start start 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8bb40 0x562364a8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5623649f02f0 con 0x562364a95370 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562364997cf0 con 0x562364a8bb40 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f071812e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5623649ec2b0 con 0x562364a8a5d0 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.585+0000 7f070cff9640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f06f80a33b0 0x7f06f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070ffff640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f06f80a33b0 0x7f06f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a97760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44398/0 (socket says 172.21.15.64:44398) 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 -- 172.21.15.64:0/2166851873 learned_addr learned my addr 172.21.15.64:0/2166851873 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8bb40 0x562364a8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070f7fe640 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070cff9640 1 -- 172.21.15.64:0/3379764868 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f07080d0370 con 0x7f071010c170 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 -- 172.21.15.64:0/2166851873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 msgr2=0x562364a8aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a8aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 -- 172.21.15.64:0/2166851873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8bb40 msgr2=0x562364a8a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8bb40 0x562364a8a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 -- 172.21.15.64:0/2166851873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236497c3a0 con 0x562364a95370 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070ffff640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f06f80a33b0 0x7f06f80a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0708098f90 tx=0x7f070806f3f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f070effd640 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a97760 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0704011700 tx=0x7f070400a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550e69dbf6578b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f06f5ffb640 1 -- 172.21.15.64:0/2166851873 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704004020 con 0x562364a95370 2024-09-19T05:15:04.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f06f5ffb640 1 -- 172.21.15.64:0/2166851873 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07040054d0 con 0x562364a95370 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.586+0000 7f06f5ffb640 1 -- 172.21.15.64:0/2166851873 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704004b20 con 0x562364a95370 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 -- 172.21.15.64:0/2166851873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 msgr2=0x562364a97760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a97760 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0704011700 tx=0x7f070400a690 comp rx=0 tx=0).stop 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 -- 172.21.15.64:0/2166851873 shutdown_connections 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a97760 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a8aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 --2- 172.21.15.64:0/2166851873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8bb40 0x562364a8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 -- 172.21.15.64:0/2166851873 >> 172.21.15.64:0/2166851873 conn(0x562364a53310 msgr2=0x562364a86c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 -- 172.21.15.64:0/2166851873 shutdown_connections 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.587+0000 7f071812e740 1 -- 172.21.15.64:0/2166851873 wait complete. 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 -- start start 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 Processor -- start 2024-09-19T05:15:04.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8a5d0 0x562364a94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8bb40 0x562364a92030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562364a98070 con 0x562364a95370 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623649ab060 con 0x562364a8a5d0 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562364a981f0 con 0x562364a8bb40 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f071812e740 10 client.0 _ll_register_callbacks cb 0x5623646fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f070effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f070effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a92640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44400/0 (socket says 172.21.15.64:44400) 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f070effd640 1 -- 172.21.15.64:0/243619751 learned_addr learned my addr 172.21.15.64:0/243619751 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f070f7fe640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8bb40 0x562364a92030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.588+0000 7f070effd640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8a5d0 0x562364a94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070effd640 1 -- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8bb40 msgr2=0x562364a92030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070effd640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8bb40 0x562364a92030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070effd640 1 -- 172.21.15.64:0/243619751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8a5d0 msgr2=0x562364a94d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070effd640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8a5d0 0x562364a94d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070effd640 1 -- 172.21.15.64:0/243619751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623647613c0 con 0x562364a95370 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070f7fe640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8bb40 0x562364a92030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f070effd640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a92640 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f070400afd0 tx=0x7f070401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3d76a05b7bd0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f06eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f06eeffd640 1 -- 172.21.15.64:0/243619751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704004b50 con 0x562364a95370 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f06eeffd640 1 -- 172.21.15.64:0/243619751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f0704004cf0 con 0x562364a95370 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07040054d0 con 0x562364a95370 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f06eeffd640 1 -- 172.21.15.64:0/243619751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f070400f7c0 con 0x562364a95370 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 10 client.17655 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562364aa3b40 con 0x562364a95370 2024-09-19T05:15:04.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f06eeffd640 1 -- 172.21.15.64:0/243619751 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07040054d0 con 0x562364a95370 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f06eeffd640 1 -- 172.21.15.64:0/243619751 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f070401e780 con 0x562364a95370 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 10 client.17655 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 10 client.17655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 20 client.17655 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364aa4e90 0x562364aa7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 4 client.17655 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.589+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623649fd860 con 0x562364aa4e90 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.590+0000 7f070f7fe640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364aa4e90 0x562364aa7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.590+0000 7f070f7fe640 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364aa4e90 0x562364aa7280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57684c0aa310fc70 server_cookie=8e160d4adb2f95f8 in_seq=0 out_seq=0 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.590+0000 7f06eeffd640 10 client.17655 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f06eeffd640 1 -- 172.21.15.64:0/243619751 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623649fd860 con 0x562364aa4e90 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f06eeffd640 10 client.17655 handle_command_reply: tid=0 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 client.17655 shutdown 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 msgr2=0x562364a92640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a92640 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f070400afd0 tx=0x7f070401a040 comp rx=0 tx=0).stop 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 shutdown_connections 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364aa4e90 0x562364aa7280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a95370 0x562364a92640 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8bb40 0x562364a92030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f06eeffd640 0 client.17655 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 --2- 172.21.15.64:0/243619751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a8a5d0 0x562364a94d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 >> 172.21.15.64:0/243619751 conn(0x562364a53310 msgr2=0x562364a8c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 shutdown_connections 2024-09-19T05:15:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 1 -- 172.21.15.64:0/243619751 wait complete. 2024-09-19T05:15:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.591+0000 7f071812e740 20 client.17655 trim_cache size 0 max 16384 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.604+0000 7f071812e740 1 Processor -- start 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.604+0000 7f071812e740 1 -- start start 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f071812e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364aa1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f071812e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f071812e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f071812e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07101b2430 con 0x562364a8bb40 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f071812e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f07101afd00 con 0x562364a95370 2024-09-19T05:15:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f071812e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07101adb70 con 0x562364a8a5d0 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44404/0 (socket says 172.21.15.64:44404) 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 -- 172.21.15.64:0/3633163194 learned_addr learned my addr 172.21.15.64:0/3633163194 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070effd640 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364aa1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 -- 172.21.15.64:0/3633163194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 msgr2=0x562364aa1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070effd640 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364aa1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 -- 172.21.15.64:0/3633163194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 msgr2=0x562364a8d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 -- 172.21.15.64:0/3633163194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562364a8b300 con 0x562364a8bb40 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070effd640 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f070f7fe640 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8d120 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0700031440 tx=0x7f0700009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1affb0fca5a3405 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.605+0000 7f06f57fa640 1 -- 172.21.15.64:0/3633163194 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700035070 con 0x562364a8bb40 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f06f57fa640 1 -- 172.21.15.64:0/3633163194 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0700048070 con 0x562364a8bb40 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f06f57fa640 1 -- 172.21.15.64:0/3633163194 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700004920 con 0x562364a8bb40 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 -- 172.21.15.64:0/3633163194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 msgr2=0x562364a8d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8d120 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0700031440 tx=0x7f0700009640 comp rx=0 tx=0).stop 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 -- 172.21.15.64:0/3633163194 shutdown_connections 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8d120 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 --2- 172.21.15.64:0/3633163194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364aa1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 -- 172.21.15.64:0/3633163194 >> 172.21.15.64:0/3633163194 conn(0x562364a53310 msgr2=0x562364a940b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 -- 172.21.15.64:0/3633163194 shutdown_connections 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 -- 172.21.15.64:0/3633163194 wait complete. 2024-09-19T05:15:04.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.606+0000 7f071812e740 1 -- start start 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 Processor -- start 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a9da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07101ae1c0 con 0x562364a8bb40 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f07101af630 con 0x562364a95370 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07101ae890 con 0x562364a8a5d0 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f071812e740 10 client.0 _ll_register_callbacks cb 0x7f070400ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44416/0 (socket says 172.21.15.64:44416) 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 -- 172.21.15.64:0/1304125266 learned_addr learned my addr 172.21.15.64:0/1304125266 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070effd640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a9da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070effd640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 -- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 msgr2=0x562364a9da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a9da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 -- 172.21.15.64:0/1304125266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 msgr2=0x562364a8f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070f7fe640 1 -- 172.21.15.64:0/1304125266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562364a8f6e0 con 0x562364a8bb40 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.607+0000 7f070effd640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f070f7fe640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8eb60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0700006010 tx=0x7f0700005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa3dafa1e6f0236 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f06eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f06eeffd640 1 -- 172.21.15.64:0/1304125266 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700035040 con 0x562364a8bb40 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f06eeffd640 1 -- 172.21.15.64:0/1304125266 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f0700004240 con 0x562364a8bb40 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0700048070 con 0x562364a8bb40 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f06eeffd640 1 -- 172.21.15.64:0/1304125266 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f070005b050 con 0x562364a8bb40 2024-09-19T05:15:04.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f071812e740 10 client.17661 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562364b17e00 con 0x562364a8bb40 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f06eeffd640 1 -- 172.21.15.64:0/1304125266 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0700048070 con 0x562364a8bb40 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f06eeffd640 1 -- 172.21.15.64:0/1304125266 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f070004c430 con 0x562364a8bb40 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f071812e740 10 client.17661 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.608+0000 7f071812e740 10 client.17661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f071812e740 20 client.17661 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f071812e740 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364b18570 0x562364b1a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f071812e740 4 client.17661 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623649f7d70 con 0x562364b18570 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f070effd640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364b18570 0x562364b1a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f06eeffd640 10 client.17661 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f070effd640 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364b18570 0x562364b1a990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4f2b5a2611e658 server_cookie=a01bf2e69562db8d in_seq=0 out_seq=0 2024-09-19T05:15:04.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f06eeffd640 1 -- 172.21.15.64:0/1304125266 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623649f7d70 con 0x562364b18570 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f06eeffd640 10 client.17661 handle_command_reply: tid=0 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f071812e740 1 client.17661 shutdown 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.609+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 msgr2=0x562364a8eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8eb60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0700006010 tx=0x7f0700005650 comp rx=0 tx=0).stop 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 shutdown_connections 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562364a95370 0x562364a8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x562364b18570 0x562364b1a990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f06eeffd640 0 client.17661 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562364a8bb40 0x562364a8eb60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 --2- 172.21.15.64:0/1304125266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562364a8a5d0 0x562364a9da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 >> 172.21.15.64:0/1304125266 conn(0x562364a53310 msgr2=0x562364a854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:04.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 shutdown_connections 2024-09-19T05:15:04.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 1 -- 172.21.15.64:0/1304125266 wait complete. 2024-09-19T05:15:04.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f071812e740 20 client.17661 trim_cache size 0 max 16384 2024-09-19T05:15:04.621 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f06f80a33b0 msgr2=0x7f06f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f06f80a33b0 0x7f06f80a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0708098f90 tx=0x7f070806f3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 msgr2=0x7f0710101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 0x7f0710101420 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f070800eb10 tx=0x7f070800c2d0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 shutdown_connections 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f06f80a33b0 0x7f06f80a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071010c170 0x7f0710101420 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0710103660 0x7f0710100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 --2- 172.21.15.64:0/3379764868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0710102d40 0x7f07101009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:04.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.610+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 >> 172.21.15.64:0/3379764868 conn(0x7f07100fc7a0 msgr2=0x7f07100fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:04.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.611+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 shutdown_connections 2024-09-19T05:15:04.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:04.611+0000 7f0715d55640 1 -- 172.21.15.64:0/3379764868 wait complete. 2024-09-19T05:15:05.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:05 smithi103 ceph-mon[34446]: pgmap v282: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:05.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:05 smithi064 ceph-mon[30960]: pgmap v282: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:05.499 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:15:05.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:05 smithi174 ceph-mon[34488]: pgmap v282: 97 pgs: 97 active+clean; 25 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:05.757 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 -- 172.21.15.64:0/2541897242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c101de0 msgr2=0x7f430c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 --2- 172.21.15.64:0/2541897242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c101de0 0x7f430c1021c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009900 tx=0x7f42fc02a410 comp rx=0 tx=0).stop 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 -- 172.21.15.64:0/2541897242 shutdown_connections 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 --2- 172.21.15.64:0/2541897242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430c106f10 0x7f430c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 --2- 172.21.15.64:0/2541897242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430c102790 0x7f430c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 --2- 172.21.15.64:0/2541897242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c101de0 0x7f430c1021c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 -- 172.21.15.64:0/2541897242 >> 172.21.15.64:0/2541897242 conn(0x7f430c0fd8c0 msgr2=0x7f430c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 -- 172.21.15.64:0/2541897242 shutdown_connections 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.205+0000 7f43116b7640 1 -- 172.21.15.64:0/2541897242 wait complete. 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 Processor -- start 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 -- start start 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430c101de0 0x7f430c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430c102790 0x7f430c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 0x7f430c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f430c111e60 con 0x7f430c106f10 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f430c111ce0 con 0x7f430c102790 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f43116b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f430c111fe0 con 0x7f430c101de0 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f430b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 0x7f430c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f430b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 0x7f430c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44428/0 (socket says 172.21.15.64:44428) 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f430b7fe640 1 -- 172.21.15.64:0/308838774 learned_addr learned my addr 172.21.15.64:0/308838774 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:06.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f430a7fc640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430c102790 0x7f430c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.206+0000 7f430affd640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430c101de0 0x7f430c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f430b7fe640 1 -- 172.21.15.64:0/308838774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430c101de0 msgr2=0x7f430c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f430b7fe640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430c101de0 0x7f430c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f430b7fe640 1 -- 172.21.15.64:0/308838774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430c102790 msgr2=0x7f430c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f430b7fe640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430c102790 0x7f430c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f430b7fe640 1 -- 172.21.15.64:0/308838774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c1a68a0 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f430b7fe640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 0x7f430c074f70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f430400b970 tx=0x7f430400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f42f3fff640 1 -- 172.21.15.64:0/308838774 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304019070 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c1a6b90 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f42f3fff640 1 -- 172.21.15.64:0/308838774 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4304014440 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f42f3fff640 1 -- 172.21.15.64:0/308838774 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040075d0 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.207+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c1a7080 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.209+0000 7f42f3fff640 1 -- 172.21.15.64:0/308838774 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f430401c070 con 0x7f430c106f10 2024-09-19T05:15:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 Processor -- start 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 -- start start 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c196090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 0x558a7c1a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558a7c0fc2f0 con 0x558a7c1a1370 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a7c0a3cf0 con 0x558a7c197b40 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f4313a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a7c0f82b0 con 0x558a7c1965d0 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f42f3fff640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f42f40a33b0 0x7f42f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 0x558a7c1a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 0x558a7c1a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44434/0 (socket says 172.21.15.64:44434) 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430a7fc640 1 -- 172.21.15.64:0/716686477 learned_addr learned my addr 172.21.15.64:0/716686477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430b7fe640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f42f40a33b0 0x7f42f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430a7fc640 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430a7fc640 1 -- 172.21.15.64:0/716686477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 msgr2=0x558a7c196a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f430a7fc640 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c196a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.210+0000 7f42f3fff640 1 -- 172.21.15.64:0/308838774 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f43040cf270 con 0x7f430c106f10 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f430a7fc640 1 -- 172.21.15.64:0/716686477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 msgr2=0x558a7c196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f430a7fc640 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f430a7fc640 1 -- 172.21.15.64:0/716686477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7c0883a0 con 0x558a7c1a1370 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f430a7fc640 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 0x558a7c1a3760 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f43000125a0 tx=0x7f430000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7159694d863c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f430b7fe640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f42f40a33b0 0x7f42f40a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4304020000 tx=0x7f430406d400 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f42f17fa640 1 -- 172.21.15.64:0/716686477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300004250 con 0x558a7c1a1370 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f42f17fa640 1 -- 172.21.15.64:0/716686477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43000043f0 con 0x558a7c1a1370 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.211+0000 7f42f17fa640 1 -- 172.21.15.64:0/716686477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300004e00 con 0x558a7c1a1370 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 -- 172.21.15.64:0/716686477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 msgr2=0x558a7c1a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 0x558a7c1a3760 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f43000125a0 tx=0x7f430000b870 comp rx=0 tx=0).stop 2024-09-19T05:15:06.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 -- 172.21.15.64:0/716686477 shutdown_connections 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1a1370 0x558a7c1a3760 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 --2- 172.21.15.64:0/716686477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 -- 172.21.15.64:0/716686477 >> 172.21.15.64:0/716686477 conn(0x558a7c15f310 msgr2=0x558a7c192c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 -- 172.21.15.64:0/716686477 shutdown_connections 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 -- 172.21.15.64:0/716686477 wait complete. 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 -- start start 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.212+0000 7f4313a90740 1 Processor -- start 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558a7c1a4070 con 0x558a7c1965d0 2024-09-19T05:15:06.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a7c0b7060 con 0x558a7c197b40 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a7c1a41f0 con 0x558a7c1a1370 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 10 client.0 _ll_register_callbacks cb 0x558a7be0a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c1a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44440/0 (socket says 172.21.15.64:44440) 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 -- 172.21.15.64:0/3935857221 learned_addr learned my addr 172.21.15.64:0/3935857221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430affd640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 -- 172.21.15.64:0/3935857221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 msgr2=0x558a7c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 -- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 msgr2=0x558a7c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c19e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 -- 172.21.15.64:0/3935857221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7be6d3c0 con 0x558a7c1965d0 2024-09-19T05:15:06.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430affd640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c1a0d80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f430000be90 tx=0x7f430000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde64142b7f6f659 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f42eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f42eeff5640 1 -- 172.21.15.64:0/3935857221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300005430 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f42eeff5640 1 -- 172.21.15.64:0/3935857221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f43000055d0 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.213+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43000043f0 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f42eeff5640 1 -- 172.21.15.64:0/3935857221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4300036050 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 10 client.17673 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a7c1afb40 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f42eeff5640 1 -- 172.21.15.64:0/3935857221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43000043f0 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f42eeff5640 1 -- 172.21.15.64:0/3935857221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f430001e440 con 0x558a7c1965d0 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 10 client.17673 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 20 client.17673 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:06.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c1b0ef0 0x558a7c1b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 4 client.17673 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a7c109860 con 0x558a7c1b0ef0 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.214+0000 7f430affd640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c1b0ef0 0x558a7c1b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.215+0000 7f430affd640 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c1b0ef0 0x558a7c1b32e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9a22394e14b482 server_cookie=ae86f8ba844e159f in_seq=0 out_seq=0 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.215+0000 7f42eeff5640 10 client.17673 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.215+0000 7f42eeff5640 1 -- 172.21.15.64:0/3935857221 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a7c109860 con 0x558a7c1b0ef0 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.215+0000 7f42eeff5640 10 client.17673 handle_command_reply: tid=0 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.215+0000 7f4313a90740 1 client.17673 shutdown 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 msgr2=0x558a7c1a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c1a0d80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f430000be90 tx=0x7f430000b390 comp rx=0 tx=0).stop 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 shutdown_connections 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c1b0ef0 0x558a7c1b32e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 --2- 172.21.15.64:0/3935857221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c1a0d80 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f42eeff5640 0 client.17673 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 >> 172.21.15.64:0/3935857221 conn(0x558a7c15f310 msgr2=0x558a7c198960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 shutdown_connections 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 1 -- 172.21.15.64:0/3935857221 wait complete. 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.216+0000 7f4313a90740 20 client.17673 trim_cache size 0 max 16384 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 Processor -- start 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 -- start start 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c199120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c199660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1b2420 con 0x558a7c1965d0 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1afcf0 con 0x558a7c197b40 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f4313a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1adb60 con 0x558a7c1a1370 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44442/0 (socket says 172.21.15.64:44442) 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 -- 172.21.15.64:0/2617287974 learned_addr learned my addr 172.21.15.64:0/2617287974 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430affd640 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 -- 172.21.15.64:0/2617287974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 msgr2=0x558a7c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 -- 172.21.15.64:0/2617287974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 msgr2=0x558a7c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.229+0000 7f430a7fc640 1 -- 172.21.15.64:0/2617287974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7c1a3ad0 con 0x558a7c1965d0 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f430a7fc640 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c19c920 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4300006010 tx=0x7f4300004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b7ff1ec4bd0347 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:06.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f42f0ff9640 1 -- 172.21.15.64:0/2617287974 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430003f070 con 0x558a7c1965d0 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f42f0ff9640 1 -- 172.21.15.64:0/2617287974 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4300033070 con 0x558a7c1965d0 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f42f0ff9640 1 -- 172.21.15.64:0/2617287974 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300022050 con 0x558a7c1965d0 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 -- 172.21.15.64:0/2617287974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 msgr2=0x558a7c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c19c920 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4300006010 tx=0x7f4300004e00 comp rx=0 tx=0).stop 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 -- 172.21.15.64:0/2617287974 shutdown_connections 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1a1370 0x558a7c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c197b40 0x558a7c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 --2- 172.21.15.64:0/2617287974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c1965d0 0x558a7c19c920 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 -- 172.21.15.64:0/2617287974 >> 172.21.15.64:0/2617287974 conn(0x558a7c15f310 msgr2=0x558a7c191d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 -- 172.21.15.64:0/2617287974 shutdown_connections 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.230+0000 7f4313a90740 1 -- 172.21.15.64:0/2617287974 wait complete. 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 -- start start 2024-09-19T05:15:06.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 Processor -- start 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c1a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 0x558a7c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c1a1370 0x558a7c19af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1ae1b0 con 0x558a7c197b40 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1af620 con 0x558a7c1a1370 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f430c1ae880 con 0x558a7c1965d0 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f4313a90740 10 client.0 _ll_register_callbacks cb 0x7f43000123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 0x558a7c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 0x558a7c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44456/0 (socket says 172.21.15.64:44456) 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 -- 172.21.15.64:0/3301489645 learned_addr learned my addr 172.21.15.64:0/3301489645 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c1a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c1a1370 0x558a7c19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 -- 172.21.15.64:0/3301489645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 msgr2=0x558a7c1a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c1a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 -- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c1a1370 msgr2=0x558a7c19af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c1a1370 0x558a7c19af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430affd640 1 -- 172.21.15.64:0/3301489645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7c19b490 con 0x558a7c197b40 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.231+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c1a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f430affd640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 0x558a7c19aa10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f42fc031440 tx=0x7f42fc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20fb3f001a19dce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f42eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f42eeff5640 1 -- 172.21.15.64:0/3301489645 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc035040 con 0x558a7c197b40 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f42eeff5640 1 -- 172.21.15.64:0/3301489645 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f42fc004250 con 0x558a7c197b40 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4300033070 con 0x558a7c197b40 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f42eeff5640 1 -- 172.21.15.64:0/3301489645 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f42fc050050 con 0x558a7c197b40 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 10 client.17679 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a7c21b680 con 0x558a7c197b40 2024-09-19T05:15:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f42eeff5640 1 -- 172.21.15.64:0/3301489645 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4300033070 con 0x558a7c197b40 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f42eeff5640 1 -- 172.21.15.64:0/3301489645 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f42fc004a30 con 0x558a7c197b40 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 10 client.17679 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 20 client.17679 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c21be80 0x558a7c21e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 4 client.17679 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.232+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a7c103d70 con 0x558a7c21be80 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c21be80 0x558a7c21e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f430a7fc640 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c21be80 0x558a7c21e2a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa39baf4bf5da26 server_cookie=639602ac0cbc781d in_seq=0 out_seq=0 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f42eeff5640 10 client.17679 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f42eeff5640 1 -- 172.21.15.64:0/3301489645 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a7c103d70 con 0x558a7c21be80 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f42eeff5640 10 client.17679 handle_command_reply: tid=0 2024-09-19T05:15:06.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f4313a90740 1 client.17679 shutdown 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 msgr2=0x558a7c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f4313a90740 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 0x558a7c19aa10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f42fc031440 tx=0x7f42fc009520 comp rx=0 tx=0).stop 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 shutdown_connections 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f4313a90740 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a7c1a1370 0x558a7c19af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.233+0000 7f4313a90740 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558a7c21be80 0x558a7c21e2a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:15:06.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f4313a90740 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a7c197b40 0x558a7c19aa10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f4313a90740 1 --2- 172.21.15.64:0/3301489645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a7c1965d0 0x558a7c1a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f42eeff5640 0 client.17679 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 >> 172.21.15.64:0/3301489645 conn(0x558a7c15f310 msgr2=0x558a7c19f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 shutdown_connections 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f4313a90740 1 -- 172.21.15.64:0/3301489645 wait complete. 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f4313a90740 20 client.17679 trim_cache size 0 max 16384 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f42f40a33b0 msgr2=0x7f42f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f42f40a33b0 0x7f42f40a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4304020000 tx=0x7f430406d400 comp rx=0 tx=0).stop 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 msgr2=0x7f430c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 0x7f430c074f70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f430400b970 tx=0x7f430400be30 comp rx=0 tx=0).stop 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 shutdown_connections 2024-09-19T05:15:06.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f42f40a33b0 0x7f42f40a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f430c106f10 0x7f430c074f70 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430c102790 0x7f430c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 --2- 172.21.15.64:0/308838774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430c101de0 0x7f430c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.234+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 >> 172.21.15.64:0/308838774 conn(0x7f430c0fd8c0 msgr2=0x7f430c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.235+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 shutdown_connections 2024-09-19T05:15:06.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:06.235+0000 7f43116b7640 1 -- 172.21.15.64:0/308838774 wait complete. 2024-09-19T05:15:07.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:07 smithi064 ceph-mon[30960]: pgmap v283: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:07.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:07 smithi103 ceph-mon[34446]: pgmap v283: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:07.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:07 smithi174 ceph-mon[34488]: pgmap v283: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:07.964 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:15:07.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:15:08.237 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:08.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.729+0000 7f62da0c2640 1 -- 172.21.15.64:0/4075333130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 msgr2=0x7f62d4103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.729+0000 7f62da0c2640 1 --2- 172.21.15.64:0/4075333130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d4103130 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009900 tx=0x7f62c402a480 comp rx=0 tx=0).stop 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 -- 172.21.15.64:0/4075333130 shutdown_connections 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 --2- 172.21.15.64:0/4075333130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 0x7f62d410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 --2- 172.21.15.64:0/4075333130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62d4103670 0x7f62d410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 --2- 172.21.15.64:0/4075333130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d4103130 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 -- 172.21.15.64:0/4075333130 >> 172.21.15.64:0/4075333130 conn(0x7f62d40fc7d0 msgr2=0x7f62d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 -- 172.21.15.64:0/4075333130 shutdown_connections 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 -- 172.21.15.64:0/4075333130 wait complete. 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.730+0000 7f62da0c2640 1 Processor -- start 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 -- start start 2024-09-19T05:15:08.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d41009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62d4103670 0x7f62d4100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 0x7f62d4101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111e30 con 0x7f62d4102d50 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111cb0 con 0x7f62d4103670 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62da0c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111fb0 con 0x7f62d410c180 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d41009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d41009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44484/0 (socket says 172.21.15.64:44484) 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d37fe640 1 -- 172.21.15.64:0/3175972129 learned_addr learned my addr 172.21.15.64:0/3175972129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d3fff640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 0x7f62d4101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62d4103670 0x7f62d4100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d3fff640 1 -- 172.21.15.64:0/3175972129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62d4103670 msgr2=0x7f62d4100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d3fff640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62d4103670 0x7f62d4100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d3fff640 1 -- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 msgr2=0x7f62d41009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d3fff640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d41009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d3fff640 1 -- 172.21.15.64:0/3175972129 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d41a6820 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.731+0000 7f62d37fe640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.732+0000 7f62d3fff640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 0x7f62d4101430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00ca30 tx=0x7f62cc00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.732+0000 7f62d0ff9640 1 -- 172.21.15.64:0/3175972129 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc007d90 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.732+0000 7f62d0ff9640 1 -- 172.21.15.64:0/3175972129 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62cc005020 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.732+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d41a6b10 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.732+0000 7f62d0ff9640 1 -- 172.21.15.64:0/3175972129 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00f690 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.732+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d41a70a0 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.734+0000 7f62d0ff9640 1 -- 172.21.15.64:0/3175972129 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f62cc004b20 con 0x7f62d410c180 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 Processor -- start 2024-09-19T05:15:08.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 -- start start 2024-09-19T05:15:08.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62d0ff9640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62bc0a3430 0x7f62bc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62bc0a3430 0x7f62bc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a639f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62d0ff9640 1 -- 172.21.15.64:0/3175972129 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f62cc0cfe00 con 0x7f62d410c180 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a644800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a647130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56137a59f280 con 0x56137a639b60 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56137a546b10 con 0x56137a63a480 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62dc49b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56137a59b240 con 0x56137a644d40 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.735+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a639f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a639f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44500/0 (socket says 172.21.15.64:44500) 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 -- 172.21.15.64:0/3858090312 learned_addr learned my addr 172.21.15.64:0/3858090312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a644800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62bc0a3430 0x7f62bc0a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62d4102440 tx=0x7f62c80103d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 -- 172.21.15.64:0/3858090312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 msgr2=0x56137a647130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a647130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 -- 172.21.15.64:0/3858090312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 msgr2=0x56137a644800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a644800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 -- 172.21.15.64:0/3858090312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137a5531f0 con 0x56137a639b60 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62d37fe640 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a639f40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f62c4006010 tx=0x7f62c40315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d134a96cebef2797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62b9ffb640 1 -- 172.21.15.64:0/3858090312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4035040 con 0x56137a639b60 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62b9ffb640 1 -- 172.21.15.64:0/3858090312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c4005020 con 0x56137a639b60 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.736+0000 7f62b9ffb640 1 -- 172.21.15.64:0/3858090312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4004aa0 con 0x56137a639b60 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 -- 172.21.15.64:0/3858090312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 msgr2=0x56137a639f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a639f40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f62c4006010 tx=0x7f62c40315d0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 -- 172.21.15.64:0/3858090312 shutdown_connections 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a647130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a644800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 --2- 172.21.15.64:0/3858090312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a639f40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 -- 172.21.15.64:0/3858090312 >> 172.21.15.64:0/3858090312 conn(0x56137a6360f0 msgr2=0x56137a6364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 -- 172.21.15.64:0/3858090312 shutdown_connections 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.737+0000 7f62dc49b740 1 -- 172.21.15.64:0/3858090312 wait complete. 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 -- start start 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 Processor -- start 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a639b60 0x56137a641b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a63a480 0x56137a6420d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 0x56137a642610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56137a647910 con 0x56137a644d40 2024-09-19T05:15:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56137a559f10 con 0x56137a639b60 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56137a647a90 con 0x56137a63a480 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62dc49b740 10 client.0 _ll_register_callbacks cb 0x56137a522a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 0x56137a642610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a639b60 0x56137a641b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 0x56137a642610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44508/0 (socket says 172.21.15.64:44508) 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.738+0000 7f62d3fff640 1 -- 172.21.15.64:0/2601576997 learned_addr learned my addr 172.21.15.64:0/2601576997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a63a480 0x56137a6420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d3fff640 1 -- 172.21.15.64:0/2601576997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a63a480 msgr2=0x56137a6420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d3fff640 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a63a480 0x56137a6420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d3fff640 1 -- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a639b60 msgr2=0x56137a641b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d3fff640 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a639b60 0x56137a641b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d3fff640 1 -- 172.21.15.64:0/2601576997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137a3106a0 con 0x56137a644d40 2024-09-19T05:15:08.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62d3fff640 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 0x56137a642610 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0cd5d0 tx=0x7f62cc004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702813446d735a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62b2ffd640 1 -- 172.21.15.64:0/2601576997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc01d030 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62b2ffd640 1 -- 172.21.15.64:0/2601576997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f62cc0987d0 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c4005020 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62b2ffd640 1 -- 172.21.15.64:0/2601576997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62cc0a4d20 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62dc49b740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.739+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56137a653530 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62b2ffd640 1 -- 172.21.15.64:0/2601576997 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c4005020 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62b2ffd640 1 -- 172.21.15.64:0/2601576997 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62cc0a4890 con 0x56137a644d40 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62dc49b740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62dc49b740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62dc49b740 20 client.17691 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62dc49b740 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a654c40 0x56137a657060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62dc49b740 4 client.17691 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56137a5ac7f0 con 0x56137a654c40 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.740+0000 7f62d37fe640 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a654c40 0x56137a657060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.741+0000 7f62b2ffd640 10 client.17691 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.741+0000 7f62d37fe640 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a654c40 0x56137a657060 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ea2e2cdf52ca9a server_cookie=d9f26cd98dca336b in_seq=0 out_seq=0 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.741+0000 7f62b2ffd640 1 -- 172.21.15.64:0/2601576997 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56137a5ac7f0 con 0x56137a654c40 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.741+0000 7f62b2ffd640 10 client.17691 handle_command_reply: tid=0 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.741+0000 7f62dc49b740 1 client.17691 shutdown 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 msgr2=0x56137a642610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 0x56137a642610 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0cd5d0 tx=0x7f62cc004ed0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 shutdown_connections 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a654c40 0x56137a657060 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a644d40 0x56137a642610 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a63a480 0x56137a6420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 --2- 172.21.15.64:0/2601576997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a639b60 0x56137a641b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 >> 172.21.15.64:0/2601576997 conn(0x56137a6360f0 msgr2=0x56137a63be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62b2ffd640 0 client.17691 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 shutdown_connections 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 1 -- 172.21.15.64:0/2601576997 wait complete. 2024-09-19T05:15:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.742+0000 7f62dc49b740 20 client.17691 trim_cache size 0 max 16384 2024-09-19T05:15:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.759+0000 7f62dc49b740 1 Processor -- start 2024-09-19T05:15:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.759+0000 7f62dc49b740 1 -- start start 2024-09-19T05:15:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.759+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a639b60 0x56137a63edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.759+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 0x56137a63f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a644d40 0x56137a634a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62dc49b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41b2310 con 0x56137a63a480 2024-09-19T05:15:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62dc49b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41afbe0 con 0x56137a644d40 2024-09-19T05:15:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62dc49b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41ada50 con 0x56137a639b60 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 0x56137a63f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 0x56137a63f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44522/0 (socket says 172.21.15.64:44522) 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 -- 172.21.15.64:0/1598017188 learned_addr learned my addr 172.21.15.64:0/1598017188 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d3fff640 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a644d40 0x56137a634a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d37fe640 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a639b60 0x56137a63edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 -- 172.21.15.64:0/1598017188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a639b60 msgr2=0x56137a63edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a639b60 0x56137a63edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 -- 172.21.15.64:0/1598017188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a644d40 msgr2=0x56137a634a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a644d40 0x56137a634a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 -- 172.21.15.64:0/1598017188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137a631830 con 0x56137a63a480 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 0x56137a63f330 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f62c80108d0 tx=0x7f62c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6ad42a6b75e345 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62b97fa640 1 -- 172.21.15.64:0/1598017188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8005020 con 0x56137a63a480 2024-09-19T05:15:08.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62b97fa640 1 -- 172.21.15.64:0/1598017188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c80047a0 con 0x56137a63a480 2024-09-19T05:15:08.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.760+0000 7f62b97fa640 1 -- 172.21.15.64:0/1598017188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c801f710 con 0x56137a63a480 2024-09-19T05:15:08.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 -- 172.21.15.64:0/1598017188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 msgr2=0x56137a63f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 0x56137a63f330 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f62c80108d0 tx=0x7f62c8015000 comp rx=0 tx=0).stop 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 -- 172.21.15.64:0/1598017188 shutdown_connections 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a644d40 0x56137a634a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a63a480 0x56137a63f330 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1598017188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a639b60 0x56137a63edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 -- 172.21.15.64:0/1598017188 >> 172.21.15.64:0/1598017188 conn(0x56137a6360f0 msgr2=0x56137a650530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 -- 172.21.15.64:0/1598017188 shutdown_connections 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.761+0000 7f62dc49b740 1 -- 172.21.15.64:0/1598017188 wait complete. 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 -- start start 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 Processor -- start 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a6599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a659ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a6bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41ae0a0 con 0x56137a639b60 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41af510 con 0x56137a63a480 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41ae770 con 0x56137a644d40 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62dc49b740 10 client.0 _ll_register_callbacks cb 0x7f62cc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a6599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62d37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a6599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44534/0 (socket says 172.21.15.64:44534) 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62d37fe640 1 -- 172.21.15.64:0/1933341826 learned_addr learned my addr 172.21.15.64:0/1933341826 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a659ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.762+0000 7f62d3fff640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a6bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62d37fe640 1 -- 172.21.15.64:0/1933341826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 msgr2=0x56137a6bd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62d37fe640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a6bd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62d37fe640 1 -- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 msgr2=0x56137a659ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62d37fe640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a659ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62d37fe640 1 -- 172.21.15.64:0/1933341826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137a6bdd40 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62d37fe640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a6599a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009a20 tx=0x7f62c4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a121cbe6df27905 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62b2ffd640 1 -- 172.21.15.64:0/1933341826 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4057020 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62b2ffd640 1 -- 172.21.15.64:0/1933341826 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8571+0+0 (secure 0 0 0) 0x7f62c4005040 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c80047a0 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62b2ffd640 1 -- 172.21.15.64:0/1933341826 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62c4062050 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62dc49b740 10 client.17697 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56137a6befa0 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.763+0000 7f62b2ffd640 1 -- 172.21.15.64:0/1933341826 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c80047a0 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62b2ffd640 1 -- 172.21.15.64:0/1933341826 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f62c4052420 con 0x56137a639b60 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62dc49b740 10 client.17697 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:08.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62dc49b740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62dc49b740 20 client.17697 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a6bf300 0x56137a6c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62dc49b740 4 client.17697 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56137a5a6d00 con 0x56137a6bf300 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a6bf300 0x56137a6c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62d2ffd640 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a6bf300 0x56137a6c16f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb5e07afe75997f server_cookie=875fd7252cce7619 in_seq=0 out_seq=0 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.764+0000 7f62b2ffd640 10 client.17697 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62b2ffd640 1 -- 172.21.15.64:0/1933341826 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56137a5a6d00 con 0x56137a6bf300 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62b2ffd640 10 client.17697 handle_command_reply: tid=0 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 client.17697 shutdown 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 msgr2=0x56137a6599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a6599a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009a20 tx=0x7f62c4005e50 comp rx=0 tx=0).stop 2024-09-19T05:15:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 shutdown_connections 2024-09-19T05:15:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137a644d40 0x56137a6bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56137a63a480 0x56137a659ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56137a6bf300 0x56137a6c16f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 --2- 172.21.15.64:0/1933341826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56137a639b60 0x56137a6599a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 >> 172.21.15.64:0/1933341826 conn(0x56137a6360f0 msgr2=0x56137a634a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.765+0000 7f62b2ffd640 0 client.17697 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 shutdown_connections 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62dc49b740 1 -- 172.21.15.64:0/1933341826 wait complete. 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62dc49b740 20 client.17697 trim_cache size 0 max 16384 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "adea0732-116c-41bf-b1aa-2df48fea02e6", 2024-09-19T05:15:08.773 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:15:08.774 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:15:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62bc0a3430 msgr2=0x7f62bc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62bc0a3430 0x7f62bc0a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62d4102440 tx=0x7f62c80103d0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 msgr2=0x7f62d4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 0x7f62d4101430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00ca30 tx=0x7f62cc00cef0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 shutdown_connections 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f62bc0a3430 0x7f62bc0a58f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62d410c180 0x7f62d4101430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62d4103670 0x7f62d4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 --2- 172.21.15.64:0/3175972129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62d4102d50 0x7f62d41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 >> 172.21.15.64:0/3175972129 conn(0x7f62d40fc7d0 msgr2=0x7f62d40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.766+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 shutdown_connections 2024-09-19T05:15:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:08.767+0000 7f62da0c2640 1 -- 172.21.15.64:0/3175972129 wait complete. 2024-09-19T05:15:09.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:09 smithi064 ceph-mon[30960]: pgmap v284: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-19T05:15:09.432 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:09 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:15:09.432 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:09 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:15:09.432 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:09 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:15:09.432 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:09 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:15:09.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:15:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:09 smithi103 ceph-mon[34446]: pgmap v284: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-19T05:15:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:09 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:15:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:09 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:15:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:09 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:15:09.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:09 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:15:09.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:09 smithi174 ceph-mon[34488]: pgmap v284: 97 pgs: 97 active+clean; 25 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-19T05:15:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:09 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:15:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:09 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:15:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:09 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:15:09.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:09 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:15:09.696 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:10.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 -- 172.21.15.64:0/3397123377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184102790 msgr2=0x7f21841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 --2- 172.21.15.64:0/3397123377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184102790 0x7f21841067e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f216c009900 tx=0x7f216c02a410 comp rx=0 tx=0).stop 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 -- 172.21.15.64:0/3397123377 shutdown_connections 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 --2- 172.21.15.64:0/3397123377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2184106f10 0x7f218410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 --2- 172.21.15.64:0/3397123377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184102790 0x7f21841067e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 --2- 172.21.15.64:0/3397123377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2184101de0 0x7f21841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 -- 172.21.15.64:0/3397123377 >> 172.21.15.64:0/3397123377 conn(0x7f21840fd8c0 msgr2=0x7f21840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 -- 172.21.15.64:0/3397123377 shutdown_connections 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.339+0000 7f2189416640 1 -- 172.21.15.64:0/3397123377 wait complete. 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 Processor -- start 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 -- start start 2024-09-19T05:15:10.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2184101de0 0x7f2184198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2184102790 0x7f2184198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 0x7f218419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2184111e40 con 0x7f2184106f10 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2184111cc0 con 0x7f2184101de0 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f2189416640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2184111fc0 con 0x7f2184102790 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.340+0000 7f21837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 0x7f218419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 0x7f218419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44554/0 (socket says 172.21.15.64:44554) 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 -- 172.21.15.64:0/2692617485 learned_addr learned my addr 172.21.15.64:0/2692617485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f2182ffd640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2184101de0 0x7f2184198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f217bfff640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2184102790 0x7f2184198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 -- 172.21.15.64:0/2692617485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2184102790 msgr2=0x7f2184198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2184102790 0x7f2184198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 -- 172.21.15.64:0/2692617485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2184101de0 msgr2=0x7f2184198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2184101de0 0x7f2184198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 -- 172.21.15.64:0/2692617485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218419d3d0 con 0x7f2184106f10 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f21837fe640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 0x7f218419ccf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f217c00eaa0 tx=0x7f217c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f2180ff9640 1 -- 172.21.15.64:0/2692617485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c004d20 con 0x7f2184106f10 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f2180ff9640 1 -- 172.21.15.64:0/2692617485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f217c005020 con 0x7f2184106f10 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218419d6c0 con 0x7f2184106f10 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f2180ff9640 1 -- 172.21.15.64:0/2692617485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c0116b0 con 0x7f2184106f10 2024-09-19T05:15:10.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.341+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21841a4fa0 con 0x7f2184106f10 2024-09-19T05:15:10.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.344+0000 7f218b7ef740 1 Processor -- start 2024-09-19T05:15:10.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.344+0000 7f2180ff9640 1 -- 172.21.15.64:0/2692617485 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f217c0051d0 con 0x7f2184106f10 2024-09-19T05:15:10.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 -- start start 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d74a0 0x5584689d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689e2680 0x5584689e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55846893d2e0 con 0x5584689d7dc0 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5584688e4ce0 con 0x5584689d74a0 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f218b7ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584689392a0 con 0x5584689e2680 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44562/0 (socket says 172.21.15.64:44562) 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 -- 172.21.15.64:0/88041376 learned_addr learned my addr 172.21.15.64:0/88041376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f217bfff640 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689e2680 0x5584689e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f217bfff640 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d74a0 0x5584689d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 -- 172.21.15.64:0/88041376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689e2680 msgr2=0x5584689e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689e2680 0x5584689e4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 -- 172.21.15.64:0/88041376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d74a0 msgr2=0x5584689d7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d74a0 0x5584689d7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2182ffd640 1 -- 172.21.15.64:0/88041376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584688c9440 con 0x5584689d7dc0 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f217bfff640 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d74a0 0x5584689d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.345+0000 7f2180ff9640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21700a33b0 0x7f21700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.346+0000 7f217bfff640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21700a33b0 0x7f21700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.346+0000 7f2180ff9640 1 -- 172.21.15.64:0/2692617485 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f217c0cff10 con 0x7f2184106f10 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.346+0000 7f2182ffd640 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e2140 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2174009a70 tx=0x7f2174009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e518951acd6b664a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.346+0000 7f21797fa640 1 -- 172.21.15.64:0/88041376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174017070 con 0x5584689d7dc0 2024-09-19T05:15:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.346+0000 7f21797fa640 1 -- 172.21.15.64:0/88041376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2174013470 con 0x5584689d7dc0 2024-09-19T05:15:10.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.346+0000 7f21797fa640 1 -- 172.21.15.64:0/88041376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174012630 con 0x5584689d7dc0 2024-09-19T05:15:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 -- 172.21.15.64:0/88041376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 msgr2=0x5584689e2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f217bfff640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21700a33b0 0x7f21700a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f216c031440 tx=0x7f216c005780 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e2140 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2174009a70 tx=0x7f2174009f30 comp rx=0 tx=0).stop 2024-09-19T05:15:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 -- 172.21.15.64:0/88041376 shutdown_connections 2024-09-19T05:15:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689e2680 0x5584689e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e2140 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 --2- 172.21.15.64:0/88041376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d74a0 0x5584689d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 -- 172.21.15.64:0/88041376 >> 172.21.15.64:0/88041376 conn(0x5584689d3a30 msgr2=0x5584689d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 -- 172.21.15.64:0/88041376 shutdown_connections 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 -- 172.21.15.64:0/88041376 wait complete. 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 -- start start 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.347+0000 7f218b7ef740 1 Processor -- start 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5584689e5250 con 0x5584689e2680 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5584688f8050 con 0x5584689d7dc0 2024-09-19T05:15:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584689e53d0 con 0x5584689d74a0 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f218b7ef740 10 client.0 _ll_register_callbacks cb 0x558468938160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689df750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44568/0 (socket says 172.21.15.64:44568) 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 -- 172.21.15.64:0/1027627122 learned_addr learned my addr 172.21.15.64:0/1027627122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f2182ffd640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f217bfff640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 -- 172.21.15.64:0/1027627122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 msgr2=0x5584689e1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 -- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 msgr2=0x5584689df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689df210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 -- 172.21.15.64:0/1027627122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584686ae3c0 con 0x5584689e2680 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f217bfff640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f21837fe640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689df750 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f217c098b60 tx=0x7f217c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd0ad62670e76c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f216a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.348+0000 7f216a7fc640 1 -- 172.21.15.64:0/1027627122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c01f030 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f216a7fc640 1 -- 172.21.15.64:0/1027627122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f217c0a0120 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2174013470 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f216a7fc640 1 -- 172.21.15.64:0/1027627122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f217c0bc270 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 10 client.17709 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584689f0b90 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f216a7fc640 1 -- 172.21.15.64:0/1027627122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2174013470 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f216a7fc640 1 -- 172.21.15.64:0/1027627122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f217c00f040 con 0x5584689e2680 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 10 client.17709 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 20 client.17709 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5584689f2050 0x5584689f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 4 client.17709 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.349+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846894a850 con 0x5584689f2050 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.350+0000 7f2182ffd640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5584689f2050 0x5584689f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.350+0000 7f216a7fc640 10 client.17709 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.350+0000 7f2182ffd640 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5584689f2050 0x5584689f44d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1b66535b4bbfe0 server_cookie=890a3a8358e84afa in_seq=0 out_seq=0 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f216a7fc640 1 -- 172.21.15.64:0/1027627122 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55846894a850 con 0x5584689f2050 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f216a7fc640 10 client.17709 handle_command_reply: tid=0 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 client.17709 shutdown 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 msgr2=0x5584689df750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689df750 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f217c098b60 tx=0x7f217c005620 comp rx=0 tx=0).stop 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 shutdown_connections 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5584689f2050 0x5584689f44d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689df750 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 --2- 172.21.15.64:0/1027627122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 >> 172.21.15.64:0/1027627122 conn(0x5584689d3a30 msgr2=0x5584689d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f216a7fc640 0 client.17709 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 shutdown_connections 2024-09-19T05:15:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 1 -- 172.21.15.64:0/1027627122 wait complete. 2024-09-19T05:15:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.351+0000 7f218b7ef740 20 client.17709 trim_cache size 0 max 16384 2024-09-19T05:15:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 Processor -- start 2024-09-19T05:15:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 -- start start 2024-09-19T05:15:10.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:10 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:15:10.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:10 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:10.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:10 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:15:10.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:10 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:10.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:10 smithi064 ceph-mon[30960]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:15:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21841b0320 con 0x5584689e2680 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21841adbf0 con 0x5584689d7dc0 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.368+0000 7f218b7ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21841aba60 con 0x5584689d74a0 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689fa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44576/0 (socket says 172.21.15.64:44576) 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 -- 172.21.15.64:0/175090335 learned_addr learned my addr 172.21.15.64:0/175090335 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f2182ffd640 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f217bfff640 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 -- 172.21.15.64:0/175090335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 msgr2=0x5584689e1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 -- 172.21.15.64:0/175090335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 msgr2=0x5584689f9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689f9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 -- 172.21.15.64:0/175090335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584689d7bf0 con 0x5584689e2680 2024-09-19T05:15:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f217bfff640 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689f9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.369+0000 7f21837fe640 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689fa330 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f217c09be90 tx=0x7f217c09c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275a947dba2bd071 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.370+0000 7f2178ff9640 1 -- 172.21.15.64:0/175090335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c01f030 con 0x5584689e2680 2024-09-19T05:15:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.370+0000 7f2178ff9640 1 -- 172.21.15.64:0/175090335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f217c0bc0c0 con 0x5584689e2680 2024-09-19T05:15:10.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.370+0000 7f2178ff9640 1 -- 172.21.15.64:0/175090335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c0a4e00 con 0x5584689e2680 2024-09-19T05:15:10.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.370+0000 7f218b7ef740 1 -- 172.21.15.64:0/175090335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 msgr2=0x5584689fa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.370+0000 7f218b7ef740 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689fa330 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f217c09be90 tx=0x7f217c09c310 comp rx=0 tx=0).stop 2024-09-19T05:15:10.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- 172.21.15.64:0/175090335 shutdown_connections 2024-09-19T05:15:10.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689e2680 0x5584689fa330 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689d7dc0 0x5584689f9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 --2- 172.21.15.64:0/175090335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689e1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- 172.21.15.64:0/175090335 >> 172.21.15.64:0/175090335 conn(0x5584689d3a30 msgr2=0x5584689e0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- 172.21.15.64:0/175090335 shutdown_connections 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- 172.21.15.64:0/175090335 wait complete. 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- start start 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 Processor -- start 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689e2680 0x5584689e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21841ac0b0 con 0x5584689d7dc0 2024-09-19T05:15:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21841ad520 con 0x5584689e2680 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.371+0000 7f218b7ef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21841ac780 con 0x5584689d74a0 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f218b7ef740 10 client.0 _ll_register_callbacks cb 0x7f217c004ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44592/0 (socket says 172.21.15.64:44592) 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 -- 172.21.15.64:0/2508351346 learned_addr learned my addr 172.21.15.64:0/2508351346 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f21837fe640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689e2680 0x5584689e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f2182ffd640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 -- 172.21.15.64:0/2508351346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 msgr2=0x5584689eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689eaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 -- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689e2680 msgr2=0x5584689e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689e2680 0x5584689e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 -- 172.21.15.64:0/2508351346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584689e8810 con 0x5584689d7dc0 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f21837fe640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689e2680 0x5584689e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f217bfff640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e7d90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f216c009900 tx=0x7f216c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b010390b3205365 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f216a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.372+0000 7f216a7fc640 1 -- 172.21.15.64:0/2508351346 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c043070 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f216a7fc640 1 -- 172.21.15.64:0/2508351346 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f216c0035f0 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f217c0bc0c0 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f216a7fc640 1 -- 172.21.15.64:0/2508351346 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f216c035040 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 10 client.17715 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558468a5cab0 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f216a7fc640 1 -- 172.21.15.64:0/2508351346 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f217c0bc0c0 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f216a7fc640 1 -- 172.21.15.64:0/2508351346 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f216c051070 con 0x5584689d7dc0 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 10 client.17715 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 20 client.17715 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:10.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558468a5d200 0x558468a5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 4 client.17715 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.373+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558468944d60 con 0x558468a5d200 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f2182ffd640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558468a5d200 0x558468a5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f216a7fc640 10 client.17715 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f2182ffd640 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558468a5d200 0x558468a5f620 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d2b7ba14ed71969 server_cookie=5dc907447ad58e19 in_seq=0 out_seq=0 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f216a7fc640 1 -- 172.21.15.64:0/2508351346 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558468944d60 con 0x558468a5d200 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f216a7fc640 10 client.17715 handle_command_reply: tid=0 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f218b7ef740 1 client.17715 shutdown 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 msgr2=0x5584689e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f218b7ef740 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e7d90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f216c009900 tx=0x7f216c005db0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 shutdown_connections 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f218b7ef740 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584689e2680 0x5584689e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.374+0000 7f218b7ef740 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x558468a5d200 0x558468a5f620 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f218b7ef740 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584689d7dc0 0x5584689e7d90 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f218b7ef740 1 --2- 172.21.15.64:0/2508351346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584689d74a0 0x5584689eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f216a7fc640 0 client.17715 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 >> 172.21.15.64:0/2508351346 conn(0x5584689d3a30 msgr2=0x5584689e1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 shutdown_connections 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f218b7ef740 1 -- 172.21.15.64:0/2508351346 wait complete. 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f218b7ef740 20 client.17715 trim_cache size 0 max 16384 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:15:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout: "adea0732-116c-41bf-b1aa-2df48fea02e6": { 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "adea0732-116c-41bf-b1aa-2df48fea02e6", 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:15:10.385 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21700a33b0 msgr2=0x7f21700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21700a33b0 0x7f21700a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f216c031440 tx=0x7f216c005780 comp rx=0 tx=0).stop 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 msgr2=0x7f218419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 0x7f218419ccf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f217c00eaa0 tx=0x7f217c00eea0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 shutdown_connections 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f21700a33b0 0x7f21700a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2184106f10 0x7f218419ccf0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2184102790 0x7f2184198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 --2- 172.21.15.64:0/2692617485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2184101de0 0x7f2184198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:10.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.375+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 >> 172.21.15.64:0/2692617485 conn(0x7f21840fd8c0 msgr2=0x7f21840fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:10.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.376+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 shutdown_connections 2024-09-19T05:15:10.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:10.376+0000 7f2189416640 1 -- 172.21.15.64:0/2692617485 wait complete. 2024-09-19T05:15:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:10 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:15:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:10 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:10 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:15:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:10 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:10.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:10 smithi103 ceph-mon[34446]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:15:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:10 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:15:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:10 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:10 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:15:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:10 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:10.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:10 smithi174 ceph-mon[34488]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:15:10.967 INFO:tasks.cephfs.filesystem:scrub status for tag:adea0732-116c-41bf-b1aa-2df48fea02e6 - {'path': '/', 'tag': 'adea0732-116c-41bf-b1aa-2df48fea02e6', 'options': 'recursive,force'} 2024-09-19T05:15:11.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:11 smithi064 ceph-mon[30960]: pgmap v286: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:11.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:11 smithi103 ceph-mon[34446]: pgmap v286: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:11.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:11 smithi174 ceph-mon[34488]: pgmap v286: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:13 smithi103 ceph-mon[34446]: pgmap v287: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:13.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:13 smithi174 ceph-mon[34488]: pgmap v287: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:13.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:13 smithi064 ceph-mon[30960]: pgmap v287: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:14.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:14 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:15:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:14 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:15:14.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:14 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:15:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:15 smithi103 ceph-mon[34446]: pgmap v288: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:15.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:15 smithi174 ceph-mon[34488]: pgmap v288: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:15.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:15 smithi064 ceph-mon[30960]: pgmap v288: 97 pgs: 97 active+clean; 23 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-09-19T05:15:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:17 smithi103 ceph-mon[34446]: pgmap v289: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-19T05:15:17.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:17 smithi103 ceph-mon[34446]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:15:17.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:17 smithi174 ceph-mon[34488]: pgmap v289: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-19T05:15:17.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:17 smithi174 ceph-mon[34488]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:15:17.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:17 smithi064 ceph-mon[30960]: pgmap v289: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-19T05:15:17.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:17 smithi064 ceph-mon[30960]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:15:19.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:19 smithi103 ceph-mon[34446]: pgmap v291: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 235 B/s wr, 8 op/s 2024-09-19T05:15:19.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:19 smithi174 ceph-mon[34488]: pgmap v291: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 235 B/s wr, 8 op/s 2024-09-19T05:15:19.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:19 smithi064 ceph-mon[30960]: pgmap v291: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 235 B/s wr, 8 op/s 2024-09-19T05:15:21.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:21 smithi064 ceph-mon[30960]: pgmap v292: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:21.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:21 smithi103 ceph-mon[34446]: pgmap v292: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:21.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:21 smithi174 ceph-mon[34488]: pgmap v292: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:23.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:23 smithi103 ceph-mon[34446]: pgmap v293: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:23.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:23 smithi174 ceph-mon[34488]: pgmap v293: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:23.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:23 smithi064 ceph-mon[30960]: pgmap v293: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:25.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:25 smithi103 ceph-mon[34446]: pgmap v294: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:25.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:25 smithi174 ceph-mon[34488]: pgmap v294: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:25.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:25 smithi064 ceph-mon[30960]: pgmap v294: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:15:27.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:27 smithi103 ceph-mon[34446]: pgmap v295: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T05:15:27.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:27 smithi174 ceph-mon[34488]: pgmap v295: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T05:15:27.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:27 smithi064 ceph-mon[30960]: pgmap v295: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-19T05:15:29.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:29 smithi174 ceph-mon[34488]: pgmap v296: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T05:15:29.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:29 smithi064 ceph-mon[30960]: pgmap v296: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T05:15:29.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:29 smithi103 ceph-mon[34446]: pgmap v296: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T05:15:31.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:31 smithi174 ceph-mon[34488]: pgmap v297: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:31 smithi064 ceph-mon[30960]: pgmap v297: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:31.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:31 smithi103 ceph-mon[34446]: pgmap v297: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:33.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:33 smithi103 ceph-mon[34446]: pgmap v298: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:33.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:33 smithi174 ceph-mon[34488]: pgmap v298: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:33.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:33 smithi064 ceph-mon[30960]: pgmap v298: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:35.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:35 smithi174 ceph-mon[34488]: pgmap v299: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:35 smithi064 ceph-mon[30960]: pgmap v299: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:35 smithi103 ceph-mon[34446]: pgmap v299: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:37.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:37 smithi174 ceph-mon[34488]: pgmap v300: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:37.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:37 smithi064 ceph-mon[30960]: pgmap v300: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:37.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:37 smithi103 ceph-mon[34446]: pgmap v300: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:39.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:39 smithi174 ceph-mon[34488]: pgmap v301: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:39 smithi064 ceph-mon[30960]: pgmap v301: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:39.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:39 smithi103 ceph-mon[34446]: pgmap v301: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:40.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:15:41.247 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:41.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:41 smithi064 ceph-mon[30960]: pgmap v302: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:41.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:41 smithi174 ceph-mon[34488]: pgmap v302: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:41.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.689+0000 7f1a63931640 1 -- 172.21.15.64:0/171799297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c100a80 msgr2=0x7f1a5c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.689+0000 7f1a63931640 1 --2- 172.21.15.64:0/171799297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c100a80 0x7f1a5c100ee0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009930 tx=0x7f1a4c02a420 comp rx=0 tx=0).stop 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 -- 172.21.15.64:0/171799297 shutdown_connections 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 --2- 172.21.15.64:0/171799297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a5c101420 0x7f1a5c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 --2- 172.21.15.64:0/171799297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c100a80 0x7f1a5c100ee0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 --2- 172.21.15.64:0/171799297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a5c106a80 0x7f1a5c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 -- 172.21.15.64:0/171799297 >> 172.21.15.64:0/171799297 conn(0x7f1a5c0fc7a0 msgr2=0x7f1a5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 -- 172.21.15.64:0/171799297 shutdown_connections 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 -- 172.21.15.64:0/171799297 wait complete. 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 Processor -- start 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.690+0000 7f1a63931640 1 -- start start 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a63931640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a5c100a80 0x7f1a5c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a63931640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 0x7f1a5c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a63931640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a5c106a80 0x7f1a5c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a63931640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c110cf0 con 0x7f1a5c101420 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a63931640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c110b70 con 0x7f1a5c106a80 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a63931640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c110e70 con 0x7f1a5c100a80 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 0x7f1a5c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 0x7f1a5c1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46970/0 (socket says 172.21.15.64:46970) 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 -- 172.21.15.64:0/844608437 learned_addr learned my addr 172.21.15.64:0/844608437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:41.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a616cd640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a5c100a80 0x7f1a5c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a61ece640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a5c106a80 0x7f1a5c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 -- 172.21.15.64:0/844608437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a5c100a80 msgr2=0x7f1a5c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a5c100a80 0x7f1a5c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 -- 172.21.15.64:0/844608437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a5c106a80 msgr2=0x7f1a5c19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a5c106a80 0x7f1a5c19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 -- 172.21.15.64:0/844608437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c19c230 con 0x7f1a5c101420 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.691+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 0x7f1a5c1977c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c005940 tx=0x7f1a4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.692+0000 7f1a527fc640 1 -- 172.21.15.64:0/844608437 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c03f030 con 0x7f1a5c101420 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.692+0000 7f1a527fc640 1 -- 172.21.15.64:0/844608437 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a4c0318a0 con 0x7f1a5c101420 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.692+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c19c4c0 con 0x7f1a5c101420 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.692+0000 7f1a527fc640 1 -- 172.21.15.64:0/844608437 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c035070 con 0x7f1a5c101420 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.692+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c1a3e00 con 0x7f1a5c101420 2024-09-19T05:15:41.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.694+0000 7f1a527fc640 1 -- 172.21.15.64:0/844608437 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1a4c042030 con 0x7f1a5c101420 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.694+0000 7f1a65d0a740 1 Processor -- start 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.694+0000 7f1a65d0a740 1 -- start start 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf8a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf95140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0cf97a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561b0cef02e0 con 0x561b0cf95680 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b0ce97ce0 con 0x561b0cf8a4a0 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b0ceec2a0 con 0x561b0cf8adc0 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0cf97a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0cf97a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46984/0 (socket says 172.21.15.64:46984) 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 -- 172.21.15.64:0/1548730435 learned_addr learned my addr 172.21.15.64:0/1548730435 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:41.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a527fc640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a480a3430 0x7f1a480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a616cd640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf95140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a61ece640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a480a3430 0x7f1a480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a61ece640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf8a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a527fc640 1 -- 172.21.15.64:0/844608437 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a4c0f0150 con 0x7f1a5c101420 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 -- 172.21.15.64:0/1548730435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 msgr2=0x561b0cf95140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf95140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 -- 172.21.15.64:0/1548730435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 msgr2=0x561b0cf8a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf8a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a60ecc640 1 -- 172.21.15.64:0/1548730435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0ce7c440 con 0x561b0cf95680 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.695+0000 7f1a616cd640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf95140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:41.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.696+0000 7f1a61ece640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf8a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.696+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0cf97a70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0ed750 tx=0x7f1a4c0b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244260bb72b09131 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.696+0000 7f1a477fe640 1 -- 172.21.15.64:0/1548730435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c049070 con 0x561b0cf95680 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.696+0000 7f1a61ece640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a480a3430 0x7f1a480a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a54005fd0 tx=0x7f1a54009ac0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.696+0000 7f1a477fe640 1 -- 172.21.15.64:0/1548730435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a4c003e20 con 0x561b0cf95680 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.696+0000 7f1a477fe640 1 -- 172.21.15.64:0/1548730435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0c0750 con 0x561b0cf95680 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1548730435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 msgr2=0x561b0cf97a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0cf97a70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0ed750 tx=0x7f1a4c0b8040 comp rx=0 tx=0).stop 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1548730435 shutdown_connections 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0cf97a70 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf95140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1548730435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf8a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1548730435 >> 172.21.15.64:0/1548730435 conn(0x561b0cf86a30 msgr2=0x561b0cf86e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1548730435 shutdown_connections 2024-09-19T05:15:41.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1548730435 wait complete. 2024-09-19T05:15:41.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 -- start start 2024-09-19T05:15:41.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.697+0000 7f1a65d0a740 1 Processor -- start 2024-09-19T05:15:41.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cf95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561b0cf9a220 con 0x561b0cf8a4a0 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b0ceab050 con 0x561b0cf95680 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b0cf9a3a0 con 0x561b0cf8adc0 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cf95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a65d0a740 10 client.0 _ll_register_callbacks cb 0x561b0ceeb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cf95000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46994/0 (socket says 172.21.15.64:46994) 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 -- 172.21.15.64:0/90606011 learned_addr learned my addr 172.21.15.64:0/90606011 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a61ece640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 -- 172.21.15.64:0/90606011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 msgr2=0x561b0cf922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf922b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 -- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 msgr2=0x561b0cf92830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf92830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 -- 172.21.15.64:0/90606011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0cc613c0 con 0x561b0cf8a4a0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a616cd640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cf95000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800be70 tx=0x7f1a58012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b414b77aa83654 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.698+0000 7f1a44ff9640 1 -- 172.21.15.64:0/90606011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5801f060 con 0x561b0cf8a4a0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a44ff9640 1 -- 172.21.15.64:0/90606011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a58010910 con 0x561b0cf8a4a0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561b0cefd850 con 0x561b0cf8a4a0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a44ff9640 1 -- 172.21.15.64:0/90606011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a58010b10 con 0x561b0cf8a4a0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 10 client.17727 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561b0cfa5ad0 con 0x561b0cf8a4a0 2024-09-19T05:15:41.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a44ff9640 1 -- 172.21.15.64:0/90606011 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561b0cefd850 con 0x561b0cf8a4a0 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a44ff9640 1 -- 172.21.15.64:0/90606011 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a58019030 con 0x561b0cf8a4a0 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 10 client.17727 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 20 client.17727 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0cfa6f90 0x561b0cfa93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 4 client.17727 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b0cef7d60 con 0x561b0cfa6f90 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.699+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0cfa6f90 0x561b0cfa93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.700+0000 7f1a44ff9640 10 client.17727 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.700+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0cfa6f90 0x561b0cfa93e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4767f630d7094b5c server_cookie=ba8d623fc627f30c in_seq=0 out_seq=0 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a44ff9640 1 -- 172.21.15.64:0/90606011 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b0cef7d60 con 0x561b0cfa6f90 2024-09-19T05:15:41.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a44ff9640 10 client.17727 handle_command_reply: tid=0 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 client.17727 shutdown 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 msgr2=0x561b0cf95000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cf95000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800be70 tx=0x7f1a58012820 comp rx=0 tx=0).stop 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 shutdown_connections 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf92830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0cfa6f90 0x561b0cfa93e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/90606011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cf95000 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 >> 172.21.15.64:0/90606011 conn(0x561b0cf86a30 msgr2=0x561b0cf8c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a44ff9640 0 client.17727 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 shutdown_connections 2024-09-19T05:15:41.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 1 -- 172.21.15.64:0/90606011 wait complete. 2024-09-19T05:15:41.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.701+0000 7f1a65d0a740 20 client.17727 trim_cache size 0 max 16384 2024-09-19T05:15:41.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:41 smithi103 ceph-mon[34446]: pgmap v302: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:41.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 Processor -- start 2024-09-19T05:15:41.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 -- start start 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cfa3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf90b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf8d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1adfe0 con 0x561b0cf8a4a0 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1ab8b0 con 0x561b0cf95680 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1a9720 con 0x561b0cf8adc0 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a616cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cfa3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a616cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cfa3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46996/0 (socket says 172.21.15.64:46996) 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.718+0000 7f1a616cd640 1 -- 172.21.15.64:0/3241777821 learned_addr learned my addr 172.21.15.64:0/3241777821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a61ece640 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf8d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf90b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a616cd640 1 -- 172.21.15.64:0/3241777821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 msgr2=0x561b0cf90b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a616cd640 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf90b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a616cd640 1 -- 172.21.15.64:0/3241777821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 msgr2=0x561b0cf8d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a616cd640 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf8d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a616cd640 1 -- 172.21.15.64:0/3241777821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0cf91260 con 0x561b0cf8a4a0 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf90b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a616cd640 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cfa3390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f1a58023000 tx=0x7f1a58005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464d2943ce6cc6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a46ffd640 1 -- 172.21.15.64:0/3241777821 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58010e30 con 0x561b0cf8a4a0 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a46ffd640 1 -- 172.21.15.64:0/3241777821 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a58010710 con 0x561b0cf8a4a0 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.719+0000 7f1a46ffd640 1 -- 172.21.15.64:0/3241777821 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004660 con 0x561b0cf8a4a0 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 -- 172.21.15.64:0/3241777821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 msgr2=0x561b0cfa3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cfa3390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f1a58023000 tx=0x7f1a58005950 comp rx=0 tx=0).stop 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 -- 172.21.15.64:0/3241777821 shutdown_connections 2024-09-19T05:15:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf95680 0x561b0cf8d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf90b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/3241777821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf8a4a0 0x561b0cfa3390 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 -- 172.21.15.64:0/3241777821 >> 172.21.15.64:0/3241777821 conn(0x561b0cf86a30 msgr2=0x561b0cf8d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 -- 172.21.15.64:0/3241777821 shutdown_connections 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 -- 172.21.15.64:0/3241777821 wait complete. 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 -- start start 2024-09-19T05:15:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.720+0000 7f1a65d0a740 1 Processor -- start 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf9ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf9b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0d009c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1a9d70 con 0x561b0cf95680 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1ab1e0 con 0x561b0cf8a4a0 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1aa440 con 0x561b0cf8adc0 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 10 client.0 _ll_register_callbacks cb 0x7f1a5800ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0d009c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0d009c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47006/0 (socket says 172.21.15.64:47006) 2024-09-19T05:15:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 -- 172.21.15.64:0/1309447939 learned_addr learned my addr 172.21.15.64:0/1309447939 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a616cd640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf9ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a60ecc640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf9b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 -- 172.21.15.64:0/1309447939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 msgr2=0x561b0cf9b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf9b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 -- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 msgr2=0x561b0cf9ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf9ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 -- 172.21.15.64:0/1309447939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0cf92ba0 con 0x561b0cf95680 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a61ece640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0d009c40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1a54013df0 tx=0x7f1a54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b36858ca451f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a44ff9640 1 -- 172.21.15.64:0/1309447939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5402c060 con 0x561b0cf95680 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a44ff9640 1 -- 172.21.15.64:0/1309447939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a5401c640 con 0x561b0cf95680 2024-09-19T05:15:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.721+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a58010710 con 0x561b0cf95680 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a44ff9640 1 -- 172.21.15.64:0/1309447939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a5401c840 con 0x561b0cf95680 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 10 client.17733 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561b0d0113b0 con 0x561b0cf95680 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a44ff9640 1 -- 172.21.15.64:0/1309447939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a58010710 con 0x561b0cf95680 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a44ff9640 1 -- 172.21.15.64:0/1309447939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a5401a950 con 0x561b0cf95680 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 10 client.17733 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 20 client.17733 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0d011b90 0x561b0d013fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 4 client.17733 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b0ce74210 con 0x561b0d011b90 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.722+0000 7f1a616cd640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0d011b90 0x561b0d013fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a44ff9640 10 client.17733 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a616cd640 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0d011b90 0x561b0d013fb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827312161786cdb6 server_cookie=ab6fb75fb9239d25 in_seq=0 out_seq=0 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a44ff9640 1 -- 172.21.15.64:0/1309447939 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561b0ce74210 con 0x561b0d011b90 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a44ff9640 10 client.17733 handle_command_reply: tid=0 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 client.17733 shutdown 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 msgr2=0x561b0d009c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0d009c40 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1a54013df0 tx=0x7f1a54004ab0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 shutdown_connections 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x561b0d011b90 0x561b0d013fb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561b0cf95680 0x561b0d009c40 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b0cf8adc0 0x561b0cf9b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 --2- 172.21.15.64:0/1309447939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b0cf8a4a0 0x561b0cf9ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 >> 172.21.15.64:0/1309447939 conn(0x561b0cf86a30 msgr2=0x561b0cf85350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a44ff9640 0 client.17733 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 shutdown_connections 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 1 -- 172.21.15.64:0/1309447939 wait complete. 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.723+0000 7f1a65d0a740 20 client.17733 trim_cache size 0 max 16384 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:15:41.731 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:15:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a480a3430 msgr2=0x7f1a480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a480a3430 0x7f1a480a58f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a54005fd0 tx=0x7f1a54009ac0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 msgr2=0x7f1a5c1977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 0x7f1a5c1977c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c005940 tx=0x7f1a4c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 shutdown_connections 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a480a3430 0x7f1a480a58f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a5c106a80 0x7f1a5c19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a5c101420 0x7f1a5c1977c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 --2- 172.21.15.64:0/844608437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a5c100a80 0x7f1a5c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 >> 172.21.15.64:0/844608437 conn(0x7f1a5c0fc7a0 msgr2=0x7f1a5c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 shutdown_connections 2024-09-19T05:15:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:41.724+0000 7f1a63931640 1 -- 172.21.15.64:0/844608437 wait complete. 2024-09-19T05:15:42.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:15:42.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:15:42.806 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 -- 172.21.15.64:0/2755224354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 msgr2=0x7f3b7010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/2755224354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7010df40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009900 tx=0x7f3b6002a410 comp rx=0 tx=0).stop 2024-09-19T05:15:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 -- 172.21.15.64:0/2755224354 shutdown_connections 2024-09-19T05:15:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/2755224354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 0x7f3b70110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/2755224354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7010df40 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/2755224354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b701008d0 0x7f3b70100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 -- 172.21.15.64:0/2755224354 >> 172.21.15.64:0/2755224354 conn(0x7f3b700fc7a0 msgr2=0x7f3b700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 -- 172.21.15.64:0/2755224354 shutdown_connections 2024-09-19T05:15:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.244+0000 7f3b77bc6640 1 -- 172.21.15.64:0/2755224354 wait complete. 2024-09-19T05:15:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 Processor -- start 2024-09-19T05:15:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 -- start start 2024-09-19T05:15:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b701008d0 0x7f3b7019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 0x7f3b7019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70114060 con 0x7f3b70101280 2024-09-19T05:15:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70113ee0 con 0x7f3b7010e480 2024-09-19T05:15:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b77bc6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701141e0 con 0x7f3b701008d0 2024-09-19T05:15:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b75161640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b75161640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47018/0 (socket says 172.21.15.64:47018) 2024-09-19T05:15:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b75161640 1 -- 172.21.15.64:0/456472659 learned_addr learned my addr 172.21.15.64:0/456472659 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b75962640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b701008d0 0x7f3b7019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.245+0000 7f3b76163640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 0x7f3b7019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b75161640 1 -- 172.21.15.64:0/456472659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b701008d0 msgr2=0x7f3b7019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b75161640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b701008d0 0x7f3b7019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b75161640 1 -- 172.21.15.64:0/456472659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 msgr2=0x7f3b7019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b75161640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 0x7f3b7019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b75161640 1 -- 172.21.15.64:0/456472659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7019f570 con 0x7f3b70101280 2024-09-19T05:15:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b76163640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 0x7f3b7019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b75161640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7019ab00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005940 tx=0x7f3b60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6003f030 con 0x7f3b70101280 2024-09-19T05:15:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7019f800 con 0x7f3b70101280 2024-09-19T05:15:43.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b60003aa0 con 0x7f3b70101280 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60035070 con 0x7f3b70101280 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.246+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701a7140 con 0x7f3b70101280 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.247+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b70100cb0 con 0x7f3b70101280 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.248+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f3b60042030 con 0x7f3b70101280 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.249+0000 7f3b66ffd640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b5c0a33b0 0x7f3b5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.249+0000 7f3b75962640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b5c0a33b0 0x7f3b5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.249+0000 7f3b75962640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b5c0a33b0 0x7f3b5c0a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c005fd0 tx=0x7f3b6c00b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:43.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.250+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f3b600ba520 con 0x7f3b70101280 2024-09-19T05:15:43.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.254+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3b600bcb80 con 0x7f3b70101280 2024-09-19T05:15:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.465+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b7019b920 con 0x7f3b70101280 2024-09-19T05:15:43.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.465+0000 7f3b66ffd640 1 -- 172.21.15.64:0/456472659 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f3b7019b920 con 0x7f3b70101280 2024-09-19T05:15:43.468 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:15:43.468 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:15:43.469 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:15:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.467+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b5c0a33b0 msgr2=0x7f3b5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.467+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b5c0a33b0 0x7f3b5c0a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c005fd0 tx=0x7f3b6c00b390 comp rx=0 tx=0).stop 2024-09-19T05:15:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.467+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 msgr2=0x7f3b7019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.467+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7019ab00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005940 tx=0x7f3b60004270 comp rx=0 tx=0).stop 2024-09-19T05:15:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 shutdown_connections 2024-09-19T05:15:43.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f3b5c0a33b0 0x7f3b5c0a5870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b7010e480 0x7f3b7019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b70101280 0x7f3b7019ab00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 --2- 172.21.15.64:0/456472659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b701008d0 0x7f3b7019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 >> 172.21.15.64:0/456472659 conn(0x7f3b700fc7a0 msgr2=0x7f3b700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:43.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 shutdown_connections 2024-09-19T05:15:43.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:43.468+0000 7f3b77bc6640 1 -- 172.21.15.64:0/456472659 wait complete. 2024-09-19T05:15:43.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:43 smithi174 ceph-mon[34488]: pgmap v303: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:43.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:43 smithi064 ceph-mon[30960]: pgmap v303: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:43.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:43 smithi103 ceph-mon[34446]: pgmap v303: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:44.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:15:44.399 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:44.519 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:44 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/456472659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:44.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:44 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/456472659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:44.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:44 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/456472659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:44.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 -- 172.21.15.64:0/2369981843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec100e20 msgr2=0x7f8fec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 --2- 172.21.15.64:0/2369981843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec100e20 0x7f8fec101200 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009900 tx=0x7f8fe802a480 comp rx=0 tx=0).stop 2024-09-19T05:15:44.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 -- 172.21.15.64:0/2369981843 shutdown_connections 2024-09-19T05:15:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 --2- 172.21.15.64:0/2369981843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 0x7f8fec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 --2- 172.21.15.64:0/2369981843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fec102570 0x7f8fec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 --2- 172.21.15.64:0/2369981843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec100e20 0x7f8fec101200 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 -- 172.21.15.64:0/2369981843 >> 172.21.15.64:0/2369981843 conn(0x7f8fec0fc7d0 msgr2=0x7f8fec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 -- 172.21.15.64:0/2369981843 shutdown_connections 2024-09-19T05:15:44.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.837+0000 7f8ff3586640 1 -- 172.21.15.64:0/2369981843 wait complete. 2024-09-19T05:15:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 Processor -- start 2024-09-19T05:15:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 -- start start 2024-09-19T05:15:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fec100e20 0x7f8fec197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 0x7f8fec1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 0x7f8fec19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec110e30 con 0x7f8fec102570 2024-09-19T05:15:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec110cb0 con 0x7f8fec102ef0 2024-09-19T05:15:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff3586640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec110fb0 con 0x7f8fec100e20 2024-09-19T05:15:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff0b21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 0x7f8fec1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff0b21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 0x7f8fec1977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47642/0 (socket says 172.21.15.64:47642) 2024-09-19T05:15:44.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.838+0000 7f8ff0b21640 1 -- 172.21.15.64:0/3184161414 learned_addr learned my addr 172.21.15.64:0/3184161414 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 0x7f8fec19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff1322640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fec100e20 0x7f8fec197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff0b21640 1 -- 172.21.15.64:0/3184161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fec100e20 msgr2=0x7f8fec197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fec100e20 0x7f8fec197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff0b21640 1 -- 172.21.15.64:0/3184161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 msgr2=0x7f8fec19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 0x7f8fec19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff0b21640 1 -- 172.21.15.64:0/3184161414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec19c240 con 0x7f8fec102570 2024-09-19T05:15:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 0x7f8fec19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 0x7f8fec1977d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc007b50 tx=0x7f8fdc00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8fe27fc640 1 -- 172.21.15.64:0/3184161414 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc007c40 con 0x7f8fec102570 2024-09-19T05:15:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec19c530 con 0x7f8fec102570 2024-09-19T05:15:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8fe27fc640 1 -- 172.21.15.64:0/3184161414 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fdc007de0 con 0x7f8fec102570 2024-09-19T05:15:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8fe27fc640 1 -- 172.21.15.64:0/3184161414 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00ad40 con 0x7f8fec102570 2024-09-19T05:15:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.839+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec069f70 con 0x7f8fec102570 2024-09-19T05:15:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8fe27fc640 1 -- 172.21.15.64:0/3184161414 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f8fdc00eb30 con 0x7f8fec102570 2024-09-19T05:15:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 Processor -- start 2024-09-19T05:15:44.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 -- start start 2024-09-19T05:15:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bfe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90bfea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90c0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f90b642f0 con 0x564f90bffb40 2024-09-19T05:15:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f90b0bcf0 con 0x564f90c09370 2024-09-19T05:15:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff595f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f90b602b0 con 0x564f90bfe5d0 2024-09-19T05:15:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1b23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bfe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1b23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bfe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47650/0 (socket says 172.21.15.64:47650) 2024-09-19T05:15:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1b23640 1 -- 172.21.15.64:0/3698361249 learned_addr learned my addr 172.21.15.64:0/3698361249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1322640 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90bfea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90c0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8fe27fc640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f8fd80a33b0 0x7f8fd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1b23640 1 -- 172.21.15.64:0/3698361249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 msgr2=0x564f90bfea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90bfea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.842+0000 7f8ff1b23640 1 -- 172.21.15.64:0/3698361249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 msgr2=0x564f90c0b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90c0b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8ff1b23640 1 -- 172.21.15.64:0/3698361249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f90af03a0 con 0x564f90bffb40 2024-09-19T05:15:44.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90c0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8fe27fc640 1 -- 172.21.15.64:0/3184161414 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f8fdc0d13d0 con 0x7f8fec102570 2024-09-19T05:15:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f8fd80a33b0 0x7f8fd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bfe090 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400f410 tx=0x7f8fe400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273e34beacf9661e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8fd77fe640 1 -- 172.21.15.64:0/3698361249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4021070 con 0x564f90bffb40 2024-09-19T05:15:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8fd77fe640 1 -- 172.21.15.64:0/3698361249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe401be30 con 0x564f90bffb40 2024-09-19T05:15:44.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.843+0000 7f8fd77fe640 1 -- 172.21.15.64:0/3698361249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4015070 con 0x564f90bffb40 2024-09-19T05:15:44.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 -- 172.21.15.64:0/3698361249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 msgr2=0x564f90bfe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f8fd80a33b0 0x7f8fd80a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fe401e000 tx=0x7f8fe4021000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:44.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bfe090 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400f410 tx=0x7f8fe400f8d0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 -- 172.21.15.64:0/3698361249 shutdown_connections 2024-09-19T05:15:44.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90c0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90bfea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 --2- 172.21.15.64:0/3698361249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bfe090 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 -- 172.21.15.64:0/3698361249 >> 172.21.15.64:0/3698361249 conn(0x564f90bc7310 msgr2=0x564f90bfac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:44.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 -- 172.21.15.64:0/3698361249 shutdown_connections 2024-09-19T05:15:44.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 -- 172.21.15.64:0/3698361249 wait complete. 2024-09-19T05:15:44.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.844+0000 7f8ff595f740 1 -- start start 2024-09-19T05:15:44.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 Processor -- start 2024-09-19T05:15:44.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 0x564f90c08d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90c09370 0x564f90c06640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f90c0c070 con 0x564f90bfe5d0 2024-09-19T05:15:44.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f90b1f060 con 0x564f90bffb40 2024-09-19T05:15:44.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f90c0c1f0 con 0x564f90c09370 2024-09-19T05:15:44.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff595f740 10 client.0 _ll_register_callbacks cb 0x564f908728c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff1322640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 0x564f90c08d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff1322640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 0x564f90c08d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47656/0 (socket says 172.21.15.64:47656) 2024-09-19T05:15:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff1322640 1 -- 172.21.15.64:0/4031802936 learned_addr learned my addr 172.21.15.64:0/4031802936 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90c09370 0x564f90c06640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.845+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff1322640 1 -- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90c09370 msgr2=0x564f90c06640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff1322640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90c09370 0x564f90c06640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff1322640 1 -- 172.21.15.64:0/4031802936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 msgr2=0x564f90c06030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff1322640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c06030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff1322640 1 -- 172.21.15.64:0/4031802936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f908d53c0 con 0x564f90bfe5d0 2024-09-19T05:15:44.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff1322640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 0x564f90c08d80 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009900 tx=0x7f8fe80317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85712a327b505e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:44.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:44.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/4031802936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8035040 con 0x564f90bfe5d0 2024-09-19T05:15:44.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/4031802936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f8fe8004240 con 0x564f90bfe5d0 2024-09-19T05:15:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe401be30 con 0x564f90bfe5d0 2024-09-19T05:15:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/4031802936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8fe80318b0 con 0x564f90bfe5d0 2024-09-19T05:15:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff595f740 10 client.17751 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.846+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f90c17b90 con 0x564f90bfe5d0 2024-09-19T05:15:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/4031802936 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe401be30 con 0x564f90bfe5d0 2024-09-19T05:15:44.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/4031802936 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8fe804c070 con 0x564f90bfe5d0 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8ff595f740 10 client.17751 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8ff595f740 10 client.17751 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8ff595f740 20 client.17751 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8ff595f740 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c18fa0 0x564f90c1b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8ff595f740 4 client.17751 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.847+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f90b71860 con 0x564f90c18fa0 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.848+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c18fa0 0x564f90c1b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.848+0000 7f8fd4ff9640 10 client.17751 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.848+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c18fa0 0x564f90c1b3d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6b5648dc2e6d5b7 server_cookie=56bfa16922fec2ee in_seq=0 out_seq=0 2024-09-19T05:15:44.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/4031802936 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f90b71860 con 0x564f90c18fa0 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8fd4ff9640 10 client.17751 handle_command_reply: tid=0 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 client.17751 shutdown 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 msgr2=0x564f90c08d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 0x564f90c08d80 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009900 tx=0x7f8fe80317c0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 shutdown_connections 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c18fa0 0x564f90c1b3d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90c09370 0x564f90c06640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 --2- 172.21.15.64:0/4031802936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bfe5d0 0x564f90c08d80 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 >> 172.21.15.64:0/4031802936 conn(0x564f90bc7310 msgr2=0x564f90c00960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:44.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8fd4ff9640 0 client.17751 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:44.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 shutdown_connections 2024-09-19T05:15:44.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 1 -- 172.21.15.64:0/4031802936 wait complete. 2024-09-19T05:15:44.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.849+0000 7f8ff595f740 20 client.17751 trim_cache size 0 max 16384 2024-09-19T05:15:44.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.862+0000 7f8ff595f740 1 Processor -- start 2024-09-19T05:15:44.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.862+0000 7f8ff595f740 1 -- start start 2024-09-19T05:15:44.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bf9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90bfa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff595f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec1adf00 con 0x564f90bffb40 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff595f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec1ab7d0 con 0x564f90c09370 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff595f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec1a9640 con 0x564f90bfe5d0 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bf9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bf9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47666/0 (socket says 172.21.15.64:47666) 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 -- 172.21.15.64:0/2515782218 learned_addr learned my addr 172.21.15.64:0/2515782218 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff1322640 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 -- 172.21.15.64:0/2515782218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 msgr2=0x564f90c04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90bfa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c04f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 -- 172.21.15.64:0/2515782218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 msgr2=0x564f90bfa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90bfa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 -- 172.21.15.64:0/2515782218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f90bfe2d0 con 0x564f90bffb40 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bf9bf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc099b70 tx=0x7f8fdc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b36cca768913c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8fd6ffd640 1 -- 172.21.15.64:0/2515782218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc09d4e0 con 0x564f90bffb40 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8fd6ffd640 1 -- 172.21.15.64:0/2515782218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fdc00a440 con 0x564f90bffb40 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.863+0000 7f8fd6ffd640 1 -- 172.21.15.64:0/2515782218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc09d070 con 0x564f90bffb40 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 -- 172.21.15.64:0/2515782218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 msgr2=0x564f90bf9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bf9bf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc099b70 tx=0x7f8fdc007f90 comp rx=0 tx=0).stop 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 -- 172.21.15.64:0/2515782218 shutdown_connections 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90c09370 0x564f90bfa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90bffb40 0x564f90bf9bf0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 --2- 172.21.15.64:0/2515782218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 -- 172.21.15.64:0/2515782218 >> 172.21.15.64:0/2515782218 conn(0x564f90bc7310 msgr2=0x564f90bf94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 -- 172.21.15.64:0/2515782218 shutdown_connections 2024-09-19T05:15:44.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 -- 172.21.15.64:0/2515782218 wait complete. 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 -- start start 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.864+0000 7f8ff595f740 1 Processor -- start 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c11bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 0x564f90c02660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec1a9c90 con 0x564f90c09370 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec1ab100 con 0x564f90bffb40 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec1aa360 con 0x564f90bfe5d0 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff595f740 10 client.0 _ll_register_callbacks cb 0x7f8fe8031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 0x564f90c02660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 0x564f90c02660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47674/0 (socket says 172.21.15.64:47674) 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 -- 172.21.15.64:0/319943267 learned_addr learned my addr 172.21.15.64:0/319943267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:44.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1322640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff0b21640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c11bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 -- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 msgr2=0x564f90c116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 -- 172.21.15.64:0/319943267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 msgr2=0x564f90c11bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c11bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.865+0000 7f8ff1b23640 1 -- 172.21.15.64:0/319943267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f90c02d00 con 0x564f90c09370 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff1b23640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 0x564f90c02660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400fdd0 tx=0x7f8fe4011500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae7dc458cbff916 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:44.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/319943267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4036060 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/319943267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f8fe4004be0 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f90b6bd70 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/319943267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8fe4015040 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff595f740 10 client.17757 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f90c835f0 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/319943267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564f90b6bd70 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/319943267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8fe4011ae0 con 0x564f90c09370 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff595f740 10 client.17757 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff595f740 10 client.17757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.866+0000 7f8ff595f740 20 client.17757 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:44.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c83a60 0x564f90c85e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 4 client.17757 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f90ae8170 con 0x564f90c83a60 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff1322640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c83a60 0x564f90c85e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff1322640 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c83a60 0x564f90c85e80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=904bcf688110da8 server_cookie=2a654c23173e99e7 in_seq=0 out_seq=0 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8fd4ff9640 10 client.17757 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8fd4ff9640 1 -- 172.21.15.64:0/319943267 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f90ae8170 con 0x564f90c83a60 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8fd4ff9640 10 client.17757 handle_command_reply: tid=0 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 client.17757 shutdown 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 msgr2=0x564f90c02660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 0x564f90c02660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400fdd0 tx=0x7f8fe4011500 comp rx=0 tx=0).stop 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 shutdown_connections 2024-09-19T05:15:44.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564f90c83a60 0x564f90c85e80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.867+0000 7f8ff595f740 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f90c09370 0x564f90c02660 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8fd4ff9640 0 client.17757 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff595f740 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f90bffb40 0x564f90c11bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff595f740 1 --2- 172.21.15.64:0/319943267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f90bfe5d0 0x564f90c116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 >> 172.21.15.64:0/319943267 conn(0x564f90bc7310 msgr2=0x564f90bf9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 shutdown_connections 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff595f740 1 -- 172.21.15.64:0/319943267 wait complete. 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff595f740 20 client.17757 trim_cache size 0 max 16384 2024-09-19T05:15:44.886 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f8fd80a33b0 msgr2=0x7f8fd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f8fd80a33b0 0x7f8fd80a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fe401e000 tx=0x7f8fe4021000 comp rx=0 tx=0).stop 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 msgr2=0x7f8fec1977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 0x7f8fec1977d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc007b50 tx=0x7f8fdc00e420 comp rx=0 tx=0).stop 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 shutdown_connections 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f8fd80a33b0 0x7f8fd80a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fec102ef0 0x7f8fec19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fec102570 0x7f8fec1977d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 --2- 172.21.15.64:0/3184161414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fec100e20 0x7f8fec197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 >> 172.21.15.64:0/3184161414 conn(0x7f8fec0fc7d0 msgr2=0x7f8fec0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 shutdown_connections 2024-09-19T05:15:44.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:44.868+0000 7f8ff3586640 1 -- 172.21.15.64:0/3184161414 wait complete. 2024-09-19T05:15:45.483 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:45 smithi064 ceph-mon[30960]: pgmap v304: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:45.512 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:15:45.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:45 smithi174 ceph-mon[34488]: pgmap v304: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:45.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:45 smithi103 ceph-mon[34446]: pgmap v304: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:45.768 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:46.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 -- 172.21.15.64:0/1476241073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 msgr2=0x7f9f2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 --2- 172.21.15.64:0/1476241073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2010ea90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009ca0 tx=0x7f9f1802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 -- 172.21.15.64:0/1476241073 shutdown_connections 2024-09-19T05:15:46.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 --2- 172.21.15.64:0/1476241073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2010ea90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 --2- 172.21.15.64:0/1476241073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20102790 0x7f9f201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 --2- 172.21.15.64:0/1476241073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f20101de0 0x7f9f201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 -- 172.21.15.64:0/1476241073 >> 172.21.15.64:0/1476241073 conn(0x7f9f200fd8c0 msgr2=0x7f9f200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 -- 172.21.15.64:0/1476241073 shutdown_connections 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.261+0000 7f9f2486d640 1 -- 172.21.15.64:0/1476241073 wait complete. 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.262+0000 7f9f2486d640 1 Processor -- start 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.262+0000 7f9f2486d640 1 -- start start 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f2486d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20101de0 0x7f9f201982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f2486d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f20102790 0x7f9f20198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f2486d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f2486d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20111e70 con 0x7f9f20106f10 2024-09-19T05:15:46.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f2486d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20111cf0 con 0x7f9f20101de0 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f2486d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20111ff0 con 0x7f9f20102790 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47700/0 (socket says 172.21.15.64:47700) 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 -- 172.21.15.64:0/194338522 learned_addr learned my addr 172.21.15.64:0/194338522 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f20102790 0x7f9f20198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20101de0 0x7f9f201982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 -- 172.21.15.64:0/194338522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f20102790 msgr2=0x7f9f20198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f20102790 0x7f9f20198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 -- 172.21.15.64:0/194338522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20101de0 msgr2=0x7f9f201982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20101de0 0x7f9f201982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.263+0000 7f9f1ed9d640 1 -- 172.21.15.64:0/194338522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2019d2a0 con 0x7f9f20106f10 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.264+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2019cbc0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005e30 tx=0x7f9f180093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.264+0000 7f9f17fff640 1 -- 172.21.15.64:0/194338522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18003c70 con 0x7f9f20106f10 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.264+0000 7f9f17fff640 1 -- 172.21.15.64:0/194338522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f18003e10 con 0x7f9f20106f10 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.264+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2019d530 con 0x7f9f20106f10 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.264+0000 7f9f17fff640 1 -- 172.21.15.64:0/194338522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18030c10 con 0x7f9f20106f10 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.264+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f201a4e70 con 0x7f9f20106f10 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.266+0000 7f9f17fff640 1 -- 172.21.15.64:0/194338522 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9f18004040 con 0x7f9f20106f10 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 Processor -- start 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 -- start start 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def15090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def15a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5600dee7b2f0 con 0x5600def155d0 2024-09-19T05:15:46.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600dee22cf0 con 0x5600def20370 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f26c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600dee772b0 con 0x5600def16b40 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def15a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def15a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47704/0 (socket says 172.21.15.64:47704) 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2822286911 learned_addr learned my addr 172.21.15.64:0/2822286911 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.267+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def15090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2822286911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 msgr2=0x5600def15090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def15090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2822286911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 msgr2=0x5600def22760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def22760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2822286911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600dee073a0 con 0x5600def155d0 2024-09-19T05:15:46.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f17fff640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9f0c0a33b0 0x7f9f0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9f0c0a33b0 0x7f9f0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def15a10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000bd10 tx=0x7f9f100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65137cb5b0749ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f17fff640 1 -- 172.21.15.64:0/194338522 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f9f180f0f90 con 0x7f9f20106f10 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f14d99640 1 -- 172.21.15.64:0/2822286911 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10004030 con 0x5600def155d0 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f14d99640 1 -- 172.21.15.64:0/2822286911 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f100035f0 con 0x5600def155d0 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.268+0000 7f9f14d99640 1 -- 172.21.15.64:0/2822286911 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100117b0 con 0x5600def155d0 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 -- 172.21.15.64:0/2822286911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 msgr2=0x5600def15a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9f0c0a33b0 0x7f9f0c0a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5600def16070 tx=0x7f9f080046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def15a10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000bd10 tx=0x7f9f100107a0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 -- 172.21.15.64:0/2822286911 shutdown_connections 2024-09-19T05:15:46.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def15a10 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2822286911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def15090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 -- 172.21.15.64:0/2822286911 >> 172.21.15.64:0/2822286911 conn(0x5600deede310 msgr2=0x5600def11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 -- 172.21.15.64:0/2822286911 shutdown_connections 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 -- 172.21.15.64:0/2822286911 wait complete. 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 -- start start 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.269+0000 7f9f26c46740 1 Processor -- start 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def155d0 0x5600def1fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 0x5600def1d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5600def23070 con 0x5600def16b40 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600dee36060 con 0x5600def155d0 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600def231f0 con 0x5600def20370 2024-09-19T05:15:46.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f26c46740 10 client.0 _ll_register_callbacks cb 0x5600deb898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 0x5600def1d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 0x5600def1d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47716/0 (socket says 172.21.15.64:47716) 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 -- 172.21.15.64:0/3057464418 learned_addr learned my addr 172.21.15.64:0/3057464418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def155d0 0x5600def1fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 -- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 msgr2=0x5600def1d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 -- 172.21.15.64:0/3057464418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def155d0 msgr2=0x5600def1fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def155d0 0x5600def1fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.270+0000 7f9f15d9b640 1 -- 172.21.15.64:0/3057464418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600debec3c0 con 0x5600def16b40 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 0x5600def1d040 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9f0801fae0 tx=0x7f9f08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dfb61600dba457 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:46.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f05ffb640 1 -- 172.21.15.64:0/3057464418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800fa50 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f05ffb640 1 -- 172.21.15.64:0/3057464418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f9f0800fbf0 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f100035f0 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f05ffb640 1 -- 172.21.15.64:0/3057464418 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f0800fdf0 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f26c46740 10 client.17769 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600def2ebd0 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f05ffb640 1 -- 172.21.15.64:0/3057464418 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f100035f0 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f05ffb640 1 -- 172.21.15.64:0/3057464418 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f08032070 con 0x5600def16b40 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f26c46740 10 client.17769 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f26c46740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.271+0000 7f9f26c46740 20 client.17769 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:46.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.272+0000 7f9f26c46740 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def300e0 0x5600def32530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.272+0000 7f9f26c46740 4 client.17769 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:46.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.272+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600dee88860 con 0x5600def300e0 2024-09-19T05:15:46.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.272+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def300e0 0x5600def32530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.272+0000 7f9f05ffb640 10 client.17769 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:46.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.272+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def300e0 0x5600def32530 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dee2b1886e86c1f server_cookie=8d82ad1d465e3e14 in_seq=0 out_seq=0 2024-09-19T05:15:46.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f05ffb640 1 -- 172.21.15.64:0/3057464418 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600dee88860 con 0x5600def300e0 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f05ffb640 10 client.17769 handle_command_reply: tid=0 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 client.17769 shutdown 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 msgr2=0x5600def1d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 0x5600def1d040 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9f0801fae0 tx=0x7f9f08004730 comp rx=0 tx=0).stop 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 shutdown_connections 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def300e0 0x5600def32530 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def16b40 0x5600def1d040 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 --2- 172.21.15.64:0/3057464418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def155d0 0x5600def1fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 >> 172.21.15.64:0/3057464418 conn(0x5600deede310 msgr2=0x5600def17930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:46.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f05ffb640 0 client.17769 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:46.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 shutdown_connections 2024-09-19T05:15:46.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 1 -- 172.21.15.64:0/3057464418 wait complete. 2024-09-19T05:15:46.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.273+0000 7f9f26c46740 20 client.17769 trim_cache size 0 max 16384 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.286+0000 7f9f26c46740 1 Processor -- start 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.286+0000 7f9f26c46740 1 -- start start 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def1efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def18120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def18660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f26c46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201b01a0 con 0x5600def155d0 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f26c46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201ada70 con 0x5600def20370 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f26c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201ab8e0 con 0x5600def16b40 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def1efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def1efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47730/0 (socket says 172.21.15.64:47730) 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 -- 172.21.15.64:0/11004721 learned_addr learned my addr 172.21.15.64:0/11004721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def18660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 -- 172.21.15.64:0/11004721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 msgr2=0x5600def18120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def18120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def18120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 -- 172.21.15.64:0/11004721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 msgr2=0x5600def18660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def18660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 -- 172.21.15.64:0/11004721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600def22a90 con 0x5600def155d0 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def18660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def18120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def1efb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x5600def12ac0 tx=0x7f9f1000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5155f2747e17edb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f07fff640 1 -- 172.21.15.64:0/11004721 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10005240 con 0x5600def155d0 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.287+0000 7f9f07fff640 1 -- 172.21.15.64:0/11004721 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f100053e0 con 0x5600def155d0 2024-09-19T05:15:46.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f07fff640 1 -- 172.21.15.64:0/11004721 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100116a0 con 0x5600def155d0 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 -- 172.21.15.64:0/11004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 msgr2=0x5600def1efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def1efb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x5600def12ac0 tx=0x7f9f1000bf90 comp rx=0 tx=0).stop 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 -- 172.21.15.64:0/11004721 shutdown_connections 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def20370 0x5600def18660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def16b40 0x5600def18120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 --2- 172.21.15.64:0/11004721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def1efb0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 -- 172.21.15.64:0/11004721 >> 172.21.15.64:0/11004721 conn(0x5600deede310 msgr2=0x5600def1e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 -- 172.21.15.64:0/11004721 shutdown_connections 2024-09-19T05:15:46.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 -- 172.21.15.64:0/11004721 wait complete. 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.288+0000 7f9f26c46740 1 -- start start 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 Processor -- start 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def16b40 0x5600def19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201abf30 con 0x5600def155d0 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201ad3a0 con 0x5600def16b40 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201ac600 con 0x5600def20370 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f26c46740 10 client.0 _ll_register_callbacks cb 0x7f9f08009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def28d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47738/0 (socket says 172.21.15.64:47738) 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2053903933 learned_addr learned my addr 172.21.15.64:0/2053903933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1ed9d640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def16b40 0x5600def19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 msgr2=0x5600def1a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2053903933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def16b40 msgr2=0x5600def19ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def16b40 0x5600def19ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 -- 172.21.15.64:0/2053903933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600def1ad30 con 0x5600def155d0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def16b40 0x5600def19ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f1e59c640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def28d40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000b5d0 tx=0x7f9f10005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9068c44cdabe84ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.289+0000 7f9f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f05ffb640 1 -- 172.21.15.64:0/2053903933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10034070 con 0x5600def155d0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f05ffb640 1 -- 172.21.15.64:0/2053903933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f9f100035f0 con 0x5600def155d0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f100053e0 con 0x5600def155d0 2024-09-19T05:15:46.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f05ffb640 1 -- 172.21.15.64:0/2053903933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f10003810 con 0x5600def155d0 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 10 client.17775 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600def9aeb0 con 0x5600def155d0 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f05ffb640 1 -- 172.21.15.64:0/2053903933 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f100053e0 con 0x5600def155d0 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f05ffb640 1 -- 172.21.15.64:0/2053903933 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f1001b420 con 0x5600def155d0 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 10 client.17775 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 10 client.17775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 20 client.17775 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def9b380 0x5600def9d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 4 client.17775 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.290+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600dee82d70 con 0x5600def9b380 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.291+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def9b380 0x5600def9d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.291+0000 7f9f05ffb640 10 client.17775 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.291+0000 7f9f15d9b640 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def9b380 0x5600def9d770 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d8dad93cac976d server_cookie=dd3f2849c5e2f6d3 in_seq=0 out_seq=0 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f05ffb640 1 -- 172.21.15.64:0/2053903933 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600dee82d70 con 0x5600def9b380 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f05ffb640 10 client.17775 handle_command_reply: tid=0 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 client.17775 shutdown 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 msgr2=0x5600def28d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def28d40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000b5d0 tx=0x7f9f10005c10 comp rx=0 tx=0).stop 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 shutdown_connections 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600def20370 0x5600def1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600def16b40 0x5600def19ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x5600def9b380 0x5600def9d770 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 --2- 172.21.15.64:0/2053903933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5600def155d0 0x5600def28d40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f05ffb640 0 client.17775 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 >> 172.21.15.64:0/2053903933 conn(0x5600deede310 msgr2=0x5600def1f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 shutdown_connections 2024-09-19T05:15:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 1 -- 172.21.15.64:0/2053903933 wait complete. 2024-09-19T05:15:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f26c46740 20 client.17775 trim_cache size 0 max 16384 2024-09-19T05:15:46.299 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:15:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9f0c0a33b0 msgr2=0x7f9f0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f2486d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9f0c0a33b0 0x7f9f0c0a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5600def16070 tx=0x7f9f080046c0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 msgr2=0x7f9f2019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f2486d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2019cbc0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005e30 tx=0x7f9f180093d0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.292+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 shutdown_connections 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9f0c0a33b0 0x7f9f0c0a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f20106f10 0x7f9f2019cbc0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f20102790 0x7f9f20198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 --2- 172.21.15.64:0/194338522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20101de0 0x7f9f201982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 >> 172.21.15.64:0/194338522 conn(0x7f9f200fd8c0 msgr2=0x7f9f200fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 shutdown_connections 2024-09-19T05:15:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:46.293+0000 7f9f2486d640 1 -- 172.21.15.64:0/194338522 wait complete. 2024-09-19T05:15:47.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:15:47.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:47 smithi064 ceph-mon[30960]: pgmap v305: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:47.419 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:47.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:47 smithi174 ceph-mon[34488]: pgmap v305: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:47 smithi103 ceph-mon[34446]: pgmap v305: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 -- 172.21.15.64:0/3992808723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 msgr2=0x7fa958106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 --2- 172.21.15.64:0/3992808723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa958106e60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa948009930 tx=0x7fa94802a420 comp rx=0 tx=0).stop 2024-09-19T05:15:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 -- 172.21.15.64:0/3992808723 shutdown_connections 2024-09-19T05:15:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 --2- 172.21.15.64:0/3992808723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa958101420 0x7fa95810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 --2- 172.21.15.64:0/3992808723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 0x7fa958100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 --2- 172.21.15.64:0/3992808723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa958106e60 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 -- 172.21.15.64:0/3992808723 >> 172.21.15.64:0/3992808723 conn(0x7fa9580fc7a0 msgr2=0x7fa9580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 -- 172.21.15.64:0/3992808723 shutdown_connections 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.921+0000 7fa95e774640 1 -- 172.21.15.64:0/3992808723 wait complete. 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 Processor -- start 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 -- start start 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 0x7fa9581991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa958101420 0x7fa958199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa9581a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa958110d30 con 0x7fa958106a80 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa958110bb0 con 0x7fa958100a80 2024-09-19T05:15:47.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.922+0000 7fa95e774640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa958110eb0 con 0x7fa958101420 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa9581a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa9581a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47752/0 (socket says 172.21.15.64:47752) 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 -- 172.21.15.64:0/631136067 learned_addr learned my addr 172.21.15.64:0/631136067 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa957fff640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 0x7fa9581991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa9577fe640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa958101420 0x7fa958199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 -- 172.21.15.64:0/631136067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa958101420 msgr2=0x7fa958199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa958101420 0x7fa958199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 -- 172.21.15.64:0/631136067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 msgr2=0x7fa9581991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 0x7fa9581991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 -- 172.21.15.64:0/631136067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9581a4780 con 0x7fa958106a80 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa957fff640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 0x7fa9581991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:47.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95cd11640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa9581a4240 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa95000eaa0 tx=0x7fa95000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa9557fa640 1 -- 172.21.15.64:0/631136067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000cdb0 con 0x7fa958106a80 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9581a4950 con 0x7fa958106a80 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa9557fa640 1 -- 172.21.15.64:0/631136067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa950005020 con 0x7fa958106a80 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa9557fa640 1 -- 172.21.15.64:0/631136067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500116d0 con 0x7fa958106a80 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.923+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581a4ee0 con 0x7fa958106a80 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.925+0000 7fa9557fa640 1 -- 172.21.15.64:0/631136067 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa9500051d0 con 0x7fa958106a80 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 Processor -- start 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 -- start start 2024-09-19T05:15:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb51090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb51a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb5e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56039cab72f0 con 0x56039cb515d0 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56039ca5ecf0 con 0x56039cb52b40 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa960b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56039cab32b0 con 0x56039cb5c370 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb51a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa957fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb51a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47766/0 (socket says 172.21.15.64:47766) 2024-09-19T05:15:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa957fff640 1 -- 172.21.15.64:0/1308253737 learned_addr learned my addr 172.21.15.64:0/1308253737 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa9557fa640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9400a33b0 0x7fa9400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.926+0000 7fa9577fe640 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb51090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa9577fe640 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb5e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa95cd11640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9400a33b0 0x7fa9400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa957fff640 1 -- 172.21.15.64:0/1308253737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 msgr2=0x56039cb5e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa957fff640 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb5e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa957fff640 1 -- 172.21.15.64:0/1308253737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 msgr2=0x56039cb51090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa957fff640 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb51090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa957fff640 1 -- 172.21.15.64:0/1308253737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039ca433a0 con 0x56039cb515d0 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa9557fa640 1 -- 172.21.15.64:0/631136067 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7fa9500cff70 con 0x7fa958106a80 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa957fff640 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb51a10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa948031440 tx=0x7fa948032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a832840a7a8a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa95cd11640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9400a33b0 0x7fa9400a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa95009bec0 tx=0x7fa950066fa0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa93e7fc640 1 -- 172.21.15.64:0/1308253737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94803f070 con 0x56039cb515d0 2024-09-19T05:15:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa93e7fc640 1 -- 172.21.15.64:0/1308253737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9480316a0 con 0x56039cb515d0 2024-09-19T05:15:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.927+0000 7fa93e7fc640 1 -- 172.21.15.64:0/1308253737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948035070 con 0x56039cb515d0 2024-09-19T05:15:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 -- 172.21.15.64:0/1308253737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 msgr2=0x56039cb51a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb51a10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa948031440 tx=0x7fa948032e70 comp rx=0 tx=0).stop 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 -- 172.21.15.64:0/1308253737 shutdown_connections 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb5e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb51a10 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 --2- 172.21.15.64:0/1308253737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb51090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 -- 172.21.15.64:0/1308253737 >> 172.21.15.64:0/1308253737 conn(0x56039cb1a310 msgr2=0x56039cb4dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 -- 172.21.15.64:0/1308253737 shutdown_connections 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.928+0000 7fa960b4d740 1 -- 172.21.15.64:0/1308253737 wait complete. 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 -- start start 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 Processor -- start 2024-09-19T05:15:47.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb5bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb58fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb59560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56039cb5f010 con 0x56039cb5c370 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56039ca72060 con 0x56039cb52b40 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56039cb5f190 con 0x56039cb515d0 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa960b4d740 10 client.0 _ll_register_callbacks cb 0x56039c7c58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb59560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb59560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47782/0 (socket says 172.21.15.64:47782) 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa9577fe640 1 -- 172.21.15.64:0/4182290170 learned_addr learned my addr 172.21.15.64:0/4182290170 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:47.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.929+0000 7fa957fff640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb58fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb5bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 -- 172.21.15.64:0/4182290170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 msgr2=0x56039cb5bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb5bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 -- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 msgr2=0x56039cb58fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb58fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 -- 172.21.15.64:0/4182290170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039c8283c0 con 0x56039cb5c370 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9577fe640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb59560 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000b60 tx=0x7fa94c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0f37a1d1e06466 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9377fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9377fe640 1 -- 172.21.15.64:0/4182290170 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0130d0 con 0x56039cb5c370 2024-09-19T05:15:47.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9377fe640 1 -- 172.21.15.64:0/4182290170 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7fa94c004020 con 0x56039cb5c370 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9480316a0 con 0x56039cb5c370 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9377fe640 1 -- 172.21.15.64:0/4182290170 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa94c013270 con 0x56039cb5c370 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa960b4d740 10 client.17787 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56039cb6ab40 con 0x56039cb5c370 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.930+0000 7fa9377fe640 1 -- 172.21.15.64:0/4182290170 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9480316a0 con 0x56039cb5c370 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa9377fe640 1 -- 172.21.15.64:0/4182290170 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa94c011440 con 0x56039cb5c370 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa960b4d740 10 client.17787 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa960b4d740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:47.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa960b4d740 20 client.17787 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa960b4d740 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cb6c000 0x56039cb6e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa960b4d740 4 client.17787 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56039cac4860 con 0x56039cb6c000 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.931+0000 7fa957fff640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cb6c000 0x56039cb6e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa9377fe640 10 client.17787 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa957fff640 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cb6c000 0x56039cb6e3f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0289ddcd488ccd5 server_cookie=4a939819958056f7 in_seq=0 out_seq=0 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa9377fe640 1 -- 172.21.15.64:0/4182290170 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56039cac4860 con 0x56039cb6c000 2024-09-19T05:15:47.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa9377fe640 10 client.17787 handle_command_reply: tid=0 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa960b4d740 1 client.17787 shutdown 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 msgr2=0x56039cb59560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.932+0000 7fa960b4d740 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb59560 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000b60 tx=0x7fa94c015040 comp rx=0 tx=0).stop 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 shutdown_connections 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cb6c000 0x56039cb6e3f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb59560 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb58fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 --2- 172.21.15.64:0/4182290170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb5bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 >> 172.21.15.64:0/4182290170 conn(0x56039cb1a310 msgr2=0x56039cb53930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa9377fe640 0 client.17787 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:47.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 shutdown_connections 2024-09-19T05:15:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 1 -- 172.21.15.64:0/4182290170 wait complete. 2024-09-19T05:15:47.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.933+0000 7fa960b4d740 20 client.17787 trim_cache size 0 max 16384 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 Processor -- start 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 -- start start 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb4d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb4dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581b01b0 con 0x56039cb515d0 2024-09-19T05:15:47.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581ada80 con 0x56039cb52b40 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa960b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581ab8f0 con 0x56039cb5c370 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47792/0 (socket says 172.21.15.64:47792) 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 -- 172.21.15.64:0/864782594 learned_addr learned my addr 172.21.15.64:0/864782594 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa957fff640 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb4d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 -- 172.21.15.64:0/864782594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 msgr2=0x56039cb4dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb4dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 -- 172.21.15.64:0/864782594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 msgr2=0x56039cb4d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb4d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 -- 172.21.15.64:0/864782594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039cb4e180 con 0x56039cb515d0 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.946+0000 7fa9577fe640 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb682e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005e60 tx=0x7fa94c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae2fc80ecd35d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa93dffb640 1 -- 172.21.15.64:0/864782594 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c030070 con 0x56039cb515d0 2024-09-19T05:15:47.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa93dffb640 1 -- 172.21.15.64:0/864782594 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa94c011440 con 0x56039cb515d0 2024-09-19T05:15:47.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa93dffb640 1 -- 172.21.15.64:0/864782594 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c028070 con 0x56039cb515d0 2024-09-19T05:15:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 -- 172.21.15.64:0/864782594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 msgr2=0x56039cb682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb682e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005e60 tx=0x7fa94c004680 comp rx=0 tx=0).stop 2024-09-19T05:15:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 -- 172.21.15.64:0/864782594 shutdown_connections 2024-09-19T05:15:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb5c370 0x56039cb4dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb4d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 --2- 172.21.15.64:0/864782594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb515d0 0x56039cb682e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 -- 172.21.15.64:0/864782594 >> 172.21.15.64:0/864782594 conn(0x56039cb1a310 msgr2=0x56039cb58f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 -- 172.21.15.64:0/864782594 shutdown_connections 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.947+0000 7fa960b4d740 1 -- 172.21.15.64:0/864782594 wait complete. 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 -- start start 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 Processor -- start 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb64870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581abf40 con 0x56039cb5c370 2024-09-19T05:15:47.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581ad3b0 con 0x56039cb52b40 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9581ac610 con 0x56039cb515d0 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa960b4d740 10 client.0 _ll_register_callbacks cb 0x7fa94c00ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb55ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47796/0 (socket says 172.21.15.64:47796) 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa9577fe640 1 -- 172.21.15.64:0/550546232 learned_addr learned my addr 172.21.15.64:0/550546232 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa957fff640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.948+0000 7fa9577fe640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb64870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9577fe640 1 -- 172.21.15.64:0/550546232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 msgr2=0x56039cb64870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9577fe640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb64870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9577fe640 1 -- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 msgr2=0x56039cb55960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9577fe640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb55960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9577fe640 1 -- 172.21.15.64:0/550546232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039cb563e0 con 0x56039cb5c370 2024-09-19T05:15:47.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa957fff640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb55960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9577fe640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb55ea0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00ecc0 tx=0x7fa94c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f3f2df01cf1b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9377fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9377fe640 1 -- 172.21.15.64:0/550546232 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c028070 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9377fe640 1 -- 172.21.15.64:0/550546232 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7fa94c004b00 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94c011440 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9377fe640 1 -- 172.21.15.64:0/550546232 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa94c030070 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa960b4d740 10 client.17793 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56039cbdacf0 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.949+0000 7fa9377fe640 1 -- 172.21.15.64:0/550546232 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa94c011440 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa9377fe640 1 -- 172.21.15.64:0/550546232 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa94c0398d0 con 0x56039cb5c370 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa960b4d740 10 client.17793 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:47.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa960b4d740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa960b4d740 20 client.17793 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa960b4d740 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cbdb250 0x56039cbdd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa960b4d740 4 client.17793 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56039cabed70 con 0x56039cbdb250 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.950+0000 7fa957fff640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cbdb250 0x56039cbdd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa957fff640 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cbdb250 0x56039cbdd640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd898c08ac6de536 server_cookie=65509f198ab9ba0a in_seq=0 out_seq=0 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa9377fe640 10 client.17793 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa9377fe640 1 -- 172.21.15.64:0/550546232 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56039cabed70 con 0x56039cbdb250 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa9377fe640 10 client.17793 handle_command_reply: tid=0 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 client.17793 shutdown 2024-09-19T05:15:47.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 msgr2=0x56039cb55ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb55ea0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00ecc0 tx=0x7fa94c015040 comp rx=0 tx=0).stop 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 shutdown_connections 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56039cbdb250 0x56039cbdd640 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56039cb5c370 0x56039cb55ea0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56039cb52b40 0x56039cb55960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 --2- 172.21.15.64:0/550546232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56039cb515d0 0x56039cb64870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa9377fe640 0 client.17793 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 >> 172.21.15.64:0/550546232 conn(0x56039cb1a310 msgr2=0x56039cb58f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 shutdown_connections 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 1 -- 172.21.15.64:0/550546232 wait complete. 2024-09-19T05:15:47.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.951+0000 7fa960b4d740 20 client.17793 trim_cache size 0 max 16384 2024-09-19T05:15:47.959 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:15:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9400a33b0 msgr2=0x7fa9400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9400a33b0 0x7fa9400a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa95009bec0 tx=0x7fa950066fa0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 msgr2=0x7fa9581a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:47.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa9581a4240 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa95000eaa0 tx=0x7fa95000ef60 comp rx=0 tx=0).stop 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 shutdown_connections 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa9400a33b0 0x7fa9400a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958106a80 0x7fa9581a4240 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa958101420 0x7fa958199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 --2- 172.21.15.64:0/631136067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa958100a80 0x7fa9581991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 >> 172.21.15.64:0/631136067 conn(0x7fa9580fc7a0 msgr2=0x7fa9580fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 shutdown_connections 2024-09-19T05:15:47.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:47.952+0000 7fa95e774640 1 -- 172.21.15.64:0/631136067 wait complete. 2024-09-19T05:15:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:49 smithi174 ceph-mon[34488]: pgmap v306: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:49.554 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:15:49.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:15:49.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:49 smithi064 ceph-mon[30960]: pgmap v306: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:49.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:49 smithi103 ceph-mon[34446]: pgmap v306: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 -- 172.21.15.64:0/2945283610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 msgr2=0x7f1a141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 --2- 172.21.15.64:0/2945283610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a141021c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009900 tx=0x7f1a0402a410 comp rx=0 tx=0).stop 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 -- 172.21.15.64:0/2945283610 shutdown_connections 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 --2- 172.21.15.64:0/2945283610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a14106f10 0x7f1a1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 --2- 172.21.15.64:0/2945283610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a14102790 0x7f1a141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 --2- 172.21.15.64:0/2945283610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a141021c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.303+0000 7f1a194db640 1 -- 172.21.15.64:0/2945283610 >> 172.21.15.64:0/2945283610 conn(0x7f1a140fd8c0 msgr2=0x7f1a140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.304+0000 7f1a194db640 1 -- 172.21.15.64:0/2945283610 shutdown_connections 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.304+0000 7f1a194db640 1 -- 172.21.15.64:0/2945283610 wait complete. 2024-09-19T05:15:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.304+0000 7f1a194db640 1 Processor -- start 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.304+0000 7f1a194db640 1 -- start start 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a194db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a14198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a194db640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a14102790 0x7f1a141988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a194db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a14106f10 0x7f1a1419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a194db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111e20 con 0x7f1a14101de0 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a194db640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111ca0 con 0x7f1a14102790 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a194db640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111fa0 con 0x7f1a14106f10 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a14198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a14198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47804/0 (socket says 172.21.15.64:47804) 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 -- 172.21.15.64:0/1496235851 learned_addr learned my addr 172.21.15.64:0/1496235851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a127fc640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a14102790 0x7f1a141988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a137fe640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a14106f10 0x7f1a1419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 -- 172.21.15.64:0/1496235851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a14106f10 msgr2=0x7f1a1419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a14106f10 0x7f1a1419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 -- 172.21.15.64:0/1496235851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a14102790 msgr2=0x7f1a141988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a14102790 0x7f1a141988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 -- 172.21.15.64:0/1496235851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1419d330 con 0x7f1a14101de0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.305+0000 7f1a12ffd640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a14198380 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005940 tx=0x7f1a04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.306+0000 7f19fbfff640 1 -- 172.21.15.64:0/1496235851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0403f030 con 0x7f1a14101de0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.306+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1419d5c0 con 0x7f1a14101de0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.306+0000 7f19fbfff640 1 -- 172.21.15.64:0/1496235851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a04003aa0 con 0x7f1a14101de0 2024-09-19T05:15:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.306+0000 7f19fbfff640 1 -- 172.21.15.64:0/1496235851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04035070 con 0x7f1a14101de0 2024-09-19T05:15:50.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.306+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a141a4f00 con 0x7f1a14101de0 2024-09-19T05:15:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f19fbfff640 1 -- 172.21.15.64:0/1496235851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1a04042030 con 0x7f1a14101de0 2024-09-19T05:15:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 Processor -- start 2024-09-19T05:15:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 -- start start 2024-09-19T05:15:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562143527280 con 0x5621435c6890 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621434ceb10 con 0x5621435d1150 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562143523240 con 0x5621435c5f70 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47806/0 (socket says 172.21.15.64:47806) 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a12ffd640 1 -- 172.21.15.64:0/1348058730 learned_addr learned my addr 172.21.15.64:0/1348058730 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.309+0000 7f1a127fc640 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a137fe640 1 -- 172.21.15.64:0/1348058730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 msgr2=0x5621435d3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a137fe640 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435d3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a137fe640 1 -- 172.21.15.64:0/1348058730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 msgr2=0x5621435d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a137fe640 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a137fe640 1 -- 172.21.15.64:0/1348058730 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621434db1f0 con 0x5621435c5f70 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f19fbfff640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f19fc0a3430 0x7f19fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a127fc640 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435d3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a12ffd640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f19fc0a3430 0x7f19fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f1a137fe640 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435c6350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007d10 tx=0x7f1a0c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ef61454bf96830 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f19fbfff640 1 -- 172.21.15.64:0/1496235851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a040eff40 con 0x7f1a14101de0 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f19f97fa640 1 -- 172.21.15.64:0/1348058730 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c014070 con 0x5621435c5f70 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f19f97fa640 1 -- 172.21.15.64:0/1348058730 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c010640 con 0x5621435c5f70 2024-09-19T05:15:50.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.310+0000 7f19f97fa640 1 -- 172.21.15.64:0/1348058730 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c005280 con 0x5621435c5f70 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/1348058730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 msgr2=0x5621435c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435c6350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007d10 tx=0x7f1a0c00e720 comp rx=0 tx=0).stop 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a12ffd640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f19fc0a3430 0x7f19fc0a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1a040c4930 tx=0x7f1a040f3ca0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/1348058730 shutdown_connections 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435d3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/1348058730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435c6350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/1348058730 >> 172.21.15.64:0/1348058730 conn(0x5621435c2500 msgr2=0x5621435c28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:50.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/1348058730 shutdown_connections 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/1348058730 wait complete. 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 -- start start 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.311+0000 7f1a1b8b4740 1 Processor -- start 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c6890 0x5621435ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 0x5621435ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5621435d3d20 con 0x5621435d1150 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621434e1f10 con 0x5621435c6890 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621435d3ea0 con 0x5621435c5f70 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a1b8b4740 10 client.0 _ll_register_callbacks cb 0x5621434aaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 0x5621435ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 0x5621435ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47820/0 (socket says 172.21.15.64:47820) 2024-09-19T05:15:50.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 -- 172.21.15.64:0/3715662213 learned_addr learned my addr 172.21.15.64:0/3715662213 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a127fc640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c6890 0x5621435ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 -- 172.21.15.64:0/3715662213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 msgr2=0x5621435ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 -- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c6890 msgr2=0x5621435ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c6890 0x5621435ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 -- 172.21.15.64:0/3715662213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621432986a0 con 0x5621435d1150 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f1a137fe640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 0x5621435ceb10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c004f80 tx=0x7f1a0c007380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febf99512edadd81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.312+0000 7f19f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f19f6ff5640 1 -- 172.21.15.64:0/3715662213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c014070 con 0x5621435d1150 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f19f6ff5640 1 -- 172.21.15.64:0/3715662213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a0c005910 con 0x5621435d1150 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0c010640 con 0x5621435d1150 2024-09-19T05:15:50.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f19f6ff5640 1 -- 172.21.15.64:0/3715662213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a0c021040 con 0x5621435d1150 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621435df980 con 0x5621435d1150 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f19f6ff5640 1 -- 172.21.15.64:0/3715662213 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c010640 con 0x5621435d1150 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f19f6ff5640 1 -- 172.21.15.64:0/3715662213 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a0c028660 con 0x5621435d1150 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 20 client.17805 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621435e0d90 0x5621435e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 4 client.17805 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.313+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621435347f0 con 0x5621435e0d90 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.314+0000 7f1a127fc640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621435e0d90 0x5621435e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.314+0000 7f1a127fc640 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621435e0d90 0x5621435e3180 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb5980bcd4e85cf server_cookie=ef03fed522926fe0 in_seq=0 out_seq=0 2024-09-19T05:15:50.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.314+0000 7f19f6ff5640 10 client.17805 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f19f6ff5640 1 -- 172.21.15.64:0/3715662213 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621435347f0 con 0x5621435e0d90 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f19f6ff5640 10 client.17805 handle_command_reply: tid=0 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 client.17805 shutdown 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 msgr2=0x5621435ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 0x5621435ceb10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c004f80 tx=0x7f1a0c007380 comp rx=0 tx=0).stop 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 shutdown_connections 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5621435e0d90 0x5621435e3180 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435d1150 0x5621435ceb10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c6890 0x5621435ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/3715662213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c5f70 0x5621435ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 >> 172.21.15.64:0/3715662213 conn(0x5621435c2500 msgr2=0x5621435bd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f19f6ff5640 0 client.17805 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 shutdown_connections 2024-09-19T05:15:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/3715662213 wait complete. 2024-09-19T05:15:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.315+0000 7f1a1b8b4740 20 client.17805 trim_cache size 0 max 16384 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 Processor -- start 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 -- start start 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 0x5621435c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c6890 0x5621435bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141adfc0 con 0x5621435c5f70 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141ab890 con 0x5621435d1150 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.332+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141a9700 con 0x5621435c6890 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 0x5621435c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 0x5621435c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47828/0 (socket says 172.21.15.64:47828) 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 -- 172.21.15.64:0/2384919179 learned_addr learned my addr 172.21.15.64:0/2384919179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a127fc640 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c6890 0x5621435bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 -- 172.21.15.64:0/2384919179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c6890 msgr2=0x5621435bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c6890 0x5621435bd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 -- 172.21.15.64:0/2384919179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 msgr2=0x5621435e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 -- 172.21.15.64:0/2384919179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621435c66c0 con 0x5621435c5f70 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f1a137fe640 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 0x5621435c89d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c028ab0 tx=0x7f1a0c0074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5041c5025ad49325 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f19f8ff9640 1 -- 172.21.15.64:0/2384919179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c014070 con 0x5621435c5f70 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f19f8ff9640 1 -- 172.21.15.64:0/2384919179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c0077c0 con 0x5621435c5f70 2024-09-19T05:15:50.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.333+0000 7f19f8ff9640 1 -- 172.21.15.64:0/2384919179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c034070 con 0x5621435c5f70 2024-09-19T05:15:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/2384919179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 msgr2=0x5621435c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 0x5621435c89d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c028ab0 tx=0x7f1a0c0074e0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/2384919179 shutdown_connections 2024-09-19T05:15:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435d1150 0x5621435e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435c6890 0x5621435bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/2384919179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c5f70 0x5621435c89d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/2384919179 >> 172.21.15.64:0/2384919179 conn(0x5621435c2500 msgr2=0x5621435bca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/2384919179 shutdown_connections 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/2384919179 wait complete. 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 -- start start 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.334+0000 7f1a1b8b4740 1 Processor -- start 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c5f70 0x5621435d97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435d1150 0x5621435d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141a9d50 con 0x5621435c6890 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141ab1c0 con 0x5621435c5f70 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141aa420 con 0x5621435d1150 2024-09-19T05:15:50.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 10 client.0 _ll_register_callbacks cb 0x7f1a040f3b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47834/0 (socket says 172.21.15.64:47834) 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 -- 172.21.15.64:0/838792428 learned_addr learned my addr 172.21.15.64:0/838792428 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a137fe640 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c5f70 0x5621435d97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 -- 172.21.15.64:0/838792428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435d1150 msgr2=0x5621435d5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435d1150 0x5621435d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 -- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c5f70 msgr2=0x5621435d97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c5f70 0x5621435d97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 -- 172.21.15.64:0/838792428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621435d5860 con 0x5621435c6890 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a127fc640 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d4af0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x5621435cd920 tx=0x7f1a0800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120ea51ab6bb2de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f19f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f19f6ff5640 1 -- 172.21.15.64:0/838792428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08005020 con 0x5621435c6890 2024-09-19T05:15:50.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f19f6ff5640 1 -- 172.21.15.64:0/838792428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a080051c0 con 0x5621435c6890 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.335+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0c0077c0 con 0x5621435c6890 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f19f6ff5640 1 -- 172.21.15.64:0/838792428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a080053c0 con 0x5621435c6890 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 10 client.17811 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56214364b8f0 con 0x5621435c6890 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f19f6ff5640 1 -- 172.21.15.64:0/838792428 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c0077c0 con 0x5621435c6890 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f19f6ff5640 1 -- 172.21.15.64:0/838792428 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a080264d0 con 0x5621435c6890 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 10 client.17811 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 20 client.17811 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56214364c0f0 0x56214364e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 4 client.17811 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56214352ed00 con 0x56214364c0f0 2024-09-19T05:15:50.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.336+0000 7f1a137fe640 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56214364c0f0 0x56214364e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.337+0000 7f1a137fe640 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56214364c0f0 0x56214364e510 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa3cc2524d2adeb server_cookie=c5f120bb9b7682a in_seq=0 out_seq=0 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.337+0000 7f19f6ff5640 10 client.17811 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f19f6ff5640 1 -- 172.21.15.64:0/838792428 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56214352ed00 con 0x56214364c0f0 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f19f6ff5640 10 client.17811 handle_command_reply: tid=0 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 client.17811 shutdown 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 msgr2=0x5621435d4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d4af0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x5621435cd920 tx=0x7f1a0800bcc0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 shutdown_connections 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621435d1150 0x5621435d5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x56214364c0f0 0x56214364e510 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621435c6890 0x5621435d4af0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 --2- 172.21.15.64:0/838792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621435c5f70 0x5621435d97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 >> 172.21.15.64:0/838792428 conn(0x5621435c2500 msgr2=0x5621435bd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f19f6ff5640 0 client.17811 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 shutdown_connections 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 1 -- 172.21.15.64:0/838792428 wait complete. 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.338+0000 7f1a1b8b4740 20 client.17811 trim_cache size 0 max 16384 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "c5e6956e-7a25-4117-b87e-aa754921129e", 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-19T05:15:50.345 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:15:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f19fc0a3430 msgr2=0x7f19fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f19fc0a3430 0x7f19fc0a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1a040c4930 tx=0x7f1a040f3ca0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 msgr2=0x7f1a14198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a14198380 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005940 tx=0x7f1a04004270 comp rx=0 tx=0).stop 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 shutdown_connections 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f19fc0a3430 0x7f19fc0a58f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a14106f10 0x7f1a1419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a14102790 0x7f1a141988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 --2- 172.21.15.64:0/1496235851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a14101de0 0x7f1a14198380 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 >> 172.21.15.64:0/1496235851 conn(0x7f1a140fd8c0 msgr2=0x7f1a140fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 shutdown_connections 2024-09-19T05:15:50.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:50.339+0000 7f1a194db640 1 -- 172.21.15.64:0/1496235851 wait complete. 2024-09-19T05:15:51.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:15:51.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:51 smithi064 ceph-mon[30960]: pgmap v307: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:51.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:51 smithi064 ceph-mon[30960]: scrub queued for path: / 2024-09-19T05:15:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:51.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:51 smithi174 ceph-mon[34488]: pgmap v307: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:51 smithi174 ceph-mon[34488]: scrub queued for path: / 2024-09-19T05:15:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:51 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:51 smithi174 ceph-mon[34488]: scrub summary: active paths [/] 2024-09-19T05:15:51.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:51 smithi174 ceph-mon[34488]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:51.619 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:51 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:51.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:51 smithi064 ceph-mon[30960]: scrub summary: active paths [/] 2024-09-19T05:15:51.620 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:51 smithi064 ceph-mon[30960]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:51 smithi103 ceph-mon[34446]: pgmap v307: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:15:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:51 smithi103 ceph-mon[34446]: scrub queued for path: / 2024-09-19T05:15:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:51 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:51 smithi103 ceph-mon[34446]: scrub summary: active paths [/] 2024-09-19T05:15:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:51 smithi103 ceph-mon[34446]: scrub summary: idle+waiting paths [/] 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 -- 172.21.15.64:0/3177800200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 msgr2=0x7f755410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 --2- 172.21.15.64:0/3177800200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f755410bb50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7550009900 tx=0x7f755002a480 comp rx=0 tx=0).stop 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 -- 172.21.15.64:0/3177800200 shutdown_connections 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 --2- 172.21.15.64:0/3177800200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f755410c170 0x7f755410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 --2- 172.21.15.64:0/3177800200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f755410bb50 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 --2- 172.21.15.64:0/3177800200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7554102d40 0x7f7554103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 -- 172.21.15.64:0/3177800200 >> 172.21.15.64:0/3177800200 conn(0x7f75540fc7a0 msgr2=0x7f75540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 -- 172.21.15.64:0/3177800200 shutdown_connections 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.847+0000 7f755beec640 1 -- 172.21.15.64:0/3177800200 wait complete. 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 Processor -- start 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 -- start start 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7554102d40 0x7f75541983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f75541988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755410c170 0x7f755419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7554111e20 con 0x7f7554103660 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7554111ca0 con 0x7f7554102d40 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755beec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7554111fa0 con 0x7f755410c170 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f75541988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f75541988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47854/0 (socket says 172.21.15.64:47854) 2024-09-19T05:15:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 -- 172.21.15.64:0/1461969259 learned_addr learned my addr 172.21.15.64:0/1461969259 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f755a489640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755410c170 0x7f755419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559c88640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7554102d40 0x7f75541983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 -- 172.21.15.64:0/1461969259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755410c170 msgr2=0x7f755419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755410c170 0x7f755419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 -- 172.21.15.64:0/1461969259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7554102d40 msgr2=0x7f75541983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7554102d40 0x7f75541983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559487640 1 -- 172.21.15.64:0/1461969259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755419d350 con 0x7f7554103660 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.848+0000 7f7559c88640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7554102d40 0x7f75541983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.849+0000 7f7559487640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f75541988e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7550005940 tx=0x7f75500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.849+0000 7f7546ffd640 1 -- 172.21.15.64:0/1461969259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755003f030 con 0x7f7554103660 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.849+0000 7f7546ffd640 1 -- 172.21.15.64:0/1461969259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7550003cb0 con 0x7f7554103660 2024-09-19T05:15:52.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.849+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755419d5e0 con 0x7f7554103660 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.849+0000 7f7546ffd640 1 -- 172.21.15.64:0/1461969259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550035070 con 0x7f7554103660 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.849+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a4f20 con 0x7f7554103660 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.851+0000 7f7546ffd640 1 -- 172.21.15.64:0/1461969259 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7550042030 con 0x7f7554103660 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.851+0000 7f755e2c5740 1 Processor -- start 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 -- start start 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a873544a0 0x563a87354880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87361a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a872ba2e0 con 0x563a87354dc0 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563a87261ce0 con 0x563a873544a0 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755e2c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a872b62a0 con 0x563a8735f680 2024-09-19T05:15:52.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47870/0 (socket says 172.21.15.64:47870) 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 -- 172.21.15.64:0/379418207 learned_addr learned my addr 172.21.15.64:0/379418207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755a489640 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a873544a0 0x563a87354880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7546ffd640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f75400a33b0 0x7f75400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559487640 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87361a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 -- 172.21.15.64:0/379418207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 msgr2=0x563a87361a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87361a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f755a489640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f75400a33b0 0x7f75400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 -- 172.21.15.64:0/379418207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a873544a0 msgr2=0x563a87354880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a873544a0 0x563a87354880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559c88640 1 -- 172.21.15.64:0/379418207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a87246440 con 0x563a87354dc0 2024-09-19T05:15:52.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7559487640 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87361a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.852+0000 7f7546ffd640 1 -- 172.21.15.64:0/1461969259 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f75500f0680 con 0x7f7554103660 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.853+0000 7f7559c88640 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735f140 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7548009f20 tx=0x7f754800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b406193b29372c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.853+0000 7f753ffff640 1 -- 172.21.15.64:0/379418207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548004d20 con 0x563a87354dc0 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.853+0000 7f753ffff640 1 -- 172.21.15.64:0/379418207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75480054d0 con 0x563a87354dc0 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.853+0000 7f753ffff640 1 -- 172.21.15.64:0/379418207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548011890 con 0x563a87354dc0 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.853+0000 7f755a489640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f75400a33b0 0x7f75400a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f754c005fd0 tx=0x7f754c004730 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 -- 172.21.15.64:0/379418207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 msgr2=0x563a8735f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735f140 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7548009f20 tx=0x7f754800f7f0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 -- 172.21.15.64:0/379418207 shutdown_connections 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87361a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735f140 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 --2- 172.21.15.64:0/379418207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a873544a0 0x563a87354880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 -- 172.21.15.64:0/379418207 >> 172.21.15.64:0/379418207 conn(0x563a87350a30 msgr2=0x563a87350e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 -- 172.21.15.64:0/379418207 shutdown_connections 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 -- 172.21.15.64:0/379418207 wait complete. 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 -- start start 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.854+0000 7f755e2c5740 1 Processor -- start 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a8735ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87354dc0 0x563a8735c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8735c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a87362250 con 0x563a873544a0 2024-09-19T05:15:52.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563a87275050 con 0x563a8735f680 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a873623d0 con 0x563a87354dc0 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 10 client.0 _ll_register_callbacks cb 0x563a872b5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a8735ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a8735ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47884/0 (socket says 172.21.15.64:47884) 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 -- 172.21.15.64:0/1554837482 learned_addr learned my addr 172.21.15.64:0/1554837482 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559487640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87354dc0 0x563a8735c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755a489640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8735c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 -- 172.21.15.64:0/1554837482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87354dc0 msgr2=0x563a8735c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87354dc0 0x563a8735c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 -- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 msgr2=0x563a8735c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8735c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 -- 172.21.15.64:0/1554837482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8702b3c0 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f7559c88640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a8735ef90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7548006010 tx=0x7f7548005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490198d36babb91f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f753d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f753d7fa640 1 -- 172.21.15.64:0/1554837482 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548029060 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75480054d0 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.855+0000 7f753d7fa640 1 -- 172.21.15.64:0/1554837482 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f7548005890 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f753d7fa640 1 -- 172.21.15.64:0/1554837482 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75480049e0 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 10 client.17823 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a8736db40 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f753d7fa640 1 -- 172.21.15.64:0/1554837482 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75480054d0 con 0x563a873544a0 2024-09-19T05:15:52.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f753d7fa640 1 -- 172.21.15.64:0/1554837482 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7548019430 con 0x563a873544a0 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 10 client.17823 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 20 client.17823 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a8736f000 0x563a873713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 4 client.17823 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a872c7850 con 0x563a8736f000 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.856+0000 7f7559487640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a8736f000 0x563a873713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f7559487640 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a8736f000 0x563a873713f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db0816aa740f31a server_cookie=ca36a5546a68a2f in_seq=0 out_seq=0 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f753d7fa640 10 client.17823 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f753d7fa640 1 -- 172.21.15.64:0/1554837482 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a872c7850 con 0x563a8736f000 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f753d7fa640 10 client.17823 handle_command_reply: tid=0 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f755e2c5740 1 client.17823 shutdown 2024-09-19T05:15:52.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 msgr2=0x563a8735ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.857+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a8735ef90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7548006010 tx=0x7f7548005230 comp rx=0 tx=0).stop 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 shutdown_connections 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8735c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a87354dc0 0x563a8735c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a8736f000 0x563a873713f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1554837482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a8735ef90 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f753d7fa640 0 client.17823 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 >> 172.21.15.64:0/1554837482 conn(0x563a87350a30 msgr2=0x563a87356760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 shutdown_connections 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 1 -- 172.21.15.64:0/1554837482 wait complete. 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.858+0000 7f755e2c5740 20 client.17823 trim_cache size 0 max 16384 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 Processor -- start 2024-09-19T05:15:52.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 -- start start 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a873544a0 0x563a8736b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8734fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f75541b01b0 con 0x563a87354dc0 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75541ada80 con 0x563a8735f680 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f755e2c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75541ab8f0 con 0x563a873544a0 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47890/0 (socket says 172.21.15.64:47890) 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 -- 172.21.15.64:0/2225466369 learned_addr learned my addr 172.21.15.64:0/2225466369 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559c88640 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a873544a0 0x563a8736b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 -- 172.21.15.64:0/2225466369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a873544a0 msgr2=0x563a8736b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a873544a0 0x563a8736b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 -- 172.21.15.64:0/2225466369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 msgr2=0x563a8734fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8734fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.875+0000 7f7559487640 1 -- 172.21.15.64:0/2225466369 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a87350860 con 0x563a87354dc0 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f7559c88640 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a873544a0 0x563a8736b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f7559487640 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735bb20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7550003c80 tx=0x7f75500e76a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e32b3f22cbf896a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f753f7fe640 1 -- 172.21.15.64:0/2225466369 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550049040 con 0x563a87354dc0 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f753f7fe640 1 -- 172.21.15.64:0/2225466369 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75500bc8d0 con 0x563a87354dc0 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f753f7fe640 1 -- 172.21.15.64:0/2225466369 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75500bac60 con 0x563a87354dc0 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f755e2c5740 1 -- 172.21.15.64:0/2225466369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 msgr2=0x563a8735bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.876+0000 7f755e2c5740 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735bb20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7550003c80 tx=0x7f75500e76a0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- 172.21.15.64:0/2225466369 shutdown_connections 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a8735f680 0x563a8734fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a87354dc0 0x563a8735bb20 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 --2- 172.21.15.64:0/2225466369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a873544a0 0x563a8736b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- 172.21.15.64:0/2225466369 >> 172.21.15.64:0/2225466369 conn(0x563a87350a30 msgr2=0x563a87357800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- 172.21.15.64:0/2225466369 shutdown_connections 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- 172.21.15.64:0/2225466369 wait complete. 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- start start 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 Processor -- start 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a87367990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87354dc0 0x563a87367ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87362f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f75541abf40 con 0x563a873544a0 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f75541ad3b0 con 0x563a87354dc0 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75541ac610 con 0x563a8735f680 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.877+0000 7f755e2c5740 10 client.0 _ll_register_callbacks cb 0x7f754c00d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:52.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a87367990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a87367990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47900/0 (socket says 172.21.15.64:47900) 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 -- 172.21.15.64:0/1366767107 learned_addr learned my addr 172.21.15.64:0/1366767107 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559487640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87354dc0 0x563a87367ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 -- 172.21.15.64:0/1366767107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 msgr2=0x563a87362f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f755a489640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87362f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87362f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 -- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87354dc0 msgr2=0x563a87367ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87354dc0 0x563a87367ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 -- 172.21.15.64:0/1366767107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a87363740 con 0x563a873544a0 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559487640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87354dc0 0x563a87367ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f7559c88640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a87367990 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7548002230 tx=0x7f7548005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76156c9318ea9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f753d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:15:52.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.878+0000 7f753d7fa640 1 -- 172.21.15.64:0/1366767107 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548029040 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f753d7fa640 1 -- 172.21.15.64:0/1366767107 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f7548021070 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75500bc8d0 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f753d7fa640 1 -- 172.21.15.64:0/1366767107 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7548009790 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 10 client.17829 fetch_fsmap learned FSMap version 25 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a873d9580 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f753d7fa640 1 -- 172.21.15.64:0/1366767107 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75500bc8d0 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f753d7fa640 1 -- 172.21.15.64:0/1366767107 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7548025930 con 0x563a873544a0 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 10 client.17829 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 20 client.17829 populate_metadata read hostname 'smithi064' 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a873d9ca0 0x563a873dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:52.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 4 client.17829 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a872c1d60 con 0x563a873d9ca0 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.879+0000 7f7559487640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a873d9ca0 0x563a873dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f753d7fa640 10 client.17829 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f7559487640 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a873d9ca0 0x563a873dc0c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4646dc9a936022d8 server_cookie=dea1c6e5a38ea557 in_seq=0 out_seq=0 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f753d7fa640 1 -- 172.21.15.64:0/1366767107 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563a872c1d60 con 0x563a873d9ca0 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f753d7fa640 10 client.17829 handle_command_reply: tid=0 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 client.17829 shutdown 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 msgr2=0x563a87367990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a87367990 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7548002230 tx=0x7f7548005ac0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 shutdown_connections 2024-09-19T05:15:52.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8735f680 0x563a87362f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: "c5e6956e-7a25-4117-b87e-aa754921129e": { 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "c5e6956e-7a25-4117-b87e-aa754921129e", 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:15:52.038 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a87354dc0 0x563a87367ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x563a873d9ca0 0x563a873dc0c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 --2- 172.21.15.64:0/1366767107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a873544a0 0x563a87367990 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f753d7fa640 0 client.17829 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 >> 172.21.15.64:0/1366767107 conn(0x563a87350a30 msgr2=0x563a87356710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 shutdown_connections 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.880+0000 7f755e2c5740 1 -- 172.21.15.64:0/1366767107 wait complete. 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755e2c5740 20 client.17829 trim_cache size 0 max 16384 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f75400a33b0 msgr2=0x7f75400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f75400a33b0 0x7f75400a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f754c005fd0 tx=0x7f754c004730 comp rx=0 tx=0).stop 2024-09-19T05:15:52.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 msgr2=0x7f75541988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f75541988e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7550005940 tx=0x7f75500042f0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 shutdown_connections 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f75400a33b0 0x7f75400a5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755410c170 0x7f755419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7554103660 0x7f75541988e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 --2- 172.21.15.64:0/1461969259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7554102d40 0x7f75541983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 >> 172.21.15.64:0/1461969259 conn(0x7f75540fc7a0 msgr2=0x7f75540fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 shutdown_connections 2024-09-19T05:15:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:15:51.881+0000 7f755beec640 1 -- 172.21.15.64:0/1461969259 wait complete. 2024-09-19T05:15:52.670 INFO:tasks.cephfs.filesystem:scrub status for tag:c5e6956e-7a25-4117-b87e-aa754921129e - {'path': '/', 'tag': 'c5e6956e-7a25-4117-b87e-aa754921129e', 'options': 'recursive,force'} 2024-09-19T05:15:53.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:53 smithi174 ceph-mon[34488]: pgmap v308: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:53.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:53 smithi064 ceph-mon[30960]: pgmap v308: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:53.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:53 smithi103 ceph-mon[34446]: pgmap v308: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:54.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:54 smithi174 ceph-mon[34488]: scrub summary: idle 2024-09-19T05:15:54.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:54 smithi064 ceph-mon[30960]: scrub summary: idle 2024-09-19T05:15:54.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:54 smithi103 ceph-mon[34446]: scrub summary: idle 2024-09-19T05:15:55.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:55 smithi174 ceph-mon[34488]: pgmap v309: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:55.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:55 smithi064 ceph-mon[30960]: pgmap v309: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:55 smithi103 ceph-mon[34446]: pgmap v309: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:15:57.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:57 smithi064 ceph-mon[30960]: pgmap v310: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:15:57.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:57 smithi103 ceph-mon[34446]: pgmap v310: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:15:57.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:57 smithi174 ceph-mon[34488]: pgmap v310: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:15:59.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:15:59 smithi064 ceph-mon[30960]: pgmap v311: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:59.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:15:59 smithi103 ceph-mon[34446]: pgmap v311: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:15:59.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:15:59 smithi174 ceph-mon[34488]: pgmap v311: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:01.580 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:01 smithi174 ceph-mon[34488]: pgmap v312: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:16:01.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:01 smithi064 ceph-mon[30960]: pgmap v312: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:16:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:01 smithi103 ceph-mon[34446]: pgmap v312: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-19T05:16:03.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:03 smithi064 ceph-mon[30960]: pgmap v313: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:03.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:03 smithi103 ceph-mon[34446]: pgmap v313: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:03.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:03 smithi174 ceph-mon[34488]: pgmap v313: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:04.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T05:16:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 -- 172.21.15.64:0/3948634782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201069e0 msgr2=0x7f9d20106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/3948634782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201069e0 0x7f9d20106dc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009930 tx=0x7f9d1002a420 comp rx=0 tx=0).stop 2024-09-19T05:16:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 -- 172.21.15.64:0/3948634782 shutdown_connections 2024-09-19T05:16:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/3948634782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d20101370 0x7f9d2010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/3948634782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d201009d0 0x7f9d20100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/3948634782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d201069e0 0x7f9d20106dc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 -- 172.21.15.64:0/3948634782 >> 172.21.15.64:0/3948634782 conn(0x7f9d200fc6f0 msgr2=0x7f9d200feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 -- 172.21.15.64:0/3948634782 shutdown_connections 2024-09-19T05:16:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.605+0000 7f9d25bd6640 1 -- 172.21.15.64:0/3948634782 wait complete. 2024-09-19T05:16:04.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 Processor -- start 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 -- start start 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d201009d0 0x7f9d20197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 0x7f9d201976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d201069e0 0x7f9d2019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20110cd0 con 0x7f9d20101370 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20110b50 con 0x7f9d201069e0 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d25bd6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20110e50 con 0x7f9d201009d0 2024-09-19T05:16:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d1effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 0x7f9d201976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.606+0000 7f9d1effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 0x7f9d201976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53084/0 (socket says 172.21.15.64:53084) 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 -- 172.21.15.64:0/1419923717 learned_addr learned my addr 172.21.15.64:0/1419923717 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1ffff640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d201069e0 0x7f9d2019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1f7fe640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d201009d0 0x7f9d20197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 -- 172.21.15.64:0/1419923717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d201009d0 msgr2=0x7f9d20197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d201009d0 0x7f9d20197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 -- 172.21.15.64:0/1419923717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d201069e0 msgr2=0x7f9d2019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d201069e0 0x7f9d2019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 -- 172.21.15.64:0/1419923717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2019c180 con 0x7f9d20101370 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1effd640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 0x7f9d201976d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400eb00 tx=0x7f9d14009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1401c070 con 0x7f9d20101370 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d1400bc30 con 0x7f9d20101370 2024-09-19T05:16:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14016610 con 0x7f9d20101370 2024-09-19T05:16:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2019c470 con 0x7f9d20101370 2024-09-19T05:16:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.607+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2006a300 con 0x7f9d20101370 2024-09-19T05:16:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.608+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d20100e30 con 0x7f9d20101370 2024-09-19T05:16:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.609+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f9d1401f070 con 0x7f9d20101370 2024-09-19T05:16:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.610+0000 7f9d1cff9640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d080a34e0 0x7f9d080a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.611+0000 7f9d1f7fe640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d080a34e0 0x7f9d080a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.611+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f9d140d1290 con 0x7f9d20101370 2024-09-19T05:16:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.611+0000 7f9d1f7fe640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d080a34e0 0x7f9d080a59a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006010 tx=0x7f9d10000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.615+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d1400bde0 con 0x7f9d20101370 2024-09-19T05:16:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.812+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9d2006ae10 con 0x7f9d080a34e0 2024-09-19T05:16:04.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.820+0000 7f9d1cff9640 1 -- 172.21.15.64:0/1419923717 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9d2006ae10 con 0x7f9d080a34e0 2024-09-19T05:16:04.823 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T05:06:03", "first": "2024-09-19T05:07:00", "last": "2024-09-19T05:16:00", "last_pruned": "2024-09-19T05:16:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d080a34e0 msgr2=0x7f9d080a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d080a34e0 0x7f9d080a59a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006010 tx=0x7f9d10000bc0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 msgr2=0x7f9d201976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 0x7f9d201976d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400eb00 tx=0x7f9d14009f20 comp rx=0 tx=0).stop 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 shutdown_connections 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f9d080a34e0 0x7f9d080a59a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d201069e0 0x7f9d2019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d20101370 0x7f9d201976d0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 --2- 172.21.15.64:0/1419923717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d201009d0 0x7f9d20197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 >> 172.21.15.64:0/1419923717 conn(0x7f9d200fc6f0 msgr2=0x7f9d200fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:04.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 shutdown_connections 2024-09-19T05:16:04.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.822+0000 7f9d25bd6640 1 -- 172.21.15.64:0/1419923717 wait complete. 2024-09-19T05:16:04.836 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1882332149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a94101370 msgr2=0x7f1a9410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1882332149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a94101370 0x7f1a9410d8a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009930 tx=0x7f1a8c02a420 comp rx=0 tx=0).stop 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1882332149 shutdown_connections 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1882332149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a94101370 0x7f1a9410d8a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1882332149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a941009d0 0x7f1a94100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1882332149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a941069e0 0x7f1a94106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1882332149 >> 172.21.15.64:0/1882332149 conn(0x7f1a940fc6f0 msgr2=0x7f1a940feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1882332149 shutdown_connections 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1882332149 wait complete. 2024-09-19T05:16:04.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 Processor -- start 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.939+0000 7f1a9bf11640 1 -- start start 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 0x7f1a941982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94101370 0x7f1a941987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a941069e0 0x7f1a9419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94111dc0 con 0x7f1a941009d0 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94111c40 con 0x7f1a94101370 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94111f40 con 0x7f1a941069e0 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 0x7f1a941982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 0x7f1a941982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53098/0 (socket says 172.21.15.64:53098) 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 -- 172.21.15.64:0/1409604680 learned_addr learned my addr 172.21.15.64:0/1409604680 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9a4ae640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a941069e0 0x7f1a9419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a994ac640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94101370 0x7f1a941987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 -- 172.21.15.64:0/1409604680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a941069e0 msgr2=0x7f1a9419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a941069e0 0x7f1a9419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 -- 172.21.15.64:0/1409604680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94101370 msgr2=0x7f1a941987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94101370 0x7f1a941987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 -- 172.21.15.64:0/1409604680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9419d2a0 con 0x7f1a941009d0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a994ac640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94101370 0x7f1a941987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a99cad640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 0x7f1a941982b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400ba50 tx=0x7f1a8400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84019070 con 0x7f1a941009d0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9419d590 con 0x7f1a941009d0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a8400cbe0 con 0x7f1a941009d0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.940+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840075d0 con 0x7f1a941009d0 2024-09-19T05:16:04.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.941+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941a4e20 con 0x7f1a941009d0 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.941+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a94100e30 con 0x7f1a941009d0 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.943+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1a8401c070 con 0x7f1a941009d0 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.943+0000 7f1a92ffd640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a7c0a3460 0x7f1a7c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.944+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8571+0+0 (secure 0 0 0) 0x7f1a840cf790 con 0x7f1a941009d0 2024-09-19T05:16:04.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.944+0000 7f1a994ac640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a7c0a3460 0x7f1a7c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.944+0000 7f1a994ac640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a7c0a3460 0x7f1a7c0a5920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a94105f60 tx=0x7f1a88005e50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:04.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:04.946+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1a8409c250 con 0x7f1a941009d0 2024-09-19T05:16:05.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.147+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 --> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1a940fe530 con 0x7f1a7c0a3460 2024-09-19T05:16:05.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.153+0000 7f1a92ffd640 1 -- 172.21.15.64:0/1409604680 <== mgr.15126 v2:172.21.15.64:6800/2647680663 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1a940fe530 con 0x7f1a7c0a3460 2024-09-19T05:16:05.156 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1m 6m3h 2024-09-19T05:16:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a7c0a3460 msgr2=0x7f1a7c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a7c0a3460 0x7f1a7c0a5920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a94105f60 tx=0x7f1a88005e50 comp rx=0 tx=0).stop 2024-09-19T05:16:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 msgr2=0x7f1a941982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 0x7f1a941982b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400ba50 tx=0x7f1a8400bf10 comp rx=0 tx=0).stop 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 shutdown_connections 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f1a7c0a3460 0x7f1a7c0a5920 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a941069e0 0x7f1a9419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94101370 0x7f1a941987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 --2- 172.21.15.64:0/1409604680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a941009d0 0x7f1a941982b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 >> 172.21.15.64:0/1409604680 conn(0x7f1a940fc6f0 msgr2=0x7f1a940fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 shutdown_connections 2024-09-19T05:16:05.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:05.156+0000 7f1a9bf11640 1 -- 172.21.15.64:0/1409604680 wait complete. 2024-09-19T05:16:05.169 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T05:16:05.180 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T05:16:05.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:05 smithi064 ceph-mon[30960]: pgmap v314: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:05.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:05 smithi103 ceph-mon[34446]: pgmap v314: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:05.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:05 smithi174 ceph-mon[34488]: pgmap v314: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-19T05:16:06.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:06 smithi064 ceph-mon[30960]: from='client.17835 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:16:06.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:06 smithi064 ceph-mon[30960]: from='client.17841 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:16:06.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:06 smithi103 ceph-mon[34446]: from='client.17835 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:16:06.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:06 smithi103 ceph-mon[34446]: from='client.17841 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:16:06.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:06 smithi174 ceph-mon[34488]: from='client.17835 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:16:06.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:06 smithi174 ceph-mon[34488]: from='client.17841 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:16:07.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:07 smithi064 ceph-mon[30960]: pgmap v315: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:16:07.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:07 smithi103 ceph-mon[34446]: pgmap v315: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:16:07.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:07 smithi174 ceph-mon[34488]: pgmap v315: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:16:09.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:09 smithi064 ceph-mon[30960]: pgmap v316: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-19T05:16:09.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:09 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:16:09.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:09 smithi103 ceph-mon[34446]: pgmap v316: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-19T05:16:09.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:09 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:16:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:09 smithi174 ceph-mon[34488]: pgmap v316: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-19T05:16:09.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:09 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:16:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:10 smithi064 ceph-mon[30960]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:16:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:10 smithi103 ceph-mon[34446]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:16:10.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:10 smithi174 ceph-mon[34488]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:16:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: pgmap v318: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: Unable to set osd_memory_target on smithi174 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:16:11.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: pgmap v318: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: Unable to set osd_memory_target on smithi174 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:16:11.720 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:16:11.721 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: pgmap v318: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: Unable to set osd_memory_target on smithi174 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-19T05:16:11.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:16:11.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:16:12.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:12 smithi064 ceph-mon[30960]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T05:16:12.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:12 smithi064 ceph-mon[30960]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T05:16:12.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:12 smithi103 ceph-mon[34446]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T05:16:12.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:12 smithi103 ceph-mon[34446]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T05:16:12.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:12 smithi174 ceph-mon[34488]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T05:16:12.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:12 smithi174 ceph-mon[34488]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T05:16:13.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:13 smithi064 ceph-mon[30960]: pgmap v319: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:13.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:13 smithi103 ceph-mon[34446]: pgmap v319: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:13.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:13 smithi174 ceph-mon[34488]: pgmap v319: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:15.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:15 smithi064 ceph-mon[30960]: pgmap v320: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:15.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:15 smithi103 ceph-mon[34446]: pgmap v320: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:15.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:15 smithi174 ceph-mon[34488]: pgmap v320: 97 pgs: 97 active+clean; 19 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-09-19T05:16:17.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:17 smithi064 ceph-mon[30960]: pgmap v321: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-19T05:16:17.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:17 smithi064 ceph-mon[30960]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:16:17.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:17 smithi103 ceph-mon[34446]: pgmap v321: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-19T05:16:17.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:17 smithi103 ceph-mon[34446]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:16:17.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:17 smithi174 ceph-mon[34488]: pgmap v321: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-09-19T05:16:17.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:17 smithi174 ceph-mon[34488]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:16:19.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:19 smithi064 ceph-mon[30960]: pgmap v323: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 239 B/s wr, 10 op/s 2024-09-19T05:16:19.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:19 smithi103 ceph-mon[34446]: pgmap v323: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 239 B/s wr, 10 op/s 2024-09-19T05:16:19.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:19 smithi174 ceph-mon[34488]: pgmap v323: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 239 B/s wr, 10 op/s 2024-09-19T05:16:21.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:21 smithi064 ceph-mon[30960]: pgmap v324: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:21.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:21 smithi103 ceph-mon[34446]: pgmap v324: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:21.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:21 smithi174 ceph-mon[34488]: pgmap v324: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:22.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:16:22.944 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:23.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 -- 172.21.15.64:0/3401966575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784103660 msgr2=0x7f278410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 --2- 172.21.15.64:0/3401966575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784103660 0x7f278410bb50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2774009900 tx=0x7f277402a480 comp rx=0 tx=0).stop 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 -- 172.21.15.64:0/3401966575 shutdown_connections 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 --2- 172.21.15.64:0/3401966575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f278410c170 0x7f278410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 --2- 172.21.15.64:0/3401966575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784103660 0x7f278410bb50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 --2- 172.21.15.64:0/3401966575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2784102d40 0x7f2784103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 -- 172.21.15.64:0/3401966575 >> 172.21.15.64:0/3401966575 conn(0x7f27840fc7a0 msgr2=0x7f27840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 -- 172.21.15.64:0/3401966575 shutdown_connections 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 -- 172.21.15.64:0/3401966575 wait complete. 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.551+0000 7f278bdc7640 1 Processor -- start 2024-09-19T05:16:23.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 -- start start 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 0x7f2784198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2784103660 0x7f27841988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f278410c170 0x7f278419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2784111e20 con 0x7f2784102d40 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2784111ca0 con 0x7f2784103660 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278bdc7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2784111fa0 con 0x7f278410c170 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 0x7f2784198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 0x7f2784198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44228/0 (socket says 172.21.15.64:44228) 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 -- 172.21.15.64:0/1954045875 learned_addr learned my addr 172.21.15.64:0/1954045875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789362640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2784103660 0x7f27841988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f278a364640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f278410c170 0x7f278419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 -- 172.21.15.64:0/1954045875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f278410c170 msgr2=0x7f278419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f278410c170 0x7f278419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 -- 172.21.15.64:0/1954045875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2784103660 msgr2=0x7f27841988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2784103660 0x7f27841988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 -- 172.21.15.64:0/1954045875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278419d320 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789362640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2784103660 0x7f27841988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f2789b63640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 0x7f2784198370 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f278000d770 tx=0x7f278000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.552+0000 7f277affd640 1 -- 172.21.15.64:0/1954045875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780017070 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.553+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278419d610 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.553+0000 7f277affd640 1 -- 172.21.15.64:0/1954045875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27800054d0 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.553+0000 7f277affd640 1 -- 172.21.15.64:0/1954045875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004cf0 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.553+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841a4ef0 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.555+0000 7f277affd640 1 -- 172.21.15.64:0/1954045875 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2780004e90 con 0x7f2784102d40 2024-09-19T05:16:23.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.555+0000 7f278e1a0740 1 Processor -- start 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 -- start start 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe8c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe97140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbe99a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f277affd640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27700a33b0 0x7f27700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fabbdf22e0 con 0x55fabbe8c4a0 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fabbd99ce0 con 0x55fabbe8cdc0 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278e1a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fabbdee2a0 con 0x55fabbe97680 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278a364640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe8c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278a364640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe8c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44238/0 (socket says 172.21.15.64:44238) 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278a364640 1 -- 172.21.15.64:0/2974639348 learned_addr learned my addr 172.21.15.64:0/2974639348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:23.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f277affd640 1 -- 172.21.15.64:0/1954045875 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f27800cf870 con 0x7f2784102d40 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f2789362640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27700a33b0 0x7f27700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f2789b63640 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe97140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278a364640 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbe99a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278a364640 1 -- 172.21.15.64:0/2974639348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 msgr2=0x55fabbe99a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.556+0000 7f278a364640 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbe99a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f278a364640 1 -- 172.21.15.64:0/2974639348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 msgr2=0x55fabbe97140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f278a364640 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe97140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f278a364640 1 -- 172.21.15.64:0/2974639348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fabbd7e440 con 0x55fabbe8c4a0 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f2789b63640 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe97140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f278a364640 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe8c880 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x55fabbe89db0 tx=0x7f277c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43055d869e6a27b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f2789362640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27700a33b0 0x7f27700a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2774031440 tx=0x7f2774005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:23.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f276ffff640 1 -- 172.21.15.64:0/2974639348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c012110 con 0x55fabbe8c4a0 2024-09-19T05:16:23.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f276ffff640 1 -- 172.21.15.64:0/2974639348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277c003970 con 0x55fabbe8c4a0 2024-09-19T05:16:23.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.557+0000 7f276ffff640 1 -- 172.21.15.64:0/2974639348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c00eb70 con 0x55fabbe8c4a0 2024-09-19T05:16:23.564 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:23 smithi064 ceph-mon[30960]: pgmap v325: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:23.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 -- 172.21.15.64:0/2974639348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 msgr2=0x55fabbe8c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe8c880 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x55fabbe89db0 tx=0x7f277c009b10 comp rx=0 tx=0).stop 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 -- 172.21.15.64:0/2974639348 shutdown_connections 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbe99a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe97140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2974639348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe8c880 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 -- 172.21.15.64:0/2974639348 >> 172.21.15.64:0/2974639348 conn(0x55fabbe88a30 msgr2=0x55fabbe88e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 -- 172.21.15.64:0/2974639348 shutdown_connections 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 -- 172.21.15.64:0/2974639348 wait complete. 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.558+0000 7f278e1a0740 1 -- start start 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 Processor -- start 2024-09-19T05:16:23.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbe97000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 0x55fabbe94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fabbe9a250 con 0x55fabbe97680 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fabbdad050 con 0x55fabbe8cdc0 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fabbe9a3d0 con 0x55fabbe8c4a0 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278e1a0740 10 client.0 _ll_register_callbacks cb 0x55fabbded160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278a364640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 0x55fabbe94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278a364640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 0x55fabbe94830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44252/0 (socket says 172.21.15.64:44252) 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.559+0000 7f278a364640 1 -- 172.21.15.64:0/4063682667 learned_addr learned my addr 172.21.15.64:0/4063682667 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f2789362640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f2789b63640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbe97000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278a364640 1 -- 172.21.15.64:0/4063682667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 msgr2=0x55fabbe97000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278a364640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbe97000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278a364640 1 -- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 msgr2=0x55fabbe942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278a364640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278a364640 1 -- 172.21.15.64:0/4063682667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fabbb633c0 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278a364640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 0x55fabbe94830 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f277c009540 tx=0x7f277c004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb712e131984b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f276d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f276d7fa640 1 -- 172.21.15.64:0/4063682667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c00ed00 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f276d7fa640 1 -- 172.21.15.64:0/4063682667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f277c005000 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c003970 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.560+0000 7f276d7fa640 1 -- 172.21.15.64:0/4063682667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f277c00eea0 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 10 client.17853 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fabbea5c40 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f276d7fa640 1 -- 172.21.15.64:0/4063682667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277c003970 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f276d7fa640 1 -- 172.21.15.64:0/4063682667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f277c010790 con 0x55fabbe97680 2024-09-19T05:16:23.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 10 client.17853 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 20 client.17853 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbea6f90 0x55fabbea9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 4 client.17853 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fabbdff850 con 0x55fabbea6f90 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.561+0000 7f2789b63640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbea6f90 0x55fabbea9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.562+0000 7f276d7fa640 10 client.17853 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.562+0000 7f2789b63640 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbea6f90 0x55fabbea9380 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c6423d8b37f795 server_cookie=62fd4e4ebc4c7bf9 in_seq=0 out_seq=0 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.562+0000 7f276d7fa640 1 -- 172.21.15.64:0/4063682667 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fabbdff850 con 0x55fabbea6f90 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.562+0000 7f276d7fa640 10 client.17853 handle_command_reply: tid=0 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.562+0000 7f278e1a0740 1 client.17853 shutdown 2024-09-19T05:16:23.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 msgr2=0x55fabbe94830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 0x55fabbe94830 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f277c009540 tx=0x7f277c004cf0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 shutdown_connections 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbea6f90 0x55fabbea9380 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe97680 0x55fabbe94830 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbe942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 --2- 172.21.15.64:0/4063682667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbe97000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 >> 172.21.15.64:0/4063682667 conn(0x55fabbe88a30 msgr2=0x55fabbe8e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f276d7fa640 0 client.17853 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 shutdown_connections 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 1 -- 172.21.15.64:0/4063682667 wait complete. 2024-09-19T05:16:23.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.563+0000 7f278e1a0740 20 client.17853 trim_cache size 0 max 16384 2024-09-19T05:16:23.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 Processor -- start 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 -- start start 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbea33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbeaea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b0190 con 0x55fabbe8c4a0 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27841ada60 con 0x55fabbe8cdc0 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f278e1a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27841ab8d0 con 0x55fabbe97680 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f2789b63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f2789b63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe87350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44268/0 (socket says 172.21.15.64:44268) 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.580+0000 7f2789b63640 1 -- 172.21.15.64:0/3500593666 learned_addr learned my addr 172.21.15.64:0/3500593666 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789362640 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbea33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789b63640 1 -- 172.21.15.64:0/3500593666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 msgr2=0x55fabbeaea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f278a364640 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbeaea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789b63640 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbeaea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789b63640 1 -- 172.21.15.64:0/3500593666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 msgr2=0x55fabbea33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789b63640 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbea33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789b63640 1 -- 172.21.15.64:0/3500593666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fabbe8cbf0 con 0x55fabbe8c4a0 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f278a364640 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbeaea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f2789b63640 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe87350 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2780005c20 tx=0x7f2780005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bd3f5900212f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f276f7fe640 1 -- 172.21.15.64:0/3500593666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780020040 con 0x55fabbe8c4a0 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f276f7fe640 1 -- 172.21.15.64:0/3500593666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f278009ce50 con 0x55fabbe8c4a0 2024-09-19T05:16:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.581+0000 7f276f7fe640 1 -- 172.21.15.64:0/3500593666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800a4300 con 0x55fabbe8c4a0 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 -- 172.21.15.64:0/3500593666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 msgr2=0x55fabbe87350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe87350 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2780005c20 tx=0x7f2780005bb0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 -- 172.21.15.64:0/3500593666 shutdown_connections 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe97680 0x55fabbeaea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe8cdc0 0x55fabbea33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 --2- 172.21.15.64:0/3500593666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8c4a0 0x55fabbe87350 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 -- 172.21.15.64:0/3500593666 >> 172.21.15.64:0/3500593666 conn(0x55fabbe88a30 msgr2=0x55fabbea2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:23.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 -- 172.21.15.64:0/3500593666 shutdown_connections 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 -- 172.21.15.64:0/3500593666 wait complete. 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 -- start start 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.582+0000 7f278e1a0740 1 Processor -- start 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbf092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 0x55fabbf097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe97680 0x55fabbe9f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27841abf20 con 0x55fabbe8cdc0 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27841ad390 con 0x55fabbe97680 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27841ac5f0 con 0x55fabbe8c4a0 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278e1a0740 10 client.0 _ll_register_callbacks cb 0x7f277c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 0x55fabbf097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 0x55fabbf097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44270/0 (socket says 172.21.15.64:44270) 2024-09-19T05:16:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 -- 172.21.15.64:0/2446104335 learned_addr learned my addr 172.21.15.64:0/2446104335 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789b63640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbf092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278a364640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe97680 0x55fabbe9f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 -- 172.21.15.64:0/2446104335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 msgr2=0x55fabbf092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbf092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 -- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe97680 msgr2=0x55fabbe9f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe97680 0x55fabbe9f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 -- 172.21.15.64:0/2446104335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fabbe9f960 con 0x55fabbe8cdc0 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f278a364640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe97680 0x55fabbe9f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f2789362640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 0x55fabbf097f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2774009900 tx=0x7f2774009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03364c41b6e1e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.583+0000 7f276d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f276d7fa640 1 -- 172.21.15.64:0/2446104335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774035050 con 0x55fabbe8cdc0 2024-09-19T05:16:23.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f276d7fa640 1 -- 172.21.15.64:0/2446104335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f27740035f0 con 0x55fabbe8cdc0 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278009ce50 con 0x55fabbe8cdc0 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f276d7fa640 1 -- 172.21.15.64:0/2446104335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27740037f0 con 0x55fabbe8cdc0 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 10 client.17859 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fabbf11b40 con 0x55fabbe8cdc0 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f276d7fa640 1 -- 172.21.15.64:0/2446104335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f278009ce50 con 0x55fabbe8cdc0 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f276d7fa640 1 -- 172.21.15.64:0/2446104335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2774049ad0 con 0x55fabbe8cdc0 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 10 client.17859 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 20 client.17859 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbf12370 0x55fabbf14790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 4 client.17859 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.584+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fabbdf9d60 con 0x55fabbf12370 2024-09-19T05:16:23.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f2789b63640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbf12370 0x55fabbf14790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f276d7fa640 10 client.17859 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f2789b63640 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbf12370 0x55fabbf14790 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db41d49010d09304 server_cookie=c008d53207a98bec in_seq=0 out_seq=0 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f276d7fa640 1 -- 172.21.15.64:0/2446104335 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fabbdf9d60 con 0x55fabbf12370 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f276d7fa640 10 client.17859 handle_command_reply: tid=0 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f278e1a0740 1 client.17859 shutdown 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 msgr2=0x55fabbf097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.585+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 0x55fabbf097f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2774009900 tx=0x7f2774009260 comp rx=0 tx=0).stop 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 shutdown_connections 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fabbe97680 0x55fabbe9f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x55fabbf12370 0x55fabbf14790 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fabbe8cdc0 0x55fabbf097f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 --2- 172.21.15.64:0/2446104335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fabbe8c4a0 0x55fabbf092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 >> 172.21.15.64:0/2446104335 conn(0x55fabbe88a30 msgr2=0x55fabbea33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f276d7fa640 0 client.17859 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 shutdown_connections 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 1 -- 172.21.15.64:0/2446104335 wait complete. 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278e1a0740 20 client.17859 trim_cache size 0 max 16384 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2024-09-19T05:16:23.593 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27700a33b0 msgr2=0x7f27700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278bdc7640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27700a33b0 0x7f27700a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2774031440 tx=0x7f2774005a20 comp rx=0 tx=0).stop 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 msgr2=0x7f2784198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.586+0000 7f278bdc7640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 0x7f2784198370 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f278000d770 tx=0x7f278000dc30 comp rx=0 tx=0).stop 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 shutdown_connections 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f27700a33b0 0x7f27700a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f278410c170 0x7f278419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2784103660 0x7f27841988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 --2- 172.21.15.64:0/1954045875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2784102d40 0x7f2784198370 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:23.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 >> 172.21.15.64:0/1954045875 conn(0x7f27840fc7a0 msgr2=0x7f27840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:23.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 shutdown_connections 2024-09-19T05:16:23.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:23.587+0000 7f278bdc7640 1 -- 172.21.15.64:0/1954045875 wait complete. 2024-09-19T05:16:23.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:23 smithi103 ceph-mon[34446]: pgmap v325: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:23.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:23 smithi174 ceph-mon[34488]: pgmap v325: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:24.363 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:16:24.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:16:24.627 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:25.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 -- 172.21.15.64:0/2048358996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10c170 msgr2=0x7fe7ec10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/2048358996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10c170 0x7fe7ec10e610 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009ca0 tx=0x7fe7e402a950 comp rx=0 tx=0).stop 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 -- 172.21.15.64:0/2048358996 shutdown_connections 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/2048358996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10c170 0x7fe7ec10e610 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/2048358996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 0x7fe7ec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/2048358996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec102d40 0x7fe7ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 -- 172.21.15.64:0/2048358996 >> 172.21.15.64:0/2048358996 conn(0x7fe7ec0fc7a0 msgr2=0x7fe7ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 -- 172.21.15.64:0/2048358996 shutdown_connections 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 -- 172.21.15.64:0/2048358996 wait complete. 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 Processor -- start 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.105+0000 7fe7f12bc640 1 -- start start 2024-09-19T05:16:25.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7f12bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 0x7fe7ec1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7f12bc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 0x7fe7ec1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7f12bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec10c170 0x7fe7ec19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7f12bc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec111ee0 con 0x7fe7ec102d40 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7f12bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec111d60 con 0x7fe7ec10c170 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7f12bc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec112060 con 0x7fe7ec103660 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 0x7fe7ec1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 0x7fe7ec1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52314/0 (socket says 172.21.15.64:52314) 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 -- 172.21.15.64:0/1358206390 learned_addr learned my addr 172.21.15.64:0/1358206390 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7ea7fc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 0x7fe7ec1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eb7fe640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec10c170 0x7fe7ec19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 -- 172.21.15.64:0/1358206390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 msgr2=0x7fe7ec1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:25.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 0x7fe7ec1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 -- 172.21.15.64:0/1358206390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec10c170 msgr2=0x7fe7ec19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec10c170 0x7fe7ec19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 -- 172.21.15.64:0/1358206390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19d450 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7ea7fc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 0x7fe7ec1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7eaffd640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 0x7fe7ec1984a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00b4a0 tx=0x7fe7dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc014070 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.106+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc010ac0 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.107+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19d740 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.107+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc017c30 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.107+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a5020 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.108+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec1031a0 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.109+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fe7dc017070 con 0x7fe7ec102d40 2024-09-19T05:16:25.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.110+0000 7fe7d3fff640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe7d40a33b0 0x7fe7d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:25.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.110+0000 7fe7ea7fc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe7d40a33b0 0x7fe7d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:25.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.110+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fe7dc0cf3c0 con 0x7fe7ec102d40 2024-09-19T05:16:25.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.110+0000 7fe7ea7fc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe7d40a33b0 0x7fe7d40a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0006fd0 tx=0x7fe7e0008040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:25.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.114+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7dc09be80 con 0x7fe7ec102d40 2024-09-19T05:16:25.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.326+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7ec107240 con 0x7fe7ec102d40 2024-09-19T05:16:25.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.327+0000 7fe7d3fff640 1 -- 172.21.15.64:0/1358206390 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fe7dc09b780 con 0x7fe7ec102d40 2024-09-19T05:16:25.329 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:16:25.329 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:16:25.330 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe7d40a33b0 msgr2=0x7fe7d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe7d40a33b0 0x7fe7d40a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0006fd0 tx=0x7fe7e0008040 comp rx=0 tx=0).stop 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 msgr2=0x7fe7ec1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 0x7fe7ec1984a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00b4a0 tx=0x7fe7dc00b960 comp rx=0 tx=0).stop 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 shutdown_connections 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fe7d40a33b0 0x7fe7d40a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec10c170 0x7fe7ec19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ec103660 0x7fe7ec1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 --2- 172.21.15.64:0/1358206390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec102d40 0x7fe7ec1984a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:25.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 >> 172.21.15.64:0/1358206390 conn(0x7fe7ec0fc7a0 msgr2=0x7fe7ec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:25.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 shutdown_connections 2024-09-19T05:16:25.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:25.329+0000 7fe7f12bc640 1 -- 172.21.15.64:0/1358206390 wait complete. 2024-09-19T05:16:25.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:25 smithi064 ceph-mon[30960]: pgmap v326: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:25.651 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:25 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1358206390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:16:25.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:25 smithi103 ceph-mon[34446]: pgmap v326: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:25.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:25 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1358206390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:16:25.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:25 smithi174 ceph-mon[34488]: pgmap v326: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-19T05:16:25.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:25 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1358206390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:16:26.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:16:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 -- 172.21.15.64:0/3563408475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098106f10 msgr2=0x7fd09810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 --2- 172.21.15.64:0/3563408475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098106f10 0x7fd09810ea90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd090009ca0 tx=0x7fd09002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 -- 172.21.15.64:0/3563408475 shutdown_connections 2024-09-19T05:16:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 --2- 172.21.15.64:0/3563408475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098106f10 0x7fd09810ea90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 --2- 172.21.15.64:0/3563408475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd098102790 0x7fd0981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 --2- 172.21.15.64:0/3563408475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd098101de0 0x7fd0981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 -- 172.21.15.64:0/3563408475 >> 172.21.15.64:0/3563408475 conn(0x7fd0980fd8c0 msgr2=0x7fd0980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 -- 172.21.15.64:0/3563408475 shutdown_connections 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 -- 172.21.15.64:0/3563408475 wait complete. 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.864+0000 7fd09e1db640 1 Processor -- start 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 -- start start 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd098101de0 0x7fd098198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 0x7fd098198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd098106f10 0x7fd09819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd098111e20 con 0x7fd098102790 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd098111ca0 con 0x7fd098101de0 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd09e1db640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd098111fa0 con 0x7fd098106f10 2024-09-19T05:16:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 0x7fd098198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 0x7fd098198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52342/0 (socket says 172.21.15.64:52342) 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 -- 172.21.15.64:0/1236198429 learned_addr learned my addr 172.21.15.64:0/1236198429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd097fff640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd098106f10 0x7fd09819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd0977fe640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd098101de0 0x7fd098198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 -- 172.21.15.64:0/1236198429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd098106f10 msgr2=0x7fd09819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd098106f10 0x7fd09819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 -- 172.21.15.64:0/1236198429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd098101de0 msgr2=0x7fd098198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd098101de0 0x7fd098198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.865+0000 7fd096ffd640 1 -- 172.21.15.64:0/1236198429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09819d2d0 con 0x7fd098102790 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.866+0000 7fd096ffd640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 0x7fd098198860 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c890 tx=0x7fd08c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.866+0000 7fd094ff9640 1 -- 172.21.15.64:0/1236198429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004220 con 0x7fd098102790 2024-09-19T05:16:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.866+0000 7fd094ff9640 1 -- 172.21.15.64:0/1236198429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c0043c0 con 0x7fd098102790 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.866+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09819d5c0 con 0x7fd098102790 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.866+0000 7fd094ff9640 1 -- 172.21.15.64:0/1236198429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004a00 con 0x7fd098102790 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.866+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a4ea0 con 0x7fd098102790 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 Processor -- start 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 -- start start 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd094ff9640 1 -- 172.21.15.64:0/1236198429 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fd08c004ba0 con 0x7fd098102790 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82edab40 0x564c82ed9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ed95d0 0x564c82ed9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82eeab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564c82e3f2f0 con 0x564c82ee8780 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c82de6cf0 con 0x564c82edab40 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.868+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c82e3b2b0 con 0x564c82ed95d0 2024-09-19T05:16:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82eeab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82eeab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52350/0 (socket says 172.21.15.64:52350) 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 -- 172.21.15.64:0/329883109 learned_addr learned my addr 172.21.15.64:0/329883109 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd097fff640 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82edab40 0x564c82ed9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd0977fe640 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ed95d0 0x564c82ed9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 -- 172.21.15.64:0/329883109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ed95d0 msgr2=0x564c82ed9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ed95d0 0x564c82ed9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 -- 172.21.15.64:0/329883109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82edab40 msgr2=0x564c82ed9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82edab40 0x564c82ed9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 -- 172.21.15.64:0/329883109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c82dcb3a0 con 0x564c82ee8780 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd0977fe640 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ed95d0 0x564c82ed9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd094ff9640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd0800a3430 0x7fd0800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd096ffd640 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82eeab70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0ccfb0 tx=0x7fd08c0d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1df4fc4306224e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:26.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd097fff640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd0800a3430 0x7fd0800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd094ff9640 1 -- 172.21.15.64:0/1236198429 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fd08c0cf920 con 0x7fd098102790 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.869+0000 7fd07dffb640 1 -- 172.21.15.64:0/329883109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c097e70 con 0x564c82ee8780 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd07dffb640 1 -- 172.21.15.64:0/329883109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c09c2d0 con 0x564c82ee8780 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd07dffb640 1 -- 172.21.15.64:0/329883109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c09b530 con 0x564c82ee8780 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 -- 172.21.15.64:0/329883109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 msgr2=0x564c82eeab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd097fff640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd0800a3430 0x7fd0800a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd090009430 tx=0x7fd090005ef0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82eeab70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0ccfb0 tx=0x7fd08c0d7690 comp rx=0 tx=0).stop 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 -- 172.21.15.64:0/329883109 shutdown_connections 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82eeab70 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ed95d0 0x564c82ed9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/329883109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82edab40 0x564c82ed9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 -- 172.21.15.64:0/329883109 >> 172.21.15.64:0/329883109 conn(0x564c82ea2310 msgr2=0x564c82ed5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:26.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 -- 172.21.15.64:0/329883109 shutdown_connections 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.870+0000 7fd0a05b4740 1 -- 172.21.15.64:0/329883109 wait complete. 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 -- start start 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 Processor -- start 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ee8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82edab40 0x564c82ee5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82ee5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564c82eeb480 con 0x564c82ee8780 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c82dfa060 con 0x564c82ed95d0 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c82eeb600 con 0x564c82edab40 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0a05b4740 10 client.0 _ll_register_callbacks cb 0x564c82b4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd097fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82ee5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd097fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82ee5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52352/0 (socket says 172.21.15.64:52352) 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd097fff640 1 -- 172.21.15.64:0/181715042 learned_addr learned my addr 172.21.15.64:0/181715042 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:26.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd0977fe640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ee8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.871+0000 7fd096ffd640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82edab40 0x564c82ee5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd097fff640 1 -- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82edab40 msgr2=0x564c82ee5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd097fff640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82edab40 0x564c82ee5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd097fff640 1 -- 172.21.15.64:0/181715042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 msgr2=0x564c82ee8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd097fff640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ee8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd097fff640 1 -- 172.21.15.64:0/181715042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c82bb03c0 con 0x564c82ee8780 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd097fff640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82ee5a50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd090009ca0 tx=0x7fd09002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0268b9aa49832a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd076ffd640 1 -- 172.21.15.64:0/181715042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090004250 con 0x564c82ee8780 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd076ffd640 1 -- 172.21.15.64:0/181715042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fd0900043f0 con 0x564c82ee8780 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c09c2d0 con 0x564c82ee8780 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd076ffd640 1 -- 172.21.15.64:0/181715042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0900045f0 con 0x564c82ee8780 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd0a05b4740 10 client.17877 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:26.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c82ef6ef0 con 0x564c82ee8780 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.872+0000 7fd076ffd640 1 -- 172.21.15.64:0/181715042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c09c2d0 con 0x564c82ee8780 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd076ffd640 1 -- 172.21.15.64:0/181715042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd090051070 con 0x564c82ee8780 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0a05b4740 10 client.17877 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0a05b4740 10 client.17877 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0a05b4740 20 client.17877 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82ef8300 0x564c82efa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0a05b4740 4 client.17877 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c82e4c860 con 0x564c82ef8300 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.873+0000 7fd0977fe640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82ef8300 0x564c82efa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd0977fe640 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82ef8300 0x564c82efa730 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47137fba06006945 server_cookie=cca89b15701f095 in_seq=0 out_seq=0 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd076ffd640 10 client.17877 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd076ffd640 1 -- 172.21.15.64:0/181715042 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c82e4c860 con 0x564c82ef8300 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd076ffd640 10 client.17877 handle_command_reply: tid=0 2024-09-19T05:16:26.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd0a05b4740 1 client.17877 shutdown 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 msgr2=0x564c82ee5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82ee5a50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd090009ca0 tx=0x7fd09002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 shutdown_connections 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.874+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82ef8300 0x564c82efa730 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82ee8780 0x564c82ee5a50 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82edab40 0x564c82ee5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/181715042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ee8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 >> 172.21.15.64:0/181715042 conn(0x564c82ea2310 msgr2=0x564c82edb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd076ffd640 0 client.17877 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 shutdown_connections 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 1 -- 172.21.15.64:0/181715042 wait complete. 2024-09-19T05:16:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.875+0000 7fd0a05b4740 20 client.17877 trim_cache size 0 max 16384 2024-09-19T05:16:26.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 Processor -- start 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 -- start start 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ed5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82ef4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f00000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981adf60 con 0x564c82edab40 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ab830 con 0x564c82ed95d0 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.888+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981a96a0 con 0x564c82ee8780 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82ef4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82ef4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52368/0 (socket says 172.21.15.64:52368) 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 -- 172.21.15.64:0/75644705 learned_addr learned my addr 172.21.15.64:0/75644705 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:26.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd0977fe640 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ed5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd097fff640 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 -- 172.21.15.64:0/75644705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 msgr2=0x564c82f00000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f00000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 -- 172.21.15.64:0/75644705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 msgr2=0x564c82ed5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ed5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 -- 172.21.15.64:0/75644705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c82ee8500 con 0x564c82edab40 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd096ffd640 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82ef4750 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0985a0 tx=0x7fd08c0058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8a32e28200732d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd07d7fa640 1 -- 172.21.15.64:0/75644705 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c01d050 con 0x564c82edab40 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd07d7fa640 1 -- 172.21.15.64:0/75644705 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c0d7e90 con 0x564c82edab40 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.889+0000 7fd07d7fa640 1 -- 172.21.15.64:0/75644705 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c09b710 con 0x564c82edab40 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 -- 172.21.15.64:0/75644705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 msgr2=0x564c82ef4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82ef4750 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0985a0 tx=0x7fd08c0058c0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 -- 172.21.15.64:0/75644705 shutdown_connections 2024-09-19T05:16:26.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f00000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82ef4750 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/75644705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82ed5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 -- 172.21.15.64:0/75644705 >> 172.21.15.64:0/75644705 conn(0x564c82ea2310 msgr2=0x564c82ed5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 -- 172.21.15.64:0/75644705 shutdown_connections 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 -- 172.21.15.64:0/75644705 wait complete. 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.890+0000 7fd0a05b4740 1 -- start start 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 Processor -- start 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82eecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82eed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f61fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981a9cf0 con 0x564c82edab40 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ab160 con 0x564c82ed95d0 2024-09-19T05:16:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981aa3c0 con 0x564c82ee8780 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0a05b4740 10 client.0 _ll_register_callbacks cb 0x7fd090009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82eed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82eed2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52374/0 (socket says 172.21.15.64:52374) 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd096ffd640 1 -- 172.21.15.64:0/4066099972 learned_addr learned my addr 172.21.15.64:0/4066099972 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd0977fe640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82eecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.891+0000 7fd097fff640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f61fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd096ffd640 1 -- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 msgr2=0x564c82f61fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd096ffd640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f61fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd096ffd640 1 -- 172.21.15.64:0/4066099972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 msgr2=0x564c82eecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd096ffd640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82eecd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd096ffd640 1 -- 172.21.15.64:0/4066099972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c82f62590 con 0x564c82edab40 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd0977fe640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82eecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:16:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd096ffd640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82eed2d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd08c086010 tx=0x7fd08c007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4524326d85b01e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd076ffd640 1 -- 172.21.15.64:0/4066099972 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0b8070 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd076ffd640 1 -- 172.21.15.64:0/4066099972 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fd08c0050b0 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c0d7e90 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd076ffd640 1 -- 172.21.15.64:0/4066099972 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd08c01d050 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd0a05b4740 10 client.17883 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c82f63850 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd076ffd640 1 -- 172.21.15.64:0/4066099972 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c0d7e90 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.892+0000 7fd076ffd640 1 -- 172.21.15.64:0/4066099972 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd08c0d5c50 con 0x564c82edab40 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 10 client.17883 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 10 client.17883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 20 client.17883 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82f638f0 0x564c82f65ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 4 client.17883 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 --> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c82e46d70 con 0x564c82f638f0 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0977fe640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82f638f0 0x564c82f65ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0977fe640 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82f638f0 0x564c82f65ce0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=464a15473a4eca38 server_cookie=e41a91f708635283 in_seq=0 out_seq=0 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd076ffd640 10 client.17883 ms_handle_connect on v2:172.21.15.174:6836/1599126810 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd076ffd640 1 -- 172.21.15.64:0/4066099972 <== mds.2 v2:172.21.15.174:6836/1599126810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c82e46d70 con 0x564c82f638f0 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd076ffd640 10 client.17883 handle_command_reply: tid=0 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.893+0000 7fd0a05b4740 1 client.17883 shutdown 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 msgr2=0x564c82eed2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82eed2d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd08c086010 tx=0x7fd08c007730 comp rx=0 tx=0).stop 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 shutdown_connections 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:6836/1599126810,v1:172.21.15.174:6837/1599126810] conn(0x564c82f638f0 0x564c82f65ce0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c82ee8780 0x564c82f61fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c82edab40 0x564c82eed2d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 --2- 172.21.15.64:0/4066099972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c82ed95d0 0x564c82eecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 >> 172.21.15.64:0/4066099972 conn(0x564c82ea2310 msgr2=0x564c82ed44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd076ffd640 0 client.17883 ms_handle_reset on v2:172.21.15.174:6836/1599126810 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 shutdown_connections 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 1 -- 172.21.15.64:0/4066099972 wait complete. 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd0a05b4740 20 client.17883 trim_cache size 0 max 16384 2024-09-19T05:16:26.903 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd0800a3430 msgr2=0x7fd0800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd09e1db640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd0800a3430 0x7fd0800a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd090009430 tx=0x7fd090005ef0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 msgr2=0x7fd098198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.894+0000 7fd09e1db640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 0x7fd098198860 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c890 tx=0x7fd08c00cd50 comp rx=0 tx=0).stop 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 shutdown_connections 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fd0800a3430 0x7fd0800a58f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd098106f10 0x7fd09819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd098102790 0x7fd098198860 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 --2- 172.21.15.64:0/1236198429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd098101de0 0x7fd098198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:26.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 >> 172.21.15.64:0/1236198429 conn(0x7fd0980fd8c0 msgr2=0x7fd0980fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:26.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 shutdown_connections 2024-09-19T05:16:26.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:26.895+0000 7fd09e1db640 1 -- 172.21.15.64:0/1236198429 wait complete. 2024-09-19T05:16:27.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:27 smithi064 ceph-mon[30960]: pgmap v327: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:16:27.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:16:27.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:27 smithi103 ceph-mon[34446]: pgmap v327: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:16:27.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:27 smithi174 ceph-mon[34488]: pgmap v327: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T05:16:27.869 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 -- 172.21.15.64:0/3659268299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008100a70 msgr2=0x7fc00810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3659268299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008100a70 0x7fc00810ea60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc000009ca0 tx=0x7fc00002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 -- 172.21.15.64:0/3659268299 shutdown_connections 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3659268299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008100a70 0x7fc00810ea60 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3659268299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0081000d0 0x7fc008100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3659268299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008106ba0 0x7fc008106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 -- 172.21.15.64:0/3659268299 >> 172.21.15.64:0/3659268299 conn(0x7fc008077210 msgr2=0x7fc0080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 -- 172.21.15.64:0/3659268299 shutdown_connections 2024-09-19T05:16:28.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.348+0000 7fc00c91e640 1 -- 172.21.15.64:0/3659268299 wait complete. 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 Processor -- start 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 -- start start 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0081000d0 0x7fc0081012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008100a70 0x7fc0081017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 0x7fc008101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc008111f40 con 0x7fc008106ba0 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc008111dc0 con 0x7fc0081000d0 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00c91e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081120c0 con 0x7fc008100a70 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc006d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 0x7fc008101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc006d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 0x7fc008101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52384/0 (socket says 172.21.15.64:52384) 2024-09-19T05:16:28.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc006d9d640 1 -- 172.21.15.64:0/3714902660 learned_addr learned my addr 172.21.15.64:0/3714902660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.349+0000 7fc00659c640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0081000d0 0x7fc0081012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fbffffff640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008100a70 0x7fc0081017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc006d9d640 1 -- 172.21.15.64:0/3714902660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008100a70 msgr2=0x7fc0081017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc006d9d640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008100a70 0x7fc0081017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc006d9d640 1 -- 172.21.15.64:0/3714902660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0081000d0 msgr2=0x7fc0081012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc006d9d640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0081000d0 0x7fc0081012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc006d9d640 1 -- 172.21.15.64:0/3714902660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0081a67c0 con 0x7fc008106ba0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fbffffff640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008100a70 0x7fc0081017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc006d9d640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 0x7fc008101d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc000009dc0 tx=0x7fc000032fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fbfff7fe640 1 -- 172.21.15.64:0/3714902660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000003cf0 con 0x7fc008106ba0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fbfff7fe640 1 -- 172.21.15.64:0/3714902660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0000044f0 con 0x7fc008106ba0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0081a6a50 con 0x7fc008106ba0 2024-09-19T05:16:28.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fbfff7fe640 1 -- 172.21.15.64:0/3714902660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000030b10 con 0x7fc008106ba0 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.350+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0081a6fe0 con 0x7fc008106ba0 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.352+0000 7fbfff7fe640 1 -- 172.21.15.64:0/3714902660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc000030d10 con 0x7fc008106ba0 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 Processor -- start 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 -- start start 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b85d0 0x556c4f4b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 0x556c4f4c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c4f41e2f0 con 0x556c4f4b9b40 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c4f3c5cf0 con 0x556c4f4b85d0 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c4f41a2b0 con 0x556c4f4c3370 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fbfff7fe640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbff40a3430 0x7fbff40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fbffffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fbffffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52394/0 (socket says 172.21.15.64:52394) 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fbffffff640 1 -- 172.21.15.64:0/3731585093 learned_addr learned my addr 172.21.15.64:0/3731585093 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.353+0000 7fc00659c640 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b85d0 0x556c4f4b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fc006d9d640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbff40a3430 0x7fbff40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbfff7fe640 1 -- 172.21.15.64:0/3714902660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fc0000f11e0 con 0x7fc008106ba0 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffffff640 1 -- 172.21.15.64:0/3731585093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 msgr2=0x556c4f4c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffffff640 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 0x556c4f4c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffffff640 1 -- 172.21.15.64:0/3731585093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b85d0 msgr2=0x556c4f4b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffffff640 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b85d0 0x556c4f4b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffffff640 1 -- 172.21.15.64:0/3731585093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4f3aa3a0 con 0x556c4f4b9b40 2024-09-19T05:16:28.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffffff640 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4b8090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005fd0 tx=0x7fbff00106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee4ddd24bab9a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffcff9640 1 -- 172.21.15.64:0/3731585093 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0016070 con 0x556c4f4b9b40 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fc006d9d640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbff40a3430 0x7fbff40a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0000bd530 tx=0x7fc00008f320 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffcff9640 1 -- 172.21.15.64:0/3731585093 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff00054d0 con 0x556c4f4b9b40 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.354+0000 7fbffcff9640 1 -- 172.21.15.64:0/3731585093 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0012420 con 0x556c4f4b9b40 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3731585093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 msgr2=0x556c4f4b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4b8090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005fd0 tx=0x7fbff00106e0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3731585093 shutdown_connections 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 0x556c4f4c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b85d0 0x556c4f4b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3731585093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4b8090 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3731585093 >> 172.21.15.64:0/3731585093 conn(0x556c4f481310 msgr2=0x556c4f4b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3731585093 shutdown_connections 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3731585093 wait complete. 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 -- start start 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.355+0000 7fc00ecf7740 1 Processor -- start 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4c3370 0x556c4f4c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c4f4c6070 con 0x556c4f4b9b40 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c4f3d9060 con 0x556c4f4c3370 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c4f4c61f0 con 0x556c4f4b85d0 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00ecf7740 10 client.0 _ll_register_callbacks cb 0x556c4f12c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4bffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52400/0 (socket says 172.21.15.64:52400) 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 -- 172.21.15.64:0/1659652055 learned_addr learned my addr 172.21.15.64:0/1659652055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fbffffff640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fbffffff640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4c3370 0x556c4f4c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 -- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 msgr2=0x556c4f4c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 -- 172.21.15.64:0/1659652055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4c3370 msgr2=0x556c4f4c0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4c3370 0x556c4f4c0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 -- 172.21.15.64:0/1659652055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4f18f3c0 con 0x556c4f4b9b40 2024-09-19T05:16:28.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fc00659c640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4bffc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbff8010a50 tx=0x7fbff8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487015dccb977823 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:28.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.356+0000 7fbfedffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:28.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fbfedffb640 1 -- 172.21.15.64:0/1659652055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff801e060 con 0x556c4f4b9b40 2024-09-19T05:16:28.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fbfedffb640 1 -- 172.21.15.64:0/1659652055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fbff800fbd0 con 0x556c4f4b9b40 2024-09-19T05:16:28.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff00054d0 con 0x556c4f4b9b40 2024-09-19T05:16:28.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fbfedffb640 1 -- 172.21.15.64:0/1659652055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbff800fdd0 con 0x556c4f4b9b40 2024-09-19T05:16:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 10 client.17895 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c4f4d1b50 con 0x556c4f4b9b40 2024-09-19T05:16:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fbfedffb640 1 -- 172.21.15.64:0/1659652055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff00054d0 con 0x556c4f4b9b40 2024-09-19T05:16:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fbfedffb640 1 -- 172.21.15.64:0/1659652055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbff800e5b0 con 0x556c4f4b9b40 2024-09-19T05:16:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 10 client.17895 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:28.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 10 client.17895 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:16:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 20 client.17895 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f4d3060 0x556c4f4d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 4 client.17895 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:16:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.357+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4f42b860 con 0x556c4f4d3060 2024-09-19T05:16:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.358+0000 7fbffffff640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f4d3060 0x556c4f4d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.358+0000 7fbffffff640 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f4d3060 0x556c4f4d5480 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58c456be2125f038 server_cookie=5144184056195e1a in_seq=0 out_seq=0 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.358+0000 7fbfedffb640 10 client.17895 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fbfedffb640 1 -- 172.21.15.64:0/1659652055 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c4f42b860 con 0x556c4f4d3060 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fbfedffb640 10 client.17895 handle_command_reply: tid=0 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 client.17895 shutdown 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 msgr2=0x556c4f4bffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4bffc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbff8010a50 tx=0x7fbff8010f10 comp rx=0 tx=0).stop 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 shutdown_connections 2024-09-19T05:16:28.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4c3370 0x556c4f4c0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f4d3060 0x556c4f4d5480 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b9b40 0x556c4f4bffc0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1659652055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 >> 172.21.15.64:0/1659652055 conn(0x556c4f481310 msgr2=0x556c4f4ba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fbfedffb640 0 client.17895 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 shutdown_connections 2024-09-19T05:16:28.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1659652055 wait complete. 2024-09-19T05:16:28.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.359+0000 7fc00ecf7740 20 client.17895 trim_cache size 0 max 16384 2024-09-19T05:16:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 Processor -- start 2024-09-19T05:16:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 -- start start 2024-09-19T05:16:28.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4befa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 0x556c4f4dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081b22a0 con 0x556c4f4c3370 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081afb70 con 0x556c4f4b9b40 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081ad9e0 con 0x556c4f4b85d0 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 0x556c4f4dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 0x556c4f4dad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52410/0 (socket says 172.21.15.64:52410) 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 -- 172.21.15.64:0/3184950274 learned_addr learned my addr 172.21.15.64:0/3184950274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fc00659c640 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 -- 172.21.15.64:0/3184950274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 msgr2=0x556c4f4cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 -- 172.21.15.64:0/3184950274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 msgr2=0x556c4f4befa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4befa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.372+0000 7fbffffff640 1 -- 172.21.15.64:0/3184950274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4f4bb120 con 0x556c4f4c3370 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fbffffff640 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 0x556c4f4dad30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbff0011010 tx=0x7fbff0004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5366745b895734e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fbfeffff640 1 -- 172.21.15.64:0/3184950274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0016030 con 0x556c4f4c3370 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fbfeffff640 1 -- 172.21.15.64:0/3184950274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff0004680 con 0x556c4f4c3370 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fbfeffff640 1 -- 172.21.15.64:0/3184950274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0024040 con 0x556c4f4c3370 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3184950274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 msgr2=0x556c4f4dad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 0x556c4f4dad30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbff0011010 tx=0x7fbff0004b80 comp rx=0 tx=0).stop 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3184950274 shutdown_connections 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4c3370 0x556c4f4dad30 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/3184950274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4b85d0 0x556c4f4cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.373+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3184950274 >> 172.21.15.64:0/3184950274 conn(0x556c4f481310 msgr2=0x556c4f4be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3184950274 shutdown_connections 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 -- 172.21.15.64:0/3184950274 wait complete. 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 -- start start 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 Processor -- start 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 0x556c4f4bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 0x556c4f4bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081ae030 con 0x556c4f4b85d0 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081af4a0 con 0x556c4f4b9b40 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0081ae700 con 0x556c4f4c3370 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00ecf7740 10 client.0 _ll_register_callbacks cb 0x7fbff800a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fbffffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 0x556c4f4bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fbffffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 0x556c4f4bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52426/0 (socket says 172.21.15.64:52426) 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fbffffff640 1 -- 172.21.15.64:0/1919132041 learned_addr learned my addr 172.21.15.64:0/1919132041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.374+0000 7fc00659c640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbffffff640 1 -- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 msgr2=0x556c4f4bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbffffff640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 0x556c4f4bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbffffff640 1 -- 172.21.15.64:0/1919132041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 msgr2=0x556c4f4bd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbffffff640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4bd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbffffff640 1 -- 172.21.15.64:0/1919132041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4f53cdd0 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fc00659c640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4bd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbffffff640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 0x556c4f4bd090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbff00048c0 tx=0x7fbff00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d2e44751c956c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbfedffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbfedffb640 1 -- 172.21.15.64:0/1919132041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0039060 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbfedffb640 1 -- 172.21.15.64:0/1919132041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fbff002c440 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff0004680 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fbfedffb640 1 -- 172.21.15.64:0/1919132041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbff0016040 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fc00ecf7740 10 client.17901 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.375+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c4f53e0b0 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fbfedffb640 1 -- 172.21.15.64:0/1919132041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff0004680 con 0x556c4f4b85d0 2024-09-19T05:16:28.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fbfedffb640 1 -- 172.21.15.64:0/1919132041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbff0033040 con 0x556c4f4b85d0 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00ecf7740 10 client.17901 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00ecf7740 10 client.17901 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00ecf7740 20 client.17901 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f53e490 0x556c4f540880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00ecf7740 4 client.17901 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 --> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c4f425d70 con 0x556c4f53e490 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.376+0000 7fc00659c640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f53e490 0x556c4f540880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fbfedffb640 10 client.17901 ms_handle_connect on v2:172.21.15.174:6838/3328031427 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00659c640 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f53e490 0x556c4f540880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bc200f28c9224c9 server_cookie=e7f66da1a7591541 in_seq=0 out_seq=0 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fbfedffb640 1 -- 172.21.15.64:0/1919132041 <== mds.1 v2:172.21.15.174:6838/3328031427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c4f425d70 con 0x556c4f53e490 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fbfedffb640 10 client.17901 handle_command_reply: tid=0 2024-09-19T05:16:28.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 client.17901 shutdown 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 msgr2=0x556c4f4bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 0x556c4f4bd090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbff00048c0 tx=0x7fbff00048f0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 shutdown_connections 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4f4c3370 0x556c4f4bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c4f4b9b40 0x556c4f4bd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.174:6838/3328031427,v1:172.21.15.174:6839/3328031427] conn(0x556c4f53e490 0x556c4f540880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 --2- 172.21.15.64:0/1919132041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4f4b85d0 0x556c4f4bd090 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 >> 172.21.15.64:0/1919132041 conn(0x556c4f481310 msgr2=0x556c4f4bf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fbfedffb640 0 client.17901 ms_handle_reset on v2:172.21.15.174:6838/3328031427 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 shutdown_connections 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 1 -- 172.21.15.64:0/1919132041 wait complete. 2024-09-19T05:16:28.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.377+0000 7fc00ecf7740 20 client.17901 trim_cache size 0 max 16384 2024-09-19T05:16:28.389 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:16:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbff40a3430 msgr2=0x7fbff40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbff40a3430 0x7fbff40a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc0000bd530 tx=0x7fc00008f320 comp rx=0 tx=0).stop 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 msgr2=0x7fc008101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 0x7fc008101d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc000009dc0 tx=0x7fc000032fe0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 shutdown_connections 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fbff40a3430 0x7fbff40a58f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc008106ba0 0x7fc008101d30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc008100a70 0x7fc0081017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 --2- 172.21.15.64:0/3714902660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0081000d0 0x7fc0081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 >> 172.21.15.64:0/3714902660 conn(0x7fc008077210 msgr2=0x7fc0080fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 shutdown_connections 2024-09-19T05:16:28.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:28.378+0000 7fc00c91e640 1 -- 172.21.15.64:0/3714902660 wait complete. 2024-09-19T05:16:29.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:16:29.464 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:29.585 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:29 smithi064 ceph-mon[30960]: pgmap v328: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T05:16:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:29 smithi103 ceph-mon[34446]: pgmap v328: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T05:16:29.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:29 smithi174 ceph-mon[34488]: pgmap v328: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-19T05:16:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 -- 172.21.15.64:0/2779798795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 msgr2=0x7ff71010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 --2- 172.21.15.64:0/2779798795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71010eaf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff708009cd0 tx=0x7ff70802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 -- 172.21.15.64:0/2779798795 shutdown_connections 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 --2- 172.21.15.64:0/2779798795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71010eaf0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 --2- 172.21.15.64:0/2779798795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff710100ca0 0x7ff710106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 --2- 172.21.15.64:0/2779798795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7101002f0 0x7ff7101006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 -- 172.21.15.64:0/2779798795 >> 172.21.15.64:0/2779798795 conn(0x7ff710077210 msgr2=0x7ff7100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 -- 172.21.15.64:0/2779798795 shutdown_connections 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.931+0000 7ff714f68640 1 -- 172.21.15.64:0/2779798795 wait complete. 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.932+0000 7ff714f68640 1 Processor -- start 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.932+0000 7ff714f68640 1 -- start start 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff714f68640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7101002f0 0x7ff7101983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff714f68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff710100ca0 0x7ff710198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff714f68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff714f68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111e50 con 0x7ff710106f50 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff714f68640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111cd0 con 0x7ff710100ca0 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff714f68640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111fd0 con 0x7ff7101002f0 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52450/0 (socket says 172.21.15.64:52450) 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 -- 172.21.15.64:0/2241231972 learned_addr learned my addr 172.21.15.64:0/2241231972 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff710100ca0 0x7ff710198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70e59c640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7101002f0 0x7ff7101983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 -- 172.21.15.64:0/2241231972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7101002f0 msgr2=0x7ff7101983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7101002f0 0x7ff7101983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 -- 172.21.15.64:0/2241231972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff710100ca0 msgr2=0x7ff710198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff710100ca0 0x7ff710198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 -- 172.21.15.64:0/2241231972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71019d390 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff70ed9d640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71019ccb0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff708005cc0 tx=0x7ff708003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.933+0000 7ff7077fe640 1 -- 172.21.15.64:0/2241231972 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708004450 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.934+0000 7ff7077fe640 1 -- 172.21.15.64:0/2241231972 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff708031c90 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.934+0000 7ff7077fe640 1 -- 172.21.15.64:0/2241231972 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70803b400 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.934+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71019d620 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.934+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7101a4f60 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.936+0000 7ff7077fe640 1 -- 172.21.15.64:0/2241231972 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff708004680 con 0x7ff710106f50 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.936+0000 7ff717341740 1 Processor -- start 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 -- start start 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9838b40 0x5571b9837090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b98375d0 0x5571b9837a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b9848b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5571b979d2f0 con 0x5571b9846780 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5571b9744cf0 con 0x5571b98375d0 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff717341740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5571b97992b0 con 0x5571b9838b40 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff7077fe640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b9848b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b9848b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52462/0 (socket says 172.21.15.64:52462) 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1055395875 learned_addr learned my addr 172.21.15.64:0/1055395875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:29.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9838b40 0x5571b9837090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70ed9d640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70e59c640 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b98375d0 0x5571b9837a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff7077fe640 1 -- 172.21.15.64:0/2241231972 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff70802e030 con 0x7ff710106f50 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1055395875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9838b40 msgr2=0x5571b9837090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9838b40 0x5571b9837090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1055395875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b98375d0 msgr2=0x5571b9837a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b98375d0 0x5571b9837a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.937+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1055395875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b97293a0 con 0x5571b9846780 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b9848b70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8012520 tx=0x7ff6f800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e034f19ee6f5ab6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff70ed9d640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7080eeac0 tx=0x7ff7080dd040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff704ff9640 1 -- 172.21.15.64:0/1055395875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f801e640 con 0x5571b9846780 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff704ff9640 1 -- 172.21.15.64:0/1055395875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6f80108e0 con 0x5571b9846780 2024-09-19T05:16:29.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff704ff9640 1 -- 172.21.15.64:0/1055395875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f801d860 con 0x5571b9846780 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff717341740 1 -- 172.21.15.64:0/1055395875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 msgr2=0x5571b9848b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.938+0000 7ff717341740 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b9848b70 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8012520 tx=0x7ff6f800b6e0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- 172.21.15.64:0/1055395875 shutdown_connections 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b9848b70 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b98375d0 0x5571b9837a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 --2- 172.21.15.64:0/1055395875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9838b40 0x5571b9837090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- 172.21.15.64:0/1055395875 >> 172.21.15.64:0/1055395875 conn(0x5571b9800310 msgr2=0x5571b9833c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- 172.21.15.64:0/1055395875 shutdown_connections 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- 172.21.15.64:0/1055395875 wait complete. 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- start start 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 Processor -- start 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9846180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9843430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 0x5571b9843a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5571b9849480 con 0x5571b98375d0 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5571b9758060 con 0x5571b9838b40 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5571b9849600 con 0x5571b9846780 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.939+0000 7ff717341740 10 client.0 _ll_register_callbacks cb 0x5571b94ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9846180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9846180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52468/0 (socket says 172.21.15.64:52468) 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9843430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2898705270 learned_addr learned my addr 172.21.15.64:0/2898705270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2898705270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 msgr2=0x5571b9843a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 0x5571b9843a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 msgr2=0x5571b9843430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9843430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2898705270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b950e3c0 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70e59c640 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9843430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9846180 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b440 tx=0x7ff6f8010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ef474746a6a9ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/2898705270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8010d50 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f80108e0 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/2898705270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff6f8004040 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.940+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/2898705270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6f8035050 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 10 client.17913 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571b9854f70 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/2898705270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6f80108e0 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/2898705270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6f8030030 con 0x5571b98375d0 2024-09-19T05:16:29.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 10 client.17913 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 10 client.17913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:16:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 20 client.17913 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98567e0 0x5571b9858c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 4 client.17913 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:16:29.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b97aa860 con 0x5571b98567e0 2024-09-19T05:16:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.941+0000 7ff70e59c640 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98567e0 0x5571b9858c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff70e59c640 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98567e0 0x5571b9858c20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164402dcbd204360 server_cookie=5228df792a2cded9 in_seq=0 out_seq=0 2024-09-19T05:16:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff6f5ffb640 10 client.17913 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:29.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/2898705270 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571b97aa860 con 0x5571b98567e0 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff6f5ffb640 10 client.17913 handle_command_reply: tid=0 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff717341740 1 client.17913 shutdown 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 msgr2=0x5571b9846180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.942+0000 7ff717341740 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9846180 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b440 tx=0x7ff6f8010430 comp rx=0 tx=0).stop 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 shutdown_connections 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 0x5571b9843a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9843430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98567e0 0x5571b9858c20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 --2- 172.21.15.64:0/2898705270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9846180 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 >> 172.21.15.64:0/2898705270 conn(0x5571b9800310 msgr2=0x5571b9839960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff6f5ffb640 0 client.17913 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 shutdown_connections 2024-09-19T05:16:29.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 1 -- 172.21.15.64:0/2898705270 wait complete. 2024-09-19T05:16:29.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.943+0000 7ff717341740 20 client.17913 trim_cache size 0 max 16384 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 Processor -- start 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 -- start start 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9852720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9842cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 0x5571b9833700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7101adee0 con 0x5571b98375d0 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7101ab7b0 con 0x5571b9838b40 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff717341740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7101a9620 con 0x5571b9846780 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9852720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9852720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52472/0 (socket says 172.21.15.64:52472) 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2234726075 learned_addr learned my addr 172.21.15.64:0/2234726075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70e59c640 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9842cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2234726075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 msgr2=0x5571b9833700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 0x5571b9833700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2234726075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 msgr2=0x5571b9842cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9842cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 -- 172.21.15.64:0/2234726075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b98372d0 con 0x5571b98375d0 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9852720 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b410 tx=0x7ff6f8004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264f4275a1ecc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.956+0000 7ff6f7fff640 1 -- 172.21.15.64:0/2234726075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f803e070 con 0x5571b98375d0 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff6f7fff640 1 -- 172.21.15.64:0/2234726075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6f8024040 con 0x5571b98375d0 2024-09-19T05:16:29.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff6f7fff640 1 -- 172.21.15.64:0/2234726075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80123b0 con 0x5571b98375d0 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 -- 172.21.15.64:0/2234726075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 msgr2=0x5571b9852720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9852720 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b410 tx=0x7ff6f8004970 comp rx=0 tx=0).stop 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 -- 172.21.15.64:0/2234726075 shutdown_connections 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b9846780 0x5571b9833700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b9842cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 --2- 172.21.15.64:0/2234726075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b98375d0 0x5571b9852720 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 -- 172.21.15.64:0/2234726075 >> 172.21.15.64:0/2234726075 conn(0x5571b9800310 msgr2=0x5571b983e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 -- 172.21.15.64:0/2234726075 shutdown_connections 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 -- 172.21.15.64:0/2234726075 wait complete. 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.957+0000 7ff717341740 1 -- start start 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 Processor -- start 2024-09-19T05:16:29.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b98375d0 0x5571b984ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b984f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b98409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7101a9c70 con 0x5571b9846780 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7101ab0e0 con 0x5571b9838b40 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7101aa340 con 0x5571b98375d0 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff717341740 10 client.0 _ll_register_callbacks cb 0x7ff70000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b98409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b98409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52476/0 (socket says 172.21.15.64:52476) 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1398443550 learned_addr learned my addr 172.21.15.64:0/1398443550 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70e59c640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b984f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b98375d0 0x5571b984ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1398443550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b98375d0 msgr2=0x5571b984ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b98375d0 0x5571b984ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 msgr2=0x5571b984f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b984f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.958+0000 7ff70dd9b640 1 -- 172.21.15.64:0/1398443550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b98411e0 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff70dd9b640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b98409b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff6f801d450 tx=0x7ff6f803b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23b9812e40e2072 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/1398443550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f803e040 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/1398443550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff6f80122c0 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f8024040 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/1398443550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6f8049050 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 10 client.17919 fetch_fsmap learned FSMap version 25 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571b98404e0 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/1398443550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6f8024040 con 0x5571b9846780 2024-09-19T05:16:29.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/1398443550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6f8030070 con 0x5571b9846780 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 10 client.17919 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 20 client.17919 populate_metadata read hostname 'smithi064' 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98c08e0 0x5571b98c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 4 client.17919 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.959+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 --> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571b97a4d70 con 0x5571b98c08e0 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff70e59c640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98c08e0 0x5571b98c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff6f5ffb640 10 client.17919 ms_handle_connect on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff70e59c640 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98c08e0 0x5571b98c2d00 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa3ede60822276d server_cookie=f9cda039ee71d123 in_seq=0 out_seq=0 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff6f5ffb640 1 -- 172.21.15.64:0/1398443550 <== mds.0 v2:172.21.15.103:6838/100524955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571b97a4d70 con 0x5571b98c08e0 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff6f5ffb640 10 client.17919 handle_command_reply: tid=0 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff717341740 1 client.17919 shutdown 2024-09-19T05:16:29.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 msgr2=0x5571b98409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.960+0000 7ff717341740 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b98409b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff6f801d450 tx=0x7ff6f803b040 comp rx=0 tx=0).stop 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 shutdown_connections 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:6838/100524955,v1:172.21.15.103:6839/100524955] conn(0x5571b98c08e0 0x5571b98c2d00 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5571b9846780 0x5571b98409b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5571b9838b40 0x5571b984f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff6f5ffb640 0 client.17919 ms_handle_reset on v2:172.21.15.103:6838/100524955 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 --2- 172.21.15.64:0/1398443550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571b98375d0 0x5571b984ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 >> 172.21.15.64:0/1398443550 conn(0x5571b9800310 msgr2=0x5571b9833680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 shutdown_connections 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 1 -- 172.21.15.64:0/1398443550 wait complete. 2024-09-19T05:16:29.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff717341740 20 client.17919 trim_cache size 0 max 16384 2024-09-19T05:16:29.968 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-19T05:16:29.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff6fc0a33b0 msgr2=0x7ff6fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7080eeac0 tx=0x7ff7080dd040 comp rx=0 tx=0).stop 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 msgr2=0x7ff71019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71019ccb0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff708005cc0 tx=0x7ff708003c20 comp rx=0 tx=0).stop 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 shutdown_connections 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff710106f50 0x7ff71019ccb0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff710100ca0 0x7ff710198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 --2- 172.21.15.64:0/2241231972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7101002f0 0x7ff7101983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 >> 172.21.15.64:0/2241231972 conn(0x7ff710077210 msgr2=0x7ff7101052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 shutdown_connections 2024-09-19T05:16:29.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:29.961+0000 7ff714f68640 1 -- 172.21.15.64:0/2241231972 wait complete. 2024-09-19T05:16:30.733 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T05:16:30.734 INFO:tasks.fwd_scrub:done joining 2024-09-19T05:16:30.734 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T05:16:30.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.849+0000 7fba49c47640 1 -- 172.21.15.64:0/3429302590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba441069d0 msgr2=0x7fba44106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.849+0000 7fba49c47640 1 --2- 172.21.15.64:0/3429302590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba441069d0 0x7fba44106db0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fba34009930 tx=0x7fba3402a420 comp rx=0 tx=0).stop 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- 172.21.15.64:0/3429302590 shutdown_connections 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 --2- 172.21.15.64:0/3429302590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba44101360 0x7fba4410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 --2- 172.21.15.64:0/3429302590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441009c0 0x7fba44100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 --2- 172.21.15.64:0/3429302590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba441069d0 0x7fba44106db0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- 172.21.15.64:0/3429302590 >> 172.21.15.64:0/3429302590 conn(0x7fba440fc6a0 msgr2=0x7fba440feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:30.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- 172.21.15.64:0/3429302590 shutdown_connections 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- 172.21.15.64:0/3429302590 wait complete. 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 Processor -- start 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- start start 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba441009c0 0x7fba44199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 0x7fba44199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441069d0 0x7fba441a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fba44110cc0 con 0x7fba44101360 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba44110b40 con 0x7fba441009c0 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.850+0000 7fba49c47640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba44110e40 con 0x7fba441069d0 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 0x7fba44199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 0x7fba44199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52488/0 (socket says 172.21.15.64:52488) 2024-09-19T05:16:30.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 -- 172.21.15.64:0/2334598414 learned_addr learned my addr 172.21.15.64:0/2334598414 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba437fe640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba441009c0 0x7fba44199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba43fff640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441069d0 0x7fba441a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 -- 172.21.15.64:0/2334598414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441069d0 msgr2=0x7fba441a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441069d0 0x7fba441a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 -- 172.21.15.64:0/2334598414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba441009c0 msgr2=0x7fba44199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba441009c0 0x7fba44199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 -- 172.21.15.64:0/2334598414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba441a47d0 con 0x7fba44101360 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba437fe640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba441009c0 0x7fba44199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba43fff640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441069d0 0x7fba441a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba3affd640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 0x7fba44199750 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00d770 tx=0x7fba2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c017070 con 0x7fba44101360 2024-09-19T05:16:30.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba2c0054d0 con 0x7fba44101360 2024-09-19T05:16:30.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba441a49a0 con 0x7fba44101360 2024-09-19T05:16:30.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c004cf0 con 0x7fba44101360 2024-09-19T05:16:30.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.851+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba441a4e70 con 0x7fba44101360 2024-09-19T05:16:30.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.852+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba44100e20 con 0x7fba44101360 2024-09-19T05:16:30.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.854+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fba2c004e90 con 0x7fba44101360 2024-09-19T05:16:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.854+0000 7fba417fa640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba300a33c0 0x7fba300a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.854+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fba2c0cfc50 con 0x7fba44101360 2024-09-19T05:16:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.854+0000 7fba437fe640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba300a33c0 0x7fba300a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.855+0000 7fba437fe640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba300a33c0 0x7fba300a5880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c00 tx=0x7fba34009610 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:30.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:30.857+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba2c09c600 con 0x7fba44101360 2024-09-19T05:16:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.035+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba44108f30 con 0x7fba44101360 2024-09-19T05:16:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.037+0000 7fba417fa640 1 -- 172.21.15.64:0/2334598414 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23288 (secure 0 0 0) 0x7fba2c09bf00 con 0x7fba44101360 2024-09-19T05:16:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:16:31.039 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:16:17.102921+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:6800/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/868126904":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/2000236557":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/2343862077":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/1076683694":"2024-09-20T05:05:53.680120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:16:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba300a33c0 msgr2=0x7fba300a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba300a33c0 0x7fba300a5880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c00 tx=0x7fba34009610 comp rx=0 tx=0).stop 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 msgr2=0x7fba44199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 0x7fba44199750 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00d770 tx=0x7fba2c00dc30 comp rx=0 tx=0).stop 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 shutdown_connections 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fba300a33c0 0x7fba300a5880 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba441069d0 0x7fba441a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba44101360 0x7fba44199750 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 --2- 172.21.15.64:0/2334598414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba441009c0 0x7fba44199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 >> 172.21.15.64:0/2334598414 conn(0x7fba440fc6a0 msgr2=0x7fba440fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 shutdown_connections 2024-09-19T05:16:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.039+0000 7fba49c47640 1 -- 172.21.15.64:0/2334598414 wait complete. 2024-09-19T05:16:31.052 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:59:34.802774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:04:04.880203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:04:06.925227+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:16:31.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 -- 172.21.15.64:0/84629367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490810c110 msgr2=0x7f490810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 --2- 172.21.15.64:0/84629367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490810c110 0x7f490810e560 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4900009ca0 tx=0x7f490002a950 comp rx=0 tx=0).stop 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 -- 172.21.15.64:0/84629367 shutdown_connections 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 --2- 172.21.15.64:0/84629367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f490810c110 0x7f490810e560 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 --2- 172.21.15.64:0/84629367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908103660 0x7f490810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 --2- 172.21.15.64:0/84629367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4908102d40 0x7f4908103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 -- 172.21.15.64:0/84629367 >> 172.21.15.64:0/84629367 conn(0x7f49080fc690 msgr2=0x7f49080feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 -- 172.21.15.64:0/84629367 shutdown_connections 2024-09-19T05:16:31.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.157+0000 7f490da2e640 1 -- 172.21.15.64:0/84629367 wait complete. 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 Processor -- start 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 -- start start 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 0x7f4908100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908103660 0x7f4908100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490810c110 0x7f4908101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111db0 con 0x7f4908102d40 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111c30 con 0x7f490810c110 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f490da2e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111f30 con 0x7f4908103660 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f4906ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 0x7f4908100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f4906ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 0x7f4908100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52504/0 (socket says 172.21.15.64:52504) 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.158+0000 7f4906ffd640 1 -- 172.21.15.64:0/1209341447 learned_addr learned my addr 172.21.15.64:0/1209341447 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f49077fe640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490810c110 0x7f4908101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f49067fc640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908103660 0x7f4908100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f4906ffd640 1 -- 172.21.15.64:0/1209341447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908103660 msgr2=0x7f4908100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f4906ffd640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908103660 0x7f4908100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f4906ffd640 1 -- 172.21.15.64:0/1209341447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490810c110 msgr2=0x7f4908101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f4906ffd640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490810c110 0x7f4908101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f4906ffd640 1 -- 172.21.15.64:0/1209341447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49081a6760 con 0x7f4908102d40 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f4906ffd640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 0x7f4908100990 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f48f800d950 tx=0x7f48f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8017030 con 0x7f4908102d40 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49081a6a50 con 0x7f4908102d40 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48f8004020 con 0x7f4908102d40 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80047a0 con 0x7f4908102d40 2024-09-19T05:16:31.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.159+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a6ff0 con 0x7f4908102d40 2024-09-19T05:16:31.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.160+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4908103120 con 0x7f4908102d40 2024-09-19T05:16:31.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.162+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f48f80041d0 con 0x7f4908102d40 2024-09-19T05:16:31.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.162+0000 7f490ca2c640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f48f40a33c0 0x7f48f40a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.162+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f48f80cf8f0 con 0x7f4908102d40 2024-09-19T05:16:31.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.162+0000 7f49067fc640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f48f40a33c0 0x7f48f40a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.163+0000 7f49067fc640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f48f40a33c0 0x7f48f40a5880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4908102420 tx=0x7f48f000c4a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:31.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.167+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f48f809c240 con 0x7f4908102d40 2024-09-19T05:16:31.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.343+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49081071c0 con 0x7f4908102d40 2024-09-19T05:16:31.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.343+0000 7f490ca2c640 1 -- 172.21.15.64:0/1209341447 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f48f809bb40 con 0x7f4908102d40 2024-09-19T05:16:31.345 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f48f40a33c0 msgr2=0x7f48f40a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f48f40a33c0 0x7f48f40a5880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4908102420 tx=0x7f48f000c4a0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 msgr2=0x7f4908100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 0x7f4908100990 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f48f800d950 tx=0x7f48f800de10 comp rx=0 tx=0).stop 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 shutdown_connections 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f48f40a33c0 0x7f48f40a5880 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f490810c110 0x7f4908101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908103660 0x7f4908100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 --2- 172.21.15.64:0/1209341447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4908102d40 0x7f4908100990 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 >> 172.21.15.64:0/1209341447 conn(0x7f49080fc690 msgr2=0x7f49080fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 shutdown_connections 2024-09-19T05:16:31.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.345+0000 7f490da2e640 1 -- 172.21.15.64:0/1209341447 wait complete. 2024-09-19T05:16:31.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T05:16:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 -- 172.21.15.64:0/4272417977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc10bfc0 msgr2=0x7feecc10e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 --2- 172.21.15.64:0/4272417977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc10bfc0 0x7feecc10e410 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7feec4009ca0 tx=0x7feec402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 -- 172.21.15.64:0/4272417977 shutdown_connections 2024-09-19T05:16:31.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 --2- 172.21.15.64:0/4272417977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc10bfc0 0x7feecc10e410 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 --2- 172.21.15.64:0/4272417977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc103470 0x7feecc10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 --2- 172.21.15.64:0/4272417977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc102ac0 0x7feecc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 -- 172.21.15.64:0/4272417977 >> 172.21.15.64:0/4272417977 conn(0x7feecc0fc540 msgr2=0x7feecc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 -- 172.21.15.64:0/4272417977 shutdown_connections 2024-09-19T05:16:31.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.463+0000 7feed0965640 1 -- 172.21.15.64:0/4272417977 wait complete. 2024-09-19T05:16:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.464+0000 7feed0965640 1 Processor -- start 2024-09-19T05:16:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.464+0000 7feed0965640 1 -- start start 2024-09-19T05:16:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc102ac0 0x7feecc198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 0x7feecc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc10bfc0 0x7feecc19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feecc111c20 con 0x7feecc103470 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feecc111aa0 con 0x7feecc10bfc0 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feecc111da0 con 0x7feecc102ac0 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 0x7feecc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 0x7feecc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52530/0 (socket says 172.21.15.64:52530) 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 -- 172.21.15.64:0/2234490610 learned_addr learned my addr 172.21.15.64:0/2234490610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feecb59e640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc10bfc0 0x7feecc19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feecad9d640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc102ac0 0x7feecc198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 -- 172.21.15.64:0/2234490610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc102ac0 msgr2=0x7feecc198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc102ac0 0x7feecc198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 -- 172.21.15.64:0/2234490610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc10bfc0 msgr2=0x7feecc19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc10bfc0 0x7feecc19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 -- 172.21.15.64:0/2234490610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc19d140 con 0x7feecc103470 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feecad9d640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc102ac0 0x7feecc198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeca59c640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 0x7feecc198690 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7feec000d770 tx=0x7feec000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0017070 con 0x7feecc103470 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feec00054d0 con 0x7feecc103470 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc19d430 con 0x7feecc103470 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0004cf0 con 0x7feecc103470 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.465+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc1a4cc0 con 0x7feecc103470 2024-09-19T05:16:31.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.466+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc102ea0 con 0x7feecc103470 2024-09-19T05:16:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.468+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7feec0004e90 con 0x7feecc103470 2024-09-19T05:16:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.469+0000 7feeb3fff640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7feeb40a3440 0x7feeb40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.469+0000 7feecad9d640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7feeb40a3440 0x7feeb40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.469+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7feec00cfae0 con 0x7feecc103470 2024-09-19T05:16:31.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.469+0000 7feecad9d640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7feeb40a3440 0x7feeb40a5900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feebc009b40 tx=0x7feebc005e50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:31.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.472+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feec009c4b0 con 0x7feecc103470 2024-09-19T05:16:31.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.648+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feecc107020 con 0x7feecc103470 2024-09-19T05:16:31.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.648+0000 7feeb3fff640 1 -- 172.21.15.64:0/2234490610 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7feec009bdb0 con 0x7feecc103470 2024-09-19T05:16:31.650 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-09-19T05:16:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:31 smithi064 ceph-mon[30960]: pgmap v329: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:31 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2334598414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:16:31.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:31 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1209341447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7feeb40a3440 msgr2=0x7feeb40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7feeb40a3440 0x7feeb40a5900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feebc009b40 tx=0x7feebc005e50 comp rx=0 tx=0).stop 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 msgr2=0x7feecc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 0x7feecc198690 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7feec000d770 tx=0x7feec000dc30 comp rx=0 tx=0).stop 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 shutdown_connections 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7feeb40a3440 0x7feeb40a5900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc10bfc0 0x7feecc19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feecc103470 0x7feecc198690 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 --2- 172.21.15.64:0/2234490610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feecc102ac0 0x7feecc198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 >> 172.21.15.64:0/2234490610 conn(0x7feecc0fc540 msgr2=0x7feecc0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 shutdown_connections 2024-09-19T05:16:31.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.650+0000 7feed0965640 1 -- 172.21.15.64:0/2234490610 wait complete. 2024-09-19T05:16:31.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T05:16:31.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:31 smithi103 ceph-mon[34446]: pgmap v329: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:31.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:31 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2334598414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:16:31.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:31 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1209341447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:16:31.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 -- 172.21.15.64:0/2169141272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c107aa0 msgr2=0x7f046c107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 --2- 172.21.15.64:0/2169141272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c107aa0 0x7f046c107e80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f045c009930 tx=0x7f045c02a420 comp rx=0 tx=0).stop 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 -- 172.21.15.64:0/2169141272 shutdown_connections 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 --2- 172.21.15.64:0/2169141272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c102430 0x7f046c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 --2- 172.21.15.64:0/2169141272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c101a90 0x7f046c101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 --2- 172.21.15.64:0/2169141272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c107aa0 0x7f046c107e80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 -- 172.21.15.64:0/2169141272 >> 172.21.15.64:0/2169141272 conn(0x7f046c0fd7d0 msgr2=0x7f046c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 -- 172.21.15.64:0/2169141272 shutdown_connections 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.767+0000 7f04723cd640 1 -- 172.21.15.64:0/2169141272 wait complete. 2024-09-19T05:16:31.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 Processor -- start 2024-09-19T05:16:31.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 -- start start 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 0x7f046c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c102430 0x7f046c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c107aa0 0x7f046c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f046c111d50 con 0x7f046c101a90 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f046c111bd0 con 0x7f046c102430 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f04723cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f046c111ed0 con 0x7f046c107aa0 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f046bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 0x7f046c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f046bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 0x7f046c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52550/0 (socket says 172.21.15.64:52550) 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f046bfff640 1 -- 172.21.15.64:0/4289851207 learned_addr learned my addr 172.21.15.64:0/4289851207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.768+0000 7f046b7fe640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c102430 0x7f046c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f047096a640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c107aa0 0x7f046c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f046bfff640 1 -- 172.21.15.64:0/4289851207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c107aa0 msgr2=0x7f046c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f046bfff640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c107aa0 0x7f046c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f046bfff640 1 -- 172.21.15.64:0/4289851207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c102430 msgr2=0x7f046c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f046bfff640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c102430 0x7f046c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f046bfff640 1 -- 172.21.15.64:0/4289851207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c19d2d0 con 0x7f046c101a90 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f047096a640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c107aa0 0x7f046c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f046bfff640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 0x7f046c1982e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f045c004770 tx=0x7f045c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c03f070 con 0x7f046c101a90 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f045c03b460 con 0x7f046c101a90 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c19d560 con 0x7f046c101a90 2024-09-19T05:16:31.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c035070 con 0x7f046c101a90 2024-09-19T05:16:31.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.769+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1a4e50 con 0x7f046c101a90 2024-09-19T05:16:31.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.770+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c101ef0 con 0x7f046c101a90 2024-09-19T05:16:31.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.772+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f045c03b610 con 0x7f046c101a90 2024-09-19T05:16:31.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.773+0000 7f04697fa640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04540a3440 0x7f04540a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:31.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.773+0000 7f046b7fe640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04540a3440 0x7f04540a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:31.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.773+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f045c0f0320 con 0x7f046c101a90 2024-09-19T05:16:31.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.773+0000 7f046b7fe640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04540a3440 0x7f04540a5900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f046c0ffe50 tx=0x7f046000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:31.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.776+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f045c0bccd0 con 0x7f046c101a90 2024-09-19T05:16:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:31 smithi174 ceph-mon[34488]: pgmap v329: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:31.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:31 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2334598414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:16:31.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:31 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1209341447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:16:31.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.951+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f046c10a030 con 0x7f046c101a90 2024-09-19T05:16:31.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.952+0000 7f04697fa640 1 -- 172.21.15.64:0/4289851207 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f045c0bc5d0 con 0x7f046c101a90 2024-09-19T05:16:31.954 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-09-19T05:16:31.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04540a3440 msgr2=0x7f04540a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04540a3440 0x7f04540a5900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f046c0ffe50 tx=0x7f046000b390 comp rx=0 tx=0).stop 2024-09-19T05:16:31.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 msgr2=0x7f046c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 0x7f046c1982e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f045c004770 tx=0x7f045c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 shutdown_connections 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f04540a3440 0x7f04540a5900 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c107aa0 0x7f046c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c102430 0x7f046c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 --2- 172.21.15.64:0/4289851207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f046c101a90 0x7f046c1982e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 >> 172.21.15.64:0/4289851207 conn(0x7f046c0fd7d0 msgr2=0x7f046c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:31.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 shutdown_connections 2024-09-19T05:16:31.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:31.954+0000 7f04723cd640 1 -- 172.21.15.64:0/4289851207 wait complete. 2024-09-19T05:16:31.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T05:16:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/1227454270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c0810c0e0 msgr2=0x7f2c0810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/1227454270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c0810c0e0 0x7f2c0810e560 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009ca0 tx=0x7f2c0002a950 comp rx=0 tx=0).stop 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/1227454270 shutdown_connections 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/1227454270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c0810c0e0 0x7f2c0810e560 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/1227454270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c081035b0 0x7f2c0810ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/1227454270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c08102c90 0x7f2c08103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/1227454270 >> 172.21.15.64:0/1227454270 conn(0x7f2c080fc6a0 msgr2=0x7f2c080feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/1227454270 shutdown_connections 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/1227454270 wait complete. 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.071+0000 7f2c0d5e4640 1 Processor -- start 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 -- start start 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 0x7f2c081982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c081035b0 0x7f2c08198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c0810c0e0 0x7f2c0819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08111d70 con 0x7f2c08102c90 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08111bf0 con 0x7f2c081035b0 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c0d5e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08111ef0 con 0x7f2c0810c0e0 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 0x7f2c081982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 0x7f2c081982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52566/0 (socket says 172.21.15.64:52566) 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 -- 172.21.15.64:0/4023916327 learned_addr learned my addr 172.21.15.64:0/4023916327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c077fe640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c0810c0e0 0x7f2c0819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c067fc640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c081035b0 0x7f2c08198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 -- 172.21.15.64:0/4023916327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c0810c0e0 msgr2=0x7f2c0819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c0810c0e0 0x7f2c0819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 -- 172.21.15.64:0/4023916327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c081035b0 msgr2=0x7f2c08198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c081035b0 0x7f2c08198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:32.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 -- 172.21.15.64:0/4023916327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0819d2b0 con 0x7f2c08102c90 2024-09-19T05:16:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2c06ffd640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 0x7f2c081982c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800d770 tx=0x7f2bf800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.072+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8017070 con 0x7f2c08102c90 2024-09-19T05:16:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.073+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0819d5a0 con 0x7f2c08102c90 2024-09-19T05:16:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.073+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bf80054d0 con 0x7f2c08102c90 2024-09-19T05:16:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.073+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8004cf0 con 0x7f2c08102c90 2024-09-19T05:16:32.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.073+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c081a4e30 con 0x7f2c08102c90 2024-09-19T05:16:32.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.074+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c08103070 con 0x7f2c08102c90 2024-09-19T05:16:32.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.075+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2bf8004e90 con 0x7f2c08102c90 2024-09-19T05:16:32.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.076+0000 7f2beffff640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2bf00a33c0 0x7f2bf00a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:32.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.076+0000 7f2c067fc640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2bf00a33c0 0x7f2bf00a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:32.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.076+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f2bf80cfba0 con 0x7f2c08102c90 2024-09-19T05:16:32.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.076+0000 7f2c067fc640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2bf00a33c0 0x7f2bf00a5880 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc005fd0 tx=0x7f2bfc005f60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:32.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.079+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bf809c550 con 0x7f2c08102c90 2024-09-19T05:16:32.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.303+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2c08107130 con 0x7f2c08102c90 2024-09-19T05:16:32.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.311+0000 7f2beffff640 1 -- 172.21.15.64:0/4023916327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+742869 (secure 0 0 0) 0x7f2bf809be50 con 0x7f2c08102c90 2024-09-19T05:16:32.325 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:16:32.325 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:16:32.325 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 25, 2024-09-19T05:16:32.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.325 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 15126, 2024-09-19T05:16:32.325 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "x", 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6800", 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2647680663 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6801", 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2647680663 2024-09-19T05:16:32.326 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6801/2647680663", 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2024-09-19T05:05:53.680376+0000", 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24763, 2024-09-19T05:16:32.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "z", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540701547738038271, 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.332 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.333 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.334 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.336 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-19T05:16:32.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-19T05:16:32.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-19T05:16:32.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-19T05:16:32.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.343 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.344 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.346 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-19T05:16:32.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.348 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.353 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.354 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.356 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.357 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:16:32.360 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.361 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:32.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.368 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-19T05:16:32.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-19T05:16:32.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:16:32.372 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.374 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.377 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:16:32.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:16:32.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.380 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.382 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.389 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-19T05:16:32.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:16:32.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-19T05:16:32.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.396 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:16:32.398 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:16:32.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:16:32.409 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.411 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-19T05:16:32.419 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:16:32.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.426 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-19T05:16:32.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.432 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.436 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.437 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-19T05:16:32.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:16:32.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:16:32.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.446 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:16:32.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:16:32.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:16:32.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:16:32.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:16:32.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.469 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.470 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.482 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:16:32.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:16:32.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.494 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.495 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.499 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.501 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.502 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:32.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.508 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.509 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-19T05:16:32.511 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-19T05:16:32.512 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.514 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.516 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-19T05:16:32.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.526 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-19T05:16:32.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-19T05:16:32.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.526 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.526 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.530 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-19T05:16:32.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-19T05:16:32.530 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.530 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.532 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.532 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.532 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.536 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.537 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.538 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.542 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.548 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.554 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:32.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.561 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.564 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.566 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.567 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.575 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.578 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-19T05:16:32.580 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-19T05:16:32.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:16:32.588 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.591 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.592 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.593 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.596 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.598 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.599 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-19T05:16:32.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.611 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.612 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-19T05:16:32.617 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.618 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.619 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.625 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.629 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-19T05:16:32.637 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.641 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.643 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.644 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.653 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.656 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-19T05:16:32.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-19T05:16:32.666 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.667 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.670 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.679 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.681 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.687 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.688 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.692 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.699 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.701 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.702 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.708 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-19T05:16:32.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.714 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.717 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-19T05:16:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.724 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.726 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.731 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.732 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.733 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.735 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.737 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.740 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.741 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.743 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.745 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.747 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.750 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.752 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.753 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-19T05:16:32.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-19T05:16:32.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-19T05:16:32.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.763 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.764 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.768 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.771 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:16:32.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.776 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:16:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-19T05:16:32.792 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.796 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.797 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.799 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.804 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24872, 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "y", 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540701547738038271, 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.805 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.807 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.810 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.811 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.811 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.813 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.813 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.814 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-19T05:16:32.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-19T05:16:32.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:16:32.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.816 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-19T05:16:32.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-19T05:16:32.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-19T05:16:32.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-19T05:16:32.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-19T05:16:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.820 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.821 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-19T05:16:32.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-19T05:16:32.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-19T05:16:32.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.823 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-19T05:16:32.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-19T05:16:32.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.824 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-19T05:16:32.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-19T05:16:32.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.825 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-19T05:16:32.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-19T05:16:32.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:16:32.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.828 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.828 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-19T05:16:32.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-19T05:16:32.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.829 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.829 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:16:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.830 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-19T05:16:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-19T05:16:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.831 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-19T05:16:32.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.833 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.833 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.833 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-19T05:16:32.833 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.834 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.834 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.834 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-19T05:16:32.834 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-19T05:16:32.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.835 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.835 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.836 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-19T05:16:32.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-19T05:16:32.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-19T05:16:32.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:16:32.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:16:32.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.839 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-19T05:16:32.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-19T05:16:32.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-19T05:16:32.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-19T05:16:32.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-19T05:16:32.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:16:32.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.842 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-19T05:16:32.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:32.842 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-19T05:16:32.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:16:32.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.844 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-19T05:16:32.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-19T05:16:32.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:16:32.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:16:32.845 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:16:32.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.846 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-19T05:16:32.846 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-19T05:16:32.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.846 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:16:32.847 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:16:32.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.848 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-19T05:16:32.848 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-19T05:16:32.848 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-19T05:16:32.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-19T05:16:32.850 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-19T05:16:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-19T05:16:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.852 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.853 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.853 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.853 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.853 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.854 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.855 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.855 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.855 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.855 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.856 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.856 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.858 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.859 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.860 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.861 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.861 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.862 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.862 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.862 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.862 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-19T05:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-19T05:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.866 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-19T05:16:32.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-19T05:16:32.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-19T05:16:32.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.867 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-19T05:16:32.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.869 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.869 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-19T05:16:32.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-19T05:16:32.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:32.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.873 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:16:32.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.873 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:32.874 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.875 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-19T05:16:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:16:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.878 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-19T05:16:32.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-19T05:16:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-19T05:16:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-19T05:16:32.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.884 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.885 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-19T05:16:32.888 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:16:32.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:16:32.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-19T05:16:32.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.900 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.901 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.902 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.903 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:16:32.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:16:32.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.913 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.915 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-19T05:16:32.916 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-19T05:16:32.916 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:16:32.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.926 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.927 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-19T05:16:32.933 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-19T05:16:32.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:16:32.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.939 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:16:32.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-19T05:16:32.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-19T05:16:32.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-19T05:16:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-19T05:16:32.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-19T05:16:32.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.946 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.947 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:32.952 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:32.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-19T05:16:32.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-19T05:16:32.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.956 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:16:32.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:16:32.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:16:32.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:16:32.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.962 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:32.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:16:32.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:16:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:16:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:32.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:32.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.001 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-19T05:16:33.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-19T05:16:33.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.008 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.010 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-19T05:16:33.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:16:33.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:33.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.015 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.016 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-19T05:16:33.020 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.024 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.025 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:16:33.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:16:33.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:16:33.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.036 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.037 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.039 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.041 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.044 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-19T05:16:33.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-19T05:16:33.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.048 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.049 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.050 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.052 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.058 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.062 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.064 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.071 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.072 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.078 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.085 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-19T05:16:33.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-19T05:16:33.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.093 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.096 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.098 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.099 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.111 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.114 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.118 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.119 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.120 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.121 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.123 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.124 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.129 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.132 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-19T05:16:33.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.136 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.138 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.139 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.140 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.146 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.149 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.152 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.154 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.162 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.166 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.170 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.171 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.174 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.175 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.181 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.185 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.198 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.201 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.202 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.203 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-19T05:16:33.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-19T05:16:33.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-19T05:16:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-19T05:16:33.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.217 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.219 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.221 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.225 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.226 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-19T05:16:33.231 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.235 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.237 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.243 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-19T05:16:33.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-19T05:16:33.246 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-19T05:16:33.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.248 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.249 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.254 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.255 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.256 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.263 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-19T05:16:33.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-19T05:16:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.278 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.278 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.282 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:16:33.284 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.285 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.286 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:16:33.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.287 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.287 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.287 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-19T05:16:33.288 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.288 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.288 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.288 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-19T05:16:33.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.289 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.293 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.295 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.296 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.298 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.300 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.303 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-19T05:16:33.304 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard", 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "restful", 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "snap_schedule" 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.306 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.307 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.310 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.313 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-19T05:16:33.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.315 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-19T05:16:33.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.320 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.321 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-19T05:16:33.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-19T05:16:33.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.325 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:16:33.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:33.327 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.329 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.331 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.332 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.334 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.335 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.336 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.340 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.342 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:16:33.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.343 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.345 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:16:33.345 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.347 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.348 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:16:33.349 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.352 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:16:33.354 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:16:33.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.354 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:16:33.356 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-19T05:16:33.358 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-19T05:16:33.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:16:33.365 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.366 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-19T05:16:33.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-19T05:16:33.371 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:16:33.374 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-19T05:16:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-19T05:16:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.377 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:16:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-19T05:16:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.378 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:16:33.381 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.382 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.383 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:16:33.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:16:33.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.387 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-19T05:16:33.387 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-19T05:16:33.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.388 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-19T05:16:33.388 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.390 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-19T05:16:33.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-19T05:16:33.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.391 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:16:33.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.393 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-19T05:16:33.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.405 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-19T05:16:33.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-19T05:16:33.411 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.412 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:16:33.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.414 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.415 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:16:33.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.419 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:16:33.419 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.419 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.419 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.422 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:16:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-19T05:16:33.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.428 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.429 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.434 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:16:33.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:16:33.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:16:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:16:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:16:33.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:16:33.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:16:33.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:16:33.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.450 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:16:33.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.457 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:33.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:16:33.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.471 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:16:33.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:16:33.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.482 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:16:33.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.484 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.488 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:16:33.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:16:33.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:16:33.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.491 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.498 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.499 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.499 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.499 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.499 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.499 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.500 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.502 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.504 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.507 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-19T05:16:33.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.513 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.514 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.520 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-19T05:16:33.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:16:33.528 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:16:33.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.534 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.535 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.537 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.538 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-19T05:16:33.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.541 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.544 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-19T05:16:33.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.548 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:16:33.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.550 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.552 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.553 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.556 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-19T05:16:33.558 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-19T05:16:33.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:16:33.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.564 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.569 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.570 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.572 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.573 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.574 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.576 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.579 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.580 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.580 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-19T05:16:33.580 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.580 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.586 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.588 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.589 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.590 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-19T05:16:33.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-19T05:16:33.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.591 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.592 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.594 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-19T05:16:33.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:16:33.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.604 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.605 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.607 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.608 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.618 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.623 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.628 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.630 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.631 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.637 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.640 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.640 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-19T05:16:33.640 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.641 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.641 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.641 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.642 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.643 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.643 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.643 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.643 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.643 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.643 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.648 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.649 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.652 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.657 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-19T05:16:33.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.661 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-19T05:16:33.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.673 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-19T05:16:33.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-19T05:16:33.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-19T05:16:33.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-19T05:16:33.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-19T05:16:33.689 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-19T05:16:33.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-19T05:16:33.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.700 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.701 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.710 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.713 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.714 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.716 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-19T05:16:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.728 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.730 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.732 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.733 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.735 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-19T05:16:33.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.738 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.742 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.744 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-19T05:16:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.750 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-19T05:16:33.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.756 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.757 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.762 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.763 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.766 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.770 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.772 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.773 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.775 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.777 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.785 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.789 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-19T05:16:33.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-19T05:16:33.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.806 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-19T05:16:33.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.809 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.810 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.813 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.816 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.819 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.821 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.823 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-19T05:16:33.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.828 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-19T05:16:33.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-19T05:16:33.830 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-19T05:16:33.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.834 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.835 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-19T05:16:33.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.837 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-19T05:16:33.838 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-19T05:16:33.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-19T05:16:33.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-19T05:16:33.843 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.844 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "services": { 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard": "https://172.21.15.64:8443/" 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-19T05:16:33.845 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-19T05:16:33.846 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-19T05:16:33.847 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-19T05:16:33.848 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "squid": [ 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-19T05:16:33.849 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 80, 2024-09-19T05:16:33.850 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 42035407 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 735966271 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:33.852 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3751469224 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:33.853 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2768532076 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1678101610 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 881527544 2024-09-19T05:16:33.855 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.856 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.856 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:33.856 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-19T05:16:33.856 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:16:33.858 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:32 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2234490610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:16:33.858 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:32 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4289851207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:16:33.858 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:32 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4023916327' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:16:33.858 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:33 smithi174 ceph-mon[34488]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:16:33.858 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:32 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2234490610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:16:33.858 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:32 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4289851207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:16:33.859 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:32 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4023916327' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:16:33.859 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:33 smithi103 ceph-mon[34446]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:16:33.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2bf00a33c0 msgr2=0x7f2bf00a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2bf00a33c0 0x7f2bf00a5880 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc005fd0 tx=0x7f2bfc005f60 comp rx=0 tx=0).stop 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 msgr2=0x7f2c081982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 0x7f2c081982c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800d770 tx=0x7f2bf800dc30 comp rx=0 tx=0).stop 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 shutdown_connections 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2bf00a33c0 0x7f2bf00a5880 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c0810c0e0 0x7f2c0819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c081035b0 0x7f2c08198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 --2- 172.21.15.64:0/4023916327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102c90 0x7f2c081982c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 >> 172.21.15.64:0/4023916327 conn(0x7f2c080fc6a0 msgr2=0x7f2c080fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 shutdown_connections 2024-09-19T05:16:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:32.317+0000 7f2c0d5e4640 1 -- 172.21.15.64:0/4023916327 wait complete. 2024-09-19T05:16:33.922 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T05:16:33.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T05:16:33.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:32 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2234490610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:16:33.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:32 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4289851207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:16:33.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:32 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4023916327' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:16:33.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:33 smithi064 ceph-mon[30960]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:16:34.191 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:34.686 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001380922, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T05:16:34.687 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000214652, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 156, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 184624, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.013856627, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.009610484, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002037573, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:34.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 153, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.012035224, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000078661 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 184624 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.690 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 218, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 206, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1449312, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 125792, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.040495394, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.014396291, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.037475659, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 206, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.015018139, 2024-09-19T05:16:34.691 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000072903 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1449312, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 125792 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-19T05:16:34.692 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T05:16:34.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:34.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:34.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:34.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148994, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-19T05:16:34.697 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T05:16:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 216, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1416146, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 216, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T05:16:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 216, 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1416146, 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:34.700 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:16:35.409 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-19T05:16:35.409 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-19T05:16:35.409 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-19T05:16:35.409 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-19T05:16:35.409 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-19T05:16:35.409 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T05:16:35.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T05:16:35.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:35 smithi064 ceph-mon[30960]: pgmap v331: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:16:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:35.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:35 smithi103 ceph-mon[34446]: pgmap v331: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:16:35.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:35 smithi174 ceph-mon[34488]: pgmap v331: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T05:16:36.135 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.003436788, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000056945, 2024-09-19T05:16:36.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002299556, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.004212565, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001404188 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T05:16:36.137 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 219, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 209, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1447680, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 128544, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.040184854, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.015309243, 2024-09-19T05:16:36.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.036538956, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 208, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.011550673, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000055532 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1447680, 2024-09-19T05:16:36.139 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 128544 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 161, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 185504, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.013553240, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.008903046, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002236654, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:36.140 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 158, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.010985344, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000069527 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 185504 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T05:16:36.141 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:36.142 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:36.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:36.144 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148977, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:36.145 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T05:16:36.146 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 221, 2024-09-19T05:16:36.147 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1416796, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 221, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 221, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1416796, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:36.148 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:16:36.782 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T05:16:36.782 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T05:16:36.782 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T05:16:36.782 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T05:16:36.782 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T05:16:36.782 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T05:16:36.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T05:16:37.039 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:37.425 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:37 smithi064 ceph-mon[30960]: pgmap v332: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:37.528 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:16:37.528 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:37.528 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 220, 2024-09-19T05:16:37.528 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 208, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1449856, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 127072, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.044742024, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.014385624, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.041537583, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 208, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.017111692, 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000082267 2024-09-19T05:16:37.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1449856, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 127072 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 158, 2024-09-19T05:16:37.530 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 184976, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.013623942, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.009434409, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001964467, 2024-09-19T05:16:37.531 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 155, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013987976, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000090245 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:37.532 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 184976 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-19T05:16:37.533 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000331893, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000265241, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.534 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.535 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:37.536 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:37.537 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:37.538 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:37.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:37.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:37.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:37.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:37.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T05:16:37.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148977, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:37.540 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:37.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:37.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-19T05:16:37.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-19T05:16:37.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:37.541 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:37.541 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:37.542 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T05:16:37.542 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-19T05:16:37.542 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-19T05:16:37.542 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:37.542 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T05:16:37.542 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:37.543 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T05:16:37.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:37.543 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:37.543 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T05:16:37.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T05:16:37.543 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 218, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1416406, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 218, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 218, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1416406, 2024-09-19T05:16:37.544 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T05:16:37.545 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:37.545 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:37.545 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:37.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:37.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:37.545 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:16:37.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:37 smithi103 ceph-mon[34446]: pgmap v332: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:37.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:37 smithi174 ceph-mon[34488]: pgmap v332: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:38.233 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-19T05:16:38.233 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-19T05:16:38.233 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-19T05:16:38.233 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-19T05:16:38.233 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-19T05:16:38.233 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T05:16:38.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T05:16:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:38.973 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.003341839, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000044553, 2024-09-19T05:16:38.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002356381, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.004360910, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001453636 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T05:16:38.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 161, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 185504, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.012906059, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.009189093, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001593214, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:38.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 158, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.015566974, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000098525 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 185504 2024-09-19T05:16:38.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 221, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 211, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1448224, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 129824, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.040308658, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.014262926, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.038102544, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 210, 2024-09-19T05:16:38.978 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.010779563, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000051331 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1448224, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 129824 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-19T05:16:38.979 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:38.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:38.981 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:38.982 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148967, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 29, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-19T05:16:38.983 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:38.984 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 223, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1417056, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 223, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 223, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1417056, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-19T05:16:38.985 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-19T05:16:38.986 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:38.986 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-19T05:16:38.986 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-19T05:16:39.650 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:39 smithi064 ceph-mon[30960]: pgmap v333: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:39.713 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T05:16:39.713 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T05:16:39.713 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T05:16:39.714 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T05:16:39.714 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T05:16:39.714 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T05:16:39.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T05:16:39.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:39 smithi103 ceph-mon[34446]: pgmap v333: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:39.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:39 smithi174 ceph-mon[34488]: pgmap v333: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:16:40.617 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T05:16:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2604, 2024-09-19T05:16:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2577, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 742422, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 12830995, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 182, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.291930448, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.096448320, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.102469673, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.062449110, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2572, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.464056814, 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000180426 2024-09-19T05:16:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 461, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000062601, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000135 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 102432, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 356768 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 9576, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 13925, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3505834, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 9871066, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 10, 2024-09-19T05:16:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.656062726, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.239518282, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.344025226, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.103592664, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13926, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.589287724, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000042315 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 18294, 2024-09-19T05:16:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002642020, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000144 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 42720, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 126736 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3307, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2627, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2502103, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 5558502, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 32, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 7, 2024-09-19T05:16:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.327624635, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.113678501, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.147647378, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.077686014, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2626, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.223939567, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000085277 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1750, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000349186, 2024-09-19T05:16:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000199 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1807696, 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 247424 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T05:16:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3142, 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.203502470, 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000701305 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5054, 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.186788147, 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000036958 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 2590, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 2590, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2590, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.657623077, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001026109 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T05:16:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 72, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 46, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 7, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 39, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 35, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 2, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 3, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 2, 2024-09-19T05:16:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 2671, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 2656, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T05:16:40.627 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 261, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 4, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 234, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 6, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 261, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 167, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 22, 2024-09-19T05:16:40.628 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 145, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 94, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 1297, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 3616, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 3918, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 1778, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 18, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 3651 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.629 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 6, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 2, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 1212, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 1210, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T05:16:40.630 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 22, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T05:16:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 6460, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 176, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 176, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 6284, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 55, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 26, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 17, 2024-09-19T05:16:40.632 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 17, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 9, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 1, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 1, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4688965, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 22237775, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1341, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.091519374, 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001559671 2024-09-19T05:16:40.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 27, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 1259, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 1232, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 20, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 24, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 4, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 46, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 1248, 2024-09-19T05:16:40.634 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 1202, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 3, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 1229, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 1226, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 58728, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 231712 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 2719, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 2590, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 31, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T05:16:40.635 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1210, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.958005798, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000791740 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 113, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.019916282, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000176250 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.636 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000458661, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000229330 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.637 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T05:16:40.638 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 10, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001812153, 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000181215 2024-09-19T05:16:40.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 12, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.016896258, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001408021 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T05:16:40.640 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 10, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.049114306, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004911430 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009805133, 2024-09-19T05:16:40.641 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001634188 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002022612, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001011306 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.642 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.021474346, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003579057 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.017684923, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004421230 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.643 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001754257, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001754257 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 8, 2024-09-19T05:16:40.645 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.020433982, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002554247 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1206, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.538244366, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001275492 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.646 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 5, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 3, 2024-09-19T05:16:40.647 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 57, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 28, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 612, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:40.648 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:40.649 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:40.650 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 339280, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 130, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 176, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 2, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:40.651 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8320, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 183, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 145508, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 1042, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:40.652 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 5926, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 18399859, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 37, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 5926, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5926, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 28.123421700, 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004745768 2024-09-19T05:16:40.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5926, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 6182 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 5926, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 346, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 5580, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 51, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 43, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 63, 2024-09-19T05:16:40.654 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 1382, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 84, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 3937, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 283, 2024-09-19T05:16:40.655 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 86, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T05:16:40.656 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T05:16:40.657 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 88, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 11, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 12, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 12, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 224, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 29 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:16:40.658 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 13, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 6, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 3421, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 1210, 2024-09-19T05:16:40.659 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 15483, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 5291394, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 15483, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 15483, 2024-09-19T05:16:40.660 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 5291394, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:40.661 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 5926, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 18522085, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 5926, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 18522085, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T05:16:40.662 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 5926, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 5926, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 5926, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 5926, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.663 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1210, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 122226, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1210, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:40.664 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 41, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 122226, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x5641796f2960": { 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:40.665 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 6460, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 18043471, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 6460, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1341, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 18043471, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:40.666 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T05:16:41.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:41 smithi103 ceph-mon[34446]: pgmap v334: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:41.617 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T05:16:41.617 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T05:16:41.617 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T05:16:41.617 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-19T05:16:41.617 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-19T05:16:41.617 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T05:16:41.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T05:16:41.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:41 smithi174 ceph-mon[34488]: pgmap v334: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:41.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:41 smithi064 ceph-mon[30960]: pgmap v334: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:41.904 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:16:42.419 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 756, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 756, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 202585, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 239648, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.071046789, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.028216788, 2024-09-19T05:16:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.018170308, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.017423634, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 755, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.034400799, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000045563 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 890, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 878, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1637198, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 339293, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.101151612, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.033724964, 2024-09-19T05:16:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.054518773, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.021363857, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 877, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.038015363, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000043347 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1476784, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 131952 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 746, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 905, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 250304, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 679824, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.100853080, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.034032446, 2024-09-19T05:16:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.027182209, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.028698777, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 902, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.061205850, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000067855 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 443232 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T05:16:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3537, 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.097289273, 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000027506 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.427 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T05:16:42.428 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-19T05:16:42.429 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T05:16:42.430 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:16:42.431 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T05:16:42.432 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T05:16:42.433 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T05:16:42.434 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 6, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T05:16:42.435 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4196314, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4196173, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.436 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 6 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-09-19T05:16:42.437 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 48440, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-19T05:16:42.438 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.439 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T05:16:42.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.442 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.443 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.445 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T05:16:42.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.447 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T05:16:42.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.450 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 3, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2139, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T05:16:42.452 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:42.453 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:42.454 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 189463, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 52, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:42.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8320, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 183, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-09-19T05:16:42.456 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 2148, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-19T05:16:42.457 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 2148, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2148, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.058978134, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000493006 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2148, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2149 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 2148, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 2148, 2024-09-19T05:16:42.458 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 1394, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 752, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-19T05:16:42.459 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T05:16:42.460 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T05:16:42.461 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 88, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 11, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 5, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 5, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-19T05:16:42.462 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T05:16:42.463 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2388, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1859947, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:42.464 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2388, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2388, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1859947, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:42.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2148, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2015, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2148, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2015, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.466 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2148, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2148, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2148, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2148, 2024-09-19T05:16:42.467 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:42.468 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x562f67832b40": { 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T05:16:42.469 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:42.470 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.471 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:42.471 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T05:16:43.232 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T05:16:43.232 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T05:16:43.233 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-19T05:16:43.233 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:16:43.233 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:16:43.233 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T05:16:43.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T05:16:43.476 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:16:43.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:43 smithi103 ceph-mon[34446]: pgmap v335: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:43.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:43 smithi174 ceph-mon[34488]: pgmap v335: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:43.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:43 smithi064 ceph-mon[30960]: pgmap v335: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2157, 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2157, 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 614908, 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 685109, 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-19T05:16:43.977 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.168389450, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.054006810, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.044435094, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.056714087, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2156, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.071055857, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000032957 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 162, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 876, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 443754, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.010041986, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.006098521, 2024-09-19T05:16:43.979 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002064475, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000104971, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 159, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.014819792, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000093206 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:43.980 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 443056 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 238, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 226, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1476780, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 131630, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.043100699, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.015947856, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.041118747, 2024-09-19T05:16:43.981 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000022258, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 225, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.015008195, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000066703 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1476512, 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 131312 2024-09-19T05:16:43.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.983 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T05:16:43.983 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T05:16:43.983 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T05:16:43.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.983 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T05:16:43.983 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3631, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.083298507, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000022940 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.984 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T05:16:43.985 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-19T05:16:43.986 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T05:16:43.987 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:16:43.988 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-19T05:16:43.989 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-19T05:16:43.990 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T05:16:43.991 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 9, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-19T05:16:43.992 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4196989, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4196848, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.993 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 9 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-19T05:16:43.994 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 48324, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.995 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T05:16:43.996 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.998 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T05:16:43.999 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.001 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.003 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.004 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T05:16:44.006 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-09-19T05:16:44.007 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 4, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 3, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 3, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 3636, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:44.008 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:44.009 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 181320, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 47, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:44.011 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8320, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 183, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T05:16:44.012 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 2156, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-19T05:16:44.013 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 2156, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2156, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.235974478, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000573272 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2156, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2157 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 2156, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 2156, 2024-09-19T05:16:44.014 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 1399, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 755, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-19T05:16:44.015 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T05:16:44.016 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T05:16:44.017 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 88, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 11, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 4, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 4, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-19T05:16:44.018 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T05:16:44.019 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2395, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1861846, 2024-09-19T05:16:44.020 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2395, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2395, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1861846, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T05:16:44.021 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2156, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2690, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2156, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2690, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.022 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2156, 2024-09-19T05:16:44.023 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2156, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2156, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2156, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x563ec2c4eb40": { 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:44.027 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T05:16:44.807 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T05:16:44.807 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T05:16:44.808 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-19T05:16:44.808 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:16:44.808 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:16:44.808 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T05:16:44.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T05:16:45.056 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:16:45.468 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:45 smithi103 ceph-mon[34446]: pgmap v336: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2992, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002921028, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000036879, 2024-09-19T05:16:45.571 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002110441, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.003955474, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001318491 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T05:16:45.572 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 223, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 213, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1448768, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 131104, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.041711873, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.015662240, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.038478782, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:45.573 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 212, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.012952996, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000061099 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1448768, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 131104 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:45.574 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 164, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 186032, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.010866495, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007269227, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001383281, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 161, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.017792049, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000110509 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.575 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 186032 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.576 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:45.577 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148955, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T05:16:45.579 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:45.580 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 225, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1417316, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 225, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T05:16:45.581 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 225, 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1417316, 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T05:16:45.582 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T05:16:45.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:45 smithi174 ceph-mon[34488]: pgmap v336: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:45.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:45 smithi064 ceph-mon[30960]: pgmap v336: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:46.342 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-19T05:16:46.342 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-19T05:16:46.342 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-19T05:16:46.342 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-19T05:16:46.342 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-19T05:16:46.342 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T05:16:46.343 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T05:16:46.602 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:16:47.406 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 38, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 54, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8393, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 90076, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.010638206, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.001043477, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.004644287, 2024-09-19T05:16:47.407 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000530446, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 52, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.010276894, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000197632 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 38, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000008222, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000216 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:16:47.408 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 65, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 226, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 12330, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 948097, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 49, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.028307038, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.008145059, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.016745183, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000412898, 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 223, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.027423952, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000122977 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 54, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000004954, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000091 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 447728 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 639, 2024-09-19T05:16:47.410 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 782, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1583182, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 466211, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.097071426, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.037795641, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.066977854, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.004252333, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 782, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.046702409, 2024-09-19T05:16:47.411 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000059721 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 870, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000240504, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000276 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1491616, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 142992 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T05:16:47.412 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 46, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001583505, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000034424 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T05:16:47.413 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000015030, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000007515 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 3, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 3, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001510763, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000503587 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-19T05:16:47.414 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 11, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 11, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 10, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 1, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-19T05:16:47.415 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 8, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 4, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 2, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T05:16:47.416 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:16:47.417 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:16:47.418 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:16:47.419 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-19T05:16:47.419 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:16:47.419 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T05:16:47.419 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-19T05:16:47.419 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.419 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 3, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T05:16:47.420 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.421 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 6, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 6, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T05:16:47.422 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4196314, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.010862297, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002172459 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 14, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 14, 2024-09-19T05:16:47.423 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 13, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 14, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 1, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 11, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 12, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 1, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 2, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 2, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 49852, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T05:16:47.424 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 5, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 3, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 21, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000156399, 2024-09-19T05:16:47.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000156399 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T05:16:47.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001354364, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000677182 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.427 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.428 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T05:16:47.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.430 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.431 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.432 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.434 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.436 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T05:16:47.437 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 2, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 1, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 618, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:47.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:47.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:47.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 247271, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 88, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:47.441 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8320, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 183, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:47.442 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 56544, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 224, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T05:16:47.443 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 44, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 12547, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 44, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.204997131, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004659025 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44, 2024-09-19T05:16:47.444 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 68 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 44, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 44, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 5, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 8, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T05:16:47.445 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T05:16:47.446 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T05:16:47.447 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 88, 2024-09-19T05:16:47.448 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 11, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 8, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 8, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 17, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 2 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:16:47.449 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T05:16:47.450 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 738, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1520091, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 738, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 738, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1520091, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.451 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 44, 2024-09-19T05:16:47.452 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 12547, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 44, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 12547, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:47.453 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 44, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 44, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 44, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 44, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.454 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.455 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55b2a563c960": { 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 6, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2010, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 6, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:47.456 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:47.457 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5, 2024-09-19T05:16:47.457 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2010, 2024-09-19T05:16:47.457 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:47.457 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:47.457 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:47.464 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:47.464 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.464 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:47.464 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T05:16:47.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:47 smithi174 ceph-mon[34488]: pgmap v337: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:47.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:47 smithi103 ceph-mon[34446]: pgmap v337: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:47.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:47 smithi064 ceph-mon[30960]: pgmap v337: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:48.410 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T05:16:48.411 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T05:16:48.411 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T05:16:48.411 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:16:48.411 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:16:48.411 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T05:16:48.411 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T05:16:48.669 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:16:49.215 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1588, 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1590, 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 5931440, 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 504994, 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-19T05:16:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.163631223, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.049005068, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.047399742, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.052269082, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1588, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.062680407, 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000039471 2024-09-19T05:16:49.217 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 37, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 197, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8398578, 2024-09-19T05:16:49.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 456220, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.016640035, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.007323273, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.006609923, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000697931, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 194, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018427003, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000094984 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.219 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 445024 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 989, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 975, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 5858093, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 370111, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:49.220 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.120665783, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.038837185, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.062192250, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.026929742, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 975, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.049874412, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000051153 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1480208, 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 133200 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T05:16:49.222 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3860, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.111387448, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000028856 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.223 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-19T05:16:49.224 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 1241, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 13, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 1228, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 1223, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-19T05:16:49.225 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T05:16:49.226 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:16:49.227 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:16:49.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T05:16:49.229 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:16:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 6460, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T05:16:49.231 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 9, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 18, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4688965, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 22237775, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 22237663, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.232 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 6460 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 27, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 1237, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 1210, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 20, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 24, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 4, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 1241, 2024-09-19T05:16:49.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 1246, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 5, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 50436, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-19T05:16:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.236 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.237 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.238 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.239 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T05:16:49.240 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.240 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.240 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.240 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.240 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T05:16:49.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.243 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T05:16:49.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.246 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.247 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 5, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 3, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 4, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2678, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T05:16:49.248 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:49.249 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:49.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 198247, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 56, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T05:16:49.251 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8320, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 183, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 982316, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 5452, 2024-09-19T05:16:49.252 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 2368, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 26, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 4, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 2368, 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T05:16:49.253 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2368, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.282981791, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000541799 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2368, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2369 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 2368, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 2368, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T05:16:49.254 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 1521, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 844, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T05:16:49.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T05:16:49.256 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T05:16:49.257 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 88, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 11, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 6, 2024-09-19T05:16:49.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 6, 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T05:16:49.259 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.260 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T05:16:49.261 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.262 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:49.262 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:49.262 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:49.262 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:49.262 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 2610, 2024-09-19T05:16:49.262 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 19937893, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 2610, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2610, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 19937893, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.263 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:49.264 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:49.265 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:49.265 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:49.265 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:49.265 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2368, 2024-09-19T05:16:49.265 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 18043481, 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2368, 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 18043481, 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.266 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.267 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T05:16:49.267 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:49.267 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T05:16:49.267 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:49.267 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2368, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2368, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2368, 2024-09-19T05:16:49.268 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2368, 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:16:49.269 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:49.270 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.271 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55d8dfefab40": { 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:49.272 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:49.273 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.274 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:49.274 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T05:16:49.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:49 smithi174 ceph-mon[34488]: pgmap v338: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:49.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:49 smithi103 ceph-mon[34446]: pgmap v338: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:49.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:49 smithi064 ceph-mon[30960]: pgmap v338: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:49.970 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T05:16:49.971 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T05:16:49.971 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T05:16:49.971 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:16:49.971 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:16:49.971 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T05:16:49.971 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T05:16:50.217 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:16:50.721 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 346, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 335, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1505558, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 664134, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 53, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.064567211, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.017805948, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.053953765, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000528081, 2024-09-19T05:16:50.722 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 331, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.023746572, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000071741 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 76, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000008158, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000107 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1494640, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 142016 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.723 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 424, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 591, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 109489, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 391733, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.059577859, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.023664128, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.029510481, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.004350097, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 591, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.038631495, 2024-09-19T05:16:50.724 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000065366 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 958, 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000236027, 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000246 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 121616 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.725 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 25, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 157, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8133, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 387798, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 11, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.012540994, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.005600536, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.005214838, 2024-09-19T05:16:50.726 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000379292, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 156, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.029325082, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000187981 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000302, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000075 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 325296 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.727 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 46, 2024-09-19T05:16:50.728 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002239245, 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000048679 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000013794, 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000006897 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T05:16:50.729 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 11, 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 8, 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8, 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001947926, 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000243490 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T05:16:50.730 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 3, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 14, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 14, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 14, 2024-09-19T05:16:50.731 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 4, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 4, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 16, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 12, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 2, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T05:16:50.732 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-19T05:16:50.733 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 1, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 1, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:16:50.734 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 2 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 3, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T05:16:50.735 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:16:50.736 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 9, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 9, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T05:16:50.737 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4196989, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.013242052, 2024-09-19T05:16:50.738 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001655256 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 17, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 17, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 15, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 16, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 1, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 14, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 14, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-09-19T05:16:50.739 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 4, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 8, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 4, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 51064, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 13, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 11, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 21, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-19T05:16:50.740 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001644769, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000274128 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.741 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000303157, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000151578 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.742 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T05:16:50.743 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.744 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.745 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T05:16:50.745 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.745 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.745 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.746 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.747 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.748 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.750 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.751 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 4, 2024-09-19T05:16:50.753 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 3, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 7, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 2, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 657, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:50.754 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:50.755 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:50.756 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:50.756 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:50.756 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:50.756 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:50.756 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:50.756 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:50.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:50.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:50.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:50.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:50.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:50.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:50.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 264439, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 94, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 88, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 1, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8320, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 183, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:50.759 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 69213, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 303, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T05:16:50.760 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 45, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 13268, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 28, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 45, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 45, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.212997049, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004733267 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 45, 2024-09-19T05:16:50.761 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 68 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 45, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 45, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 8, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 7, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T05:16:50.762 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T05:16:50.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T05:16:50.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T05:16:50.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-09-19T05:16:50.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T05:16:50.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T05:16:50.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T05:16:50.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-09-19T05:16:50.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T05:16:50.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:16:50.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T05:16:50.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T05:16:50.765 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T05:16:50.766 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 88, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 11, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 9, 2024-09-19T05:16:50.767 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 9, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 16, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 1 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T05:16:50.768 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T05:16:50.769 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 791, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1534650, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 791, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:50.770 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 791, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1534650, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:50.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 45, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 13268, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 45, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 13268, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T05:16:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 45, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 45, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 45, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 45, 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:50.773 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:50.774 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55de39902960": { 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:50.775 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 9, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2685, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 9, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 8, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2685, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:50.777 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T05:16:51.467 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:51 smithi174 ceph-mon[34488]: pgmap v339: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:51.494 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T05:16:51.495 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T05:16:51.495 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T05:16:51.495 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:16:51.495 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:16:51.495 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T05:16:51.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T05:16:51.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:51 smithi103 ceph-mon[34446]: pgmap v339: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:51.738 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:16:51.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:51 smithi064 ceph-mon[30960]: pgmap v339: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:52.345 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T05:16:52.345 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:16:52.345 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 0, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 0, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 0, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 0, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.000971906, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.000154386, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.346 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 6, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 168, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 3312, 2024-09-19T05:16:52.347 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 186832, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.015600178, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.007635456, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.005553336, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 166, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.024425277, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000147140 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:52.348 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 186832 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 226, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 216, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1449584, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 133024, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 2, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 1, 2024-09-19T05:16:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.042962764, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.015113260, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.042128325, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 215, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.013395545, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000062304 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:52.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1449584, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 133024 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 0, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:16:52.351 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T05:16:52.352 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:16:52.353 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 148903, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 27, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 0, 2024-09-19T05:16:52.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 928, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 2, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T05:16:52.355 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 228, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1417706, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 228, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 228, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1417706, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:52.356 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T05:16:52.357 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T05:16:52.935 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-19T05:16:52.935 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-19T05:16:52.935 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-19T05:16:52.935 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-19T05:16:52.935 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-19T05:16:52.936 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T05:16:52.947 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T05:16:52.947 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T05:16:52.947 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:16:52.969 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T05:16:52.969 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:16:53.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T05:16:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:53.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:53 smithi103 ceph-mon[34446]: pgmap v340: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:53.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:53 smithi174 ceph-mon[34488]: pgmap v340: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:53.802 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:53 smithi064 ceph-mon[30960]: pgmap v340: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:53.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 -- 172.21.15.64:0/1327897561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 msgr2=0x7f108010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 --2- 172.21.15.64:0/1327897561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108010da40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1078009ca0 tx=0x7f107802a950 comp rx=0 tx=0).stop 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 -- 172.21.15.64:0/1327897561 shutdown_connections 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 --2- 172.21.15.64:0/1327897561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108010da40 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 --2- 172.21.15.64:0/1327897561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080102570 0x7f10801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 --2- 172.21.15.64:0/1327897561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080100e20 0x7f1080101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 -- 172.21.15.64:0/1327897561 >> 172.21.15.64:0/1327897561 conn(0x7f10800fc7d0 msgr2=0x7f10800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 -- 172.21.15.64:0/1327897561 shutdown_connections 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 -- 172.21.15.64:0/1327897561 wait complete. 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.886+0000 7f108571a640 1 Processor -- start 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 -- start start 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080100e20 0x7f10801983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080102570 0x7f1080198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1080110e30 con 0x7f1080102ef0 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1080110cb0 con 0x7f1080100e20 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f108571a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1080110fb0 con 0x7f1080102570 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57042/0 (socket says 172.21.15.64:57042) 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 -- 172.21.15.64:0/4005809853 learned_addr learned my addr 172.21.15.64:0/4005809853 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f1077fff640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080102570 0x7f1080198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107effd640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080100e20 0x7f10801983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 -- 172.21.15.64:0/4005809853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080102570 msgr2=0x7f1080198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080102570 0x7f1080198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 -- 172.21.15.64:0/4005809853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080100e20 msgr2=0x7f10801983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080100e20 0x7f10801983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107f7fe640 1 -- 172.21.15.64:0/4005809853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108019d390 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.887+0000 7f107effd640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080100e20 0x7f10801983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.888+0000 7f107f7fe640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108019ccb0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1078004af0 tx=0x7f107802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.888+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078034040 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.888+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108019d620 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.888+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10780308f0 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.888+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078004670 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.888+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10801a4f60 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.889+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1080101200 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.890+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f1078037020 con 0x7f1080102ef0 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.891+0000 7f107cff9640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f106c0a33b0 0x7f106c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.891+0000 7f107effd640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f106c0a33b0 0x7f106c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.891+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f10780f0cd0 con 0x7f1080102ef0 2024-09-19T05:16:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.891+0000 7f107effd640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f106c0a33b0 0x7f106c0a5870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10700058c0 tx=0x7f1070005850 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:53.898+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1078081210 con 0x7f1080102ef0 2024-09-19T05:16:54.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.089+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1080199740 con 0x7f1080102ef0 2024-09-19T05:16:54.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.090+0000 7f107cff9640 1 -- 172.21.15.64:0/4005809853 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v88) ==== 81+0+945 (secure 0 0 0) 0x7f1080199740 con 0x7f1080102ef0 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3907259856 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3907259856 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/868126904 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4083716942 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2000236557 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3045596814 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4215744929 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3048471422 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:54.093 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/2584640926 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/4164984314 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/4164984314 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1467175216 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1065736541 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/2584640926 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2343862077 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1076683694 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:54.094 INFO:teuthology.orchestra.run.smithi064.stderr:listed 16 entries 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f106c0a33b0 msgr2=0x7f106c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f106c0a33b0 0x7f106c0a5870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10700058c0 tx=0x7f1070005850 comp rx=0 tx=0).stop 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 msgr2=0x7f108019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108019ccb0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1078004af0 tx=0x7f107802f5d0 comp rx=0 tx=0).stop 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 shutdown_connections 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f106c0a33b0 0x7f106c0a5870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080102ef0 0x7f108019ccb0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:54.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1080102570 0x7f1080198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:54.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 --2- 172.21.15.64:0/4005809853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1080100e20 0x7f10801983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:54.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.092+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 >> 172.21.15.64:0/4005809853 conn(0x7f10800fc7d0 msgr2=0x7f108006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:54.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.093+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 shutdown_connections 2024-09-19T05:16:54.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:54.093+0000 7f108571a640 1 -- 172.21.15.64:0/4005809853 wait complete. 2024-09-19T05:16:54.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:54 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4005809853' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:16:54.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:54 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4005809853' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:16:54.810 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:54 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4005809853' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:16:54.840 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-19T05:16:54.840 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:16:54.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T05:16:55.156 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:55.547 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:55 smithi064 ceph-mon[30960]: pgmap v341: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/4108801324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8106a80 msgr2=0x7ff8b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/4108801324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8106a80 0x7ff8b8106e60 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009930 tx=0x7ff8b402a420 comp rx=0 tx=0).stop 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/4108801324 shutdown_connections 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/4108801324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8b8101420 0x7ff8b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/4108801324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b8100a80 0x7ff8b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/4108801324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8106a80 0x7ff8b8106e60 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/4108801324 >> 172.21.15.64:0/4108801324 conn(0x7ff8b80fc7a0 msgr2=0x7ff8b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/4108801324 shutdown_connections 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.628+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/4108801324 wait complete. 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 Processor -- start 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 -- start start 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b8100a80 0x7ff8b81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 0x7ff8b81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8b8106a80 0x7ff8b819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8110d30 con 0x7ff8b8101420 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8110bb0 con 0x7ff8b8100a80 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bfc8a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8110eb0 con 0x7ff8b8106a80 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bd225640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 0x7ff8b81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:55.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bd225640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 0x7ff8b81977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41670/0 (socket says 172.21.15.64:41670) 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bd225640 1 -- 172.21.15.64:0/2860178234 learned_addr learned my addr 172.21.15.64:0/2860178234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8bda26640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b8100a80 0x7ff8b81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.629+0000 7ff8be227640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8b8106a80 0x7ff8b819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bd225640 1 -- 172.21.15.64:0/2860178234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8b8106a80 msgr2=0x7ff8b819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bd225640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8b8106a80 0x7ff8b819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bd225640 1 -- 172.21.15.64:0/2860178234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b8100a80 msgr2=0x7ff8b81972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bd225640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b8100a80 0x7ff8b81972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bd225640 1 -- 172.21.15.64:0/2860178234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b819c260 con 0x7ff8b8101420 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bd225640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 0x7ff8b81977f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800d950 tx=0x7ff8a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8017030 con 0x7ff8b8101420 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a8004020 con 0x7ff8b8101420 2024-09-19T05:16:55.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b819c550 con 0x7ff8b8101420 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80047a0 con 0x7ff8b8101420 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.630+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a3e30 con 0x7ff8b8101420 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.631+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b8100ee0 con 0x7ff8b8101420 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.632+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff8a80041d0 con 0x7ff8b8101420 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.633+0000 7ff8aeffd640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8a40a33b0 0x7ff8a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.633+0000 7ff8bda26640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8a40a33b0 0x7ff8a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.633+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff8a80cf7a0 con 0x7ff8b8101420 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.633+0000 7ff8bda26640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8a40a33b0 0x7ff8a40a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009900 tx=0x7ff8b4000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:55.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.637+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8a809c150 con 0x7ff8b8101420 2024-09-19T05:16:55.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:55 smithi103 ceph-mon[34446]: pgmap v341: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:55.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:55 smithi174 ceph-mon[34488]: pgmap v341: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.825+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8b802d7c0 con 0x7ff8b8101420 2024-09-19T05:16:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.826+0000 7ff8aeffd640 1 -- 172.21.15.64:0/2860178234 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v88) ==== 81+0+945 (secure 0 0 0) 0x7ff8a809ba50 con 0x7ff8b8101420 2024-09-19T05:16:55.828 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3907259856 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3907259856 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/868126904 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4083716942 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2000236557 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3045596814 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4215744929 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3048471422 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/2584640926 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/4164984314 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/4164984314 2024-09-20T04:56:54.761876+0000 2024-09-19T05:16:55.829 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1467175216 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:55.830 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1065736541 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:55.830 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/2584640926 2024-09-20T04:57:18.299954+0000 2024-09-19T05:16:55.830 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2343862077 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:55.830 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1076683694 2024-09-20T05:05:53.680120+0000 2024-09-19T05:16:55.830 INFO:teuthology.orchestra.run.smithi064.stderr:listed 16 entries 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8a40a33b0 msgr2=0x7ff8a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8a40a33b0 0x7ff8a40a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009900 tx=0x7ff8b4000bc0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 msgr2=0x7ff8b81977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 0x7ff8b81977f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800d950 tx=0x7ff8a800de10 comp rx=0 tx=0).stop 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 shutdown_connections 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff8a40a33b0 0x7ff8a40a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8b8106a80 0x7ff8b819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101420 0x7ff8b81977f0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 --2- 172.21.15.64:0/2860178234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b8100a80 0x7ff8b81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 >> 172.21.15.64:0/2860178234 conn(0x7ff8b80fc7a0 msgr2=0x7ff8b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 shutdown_connections 2024-09-19T05:16:55.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:16:55.828+0000 7ff8bfc8a640 1 -- 172.21.15.64:0/2860178234 wait complete. 2024-09-19T05:16:56.465 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-09-19T05:16:56.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:16:56.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T05:16:56.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[80601]: fuse finished with error 0 and tester_r 0 2024-09-19T05:16:56.494 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:16:56.718 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:56 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/2860178234' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:16:56.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:56 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/2860178234' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:16:56.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:56 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/2860178234' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:16:57.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:57 smithi103 ceph-mon[34446]: pgmap v342: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:57.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:57 smithi174 ceph-mon[34488]: pgmap v342: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:57.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:57 smithi064 ceph-mon[30960]: pgmap v342: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T05:16:59.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:16:59 smithi103 ceph-mon[34446]: pgmap v343: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:59.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:16:59 smithi174 ceph-mon[34488]: pgmap v343: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:16:59.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:16:59 smithi064 ceph-mon[30960]: pgmap v343: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T05:17:01.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:01 smithi103 ceph-mon[34446]: pgmap v344: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T05:17:01.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:01 smithi174 ceph-mon[34488]: pgmap v344: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T05:17:01.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:01 smithi064 ceph-mon[30960]: pgmap v344: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T05:17:02.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:17:02.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:17:02.597 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T05:17:02.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:17:02.622 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-19T05:17:02.682 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T05:17:02.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:17:02.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:17:02.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:17:02.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-09-19T05:17:02.848 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T05:17:02.864 INFO:tasks.cephadm:Teardown begin 2024-09-19T05:17:02.864 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T05:17:02.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:17:03.130 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:03.565 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:03 smithi064 ceph-mon[30960]: pgmap v345: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 -- 172.21.15.64:0/3801933361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 msgr2=0x7f2f7c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 --2- 172.21.15.64:0/3801933361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c103120 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009ca0 tx=0x7f2f6402a950 comp rx=0 tx=0).stop 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 -- 172.21.15.64:0/3801933361 shutdown_connections 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 --2- 172.21.15.64:0/3801933361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f7c10c170 0x7f2f7c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 --2- 172.21.15.64:0/3801933361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f7c103660 0x7f2f7c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 --2- 172.21.15.64:0/3801933361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c103120 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 -- 172.21.15.64:0/3801933361 >> 172.21.15.64:0/3801933361 conn(0x7f2f7c0fc7a0 msgr2=0x7f2f7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 -- 172.21.15.64:0/3801933361 shutdown_connections 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.646+0000 7f2f81022640 1 -- 172.21.15.64:0/3801933361 wait complete. 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.647+0000 7f2f81022640 1 Processor -- start 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.647+0000 7f2f81022640 1 -- start start 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f81022640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f81022640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f7c103660 0x7f2f7c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f81022640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f7c10c170 0x7f2f7c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f81022640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c111e20 con 0x7f2f7c102d40 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f81022640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c111ca0 con 0x7f2f7c103660 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f81022640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c111fa0 con 0x7f2f7c10c170 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41696/0 (socket says 172.21.15.64:41696) 2024-09-19T05:17:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 -- 172.21.15.64:0/4060444981 learned_addr learned my addr 172.21.15.64:0/4060444981 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7b59e640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f7c10c170 0x7f2f7c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7a59c640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f7c103660 0x7f2f7c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 -- 172.21.15.64:0/4060444981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f7c10c170 msgr2=0x7f2f7c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f7c10c170 0x7f2f7c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 -- 172.21.15.64:0/4060444981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f7c103660 msgr2=0x7f2f7c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f7c103660 0x7f2f7c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 -- 172.21.15.64:0/4060444981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c19d3b0 con 0x7f2f7c102d40 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f7ad9d640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c198400 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005cc0 tx=0x7f2f6402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64034040 con 0x7f2f7c102d40 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.648+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f640308f0 con 0x7f2f7c102d40 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.649+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7c19d640 con 0x7f2f7c102d40 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.649+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640047c0 con 0x7f2f7c102d40 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.649+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7c1a4f80 con 0x7f2f7c102d40 2024-09-19T05:17:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.650+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7c06b070 con 0x7f2f7c102d40 2024-09-19T05:17:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.652+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f2f64037020 con 0x7f2f7c102d40 2024-09-19T05:17:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.652+0000 7f2f73fff640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2f680a33b0 0x7f2f680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.653+0000 7f2f7a59c640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2f680a33b0 0x7f2f680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.653+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f2f640f0d40 con 0x7f2f7c102d40 2024-09-19T05:17:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.653+0000 7f2f7a59c640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2f680a33b0 0x7f2f680a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0fee70 tx=0x7f2f6c00a560 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.656+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f640bd890 con 0x7f2f7c102d40 2024-09-19T05:17:03.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:03 smithi103 ceph-mon[34446]: pgmap v345: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:17:03.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:03 smithi174 ceph-mon[34488]: pgmap v345: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:17:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.868+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f7c107210 con 0x7f2f7c102d40 2024-09-19T05:17:03.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.869+0000 7f2f73fff640 1 -- 172.21.15.64:0/4060444981 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f2f640bd190 con 0x7f2f7c102d40 2024-09-19T05:17:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:17:03.871 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:17:03.871 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:17:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2f680a33b0 msgr2=0x7f2f680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:03.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2f680a33b0 0x7f2f680a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0fee70 tx=0x7f2f6c00a560 comp rx=0 tx=0).stop 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 msgr2=0x7f2f7c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c198400 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005cc0 tx=0x7f2f6402f650 comp rx=0 tx=0).stop 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 shutdown_connections 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f2f680a33b0 0x7f2f680a5870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f7c10c170 0x7f2f7c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f7c103660 0x7f2f7c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 --2- 172.21.15.64:0/4060444981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f7c102d40 0x7f2f7c198400 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 >> 172.21.15.64:0/4060444981 conn(0x7f2f7c0fc7a0 msgr2=0x7f2f7c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 shutdown_connections 2024-09-19T05:17:03.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:03.872+0000 7f2f81022640 1 -- 172.21.15.64:0/4060444981 wait complete. 2024-09-19T05:17:04.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:04 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/4060444981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:17:04.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:17:04.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:04 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/4060444981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:17:04.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:04 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/4060444981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:17:04.805 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 -- 172.21.15.64:0/453102136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 msgr2=0x7ff598106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 --2- 172.21.15.64:0/453102136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff598106e60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff588009930 tx=0x7ff58802a420 comp rx=0 tx=0).stop 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 -- 172.21.15.64:0/453102136 shutdown_connections 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 --2- 172.21.15.64:0/453102136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff598101420 0x7ff59810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 --2- 172.21.15.64:0/453102136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff598100a80 0x7ff598100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 --2- 172.21.15.64:0/453102136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff598106e60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 -- 172.21.15.64:0/453102136 >> 172.21.15.64:0/453102136 conn(0x7ff5980fc7a0 msgr2=0x7ff5980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 -- 172.21.15.64:0/453102136 shutdown_connections 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.286+0000 7ff59cd51640 1 -- 172.21.15.64:0/453102136 wait complete. 2024-09-19T05:17:05.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 Processor -- start 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 -- start start 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff598100a80 0x7ff59810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff598101420 0x7ff59810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff59810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff598111ed0 con 0x7ff598106a80 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff598111d50 con 0x7ff598100a80 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59cd51640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff598112050 con 0x7ff598101420 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff596d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff59810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff596d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff59810ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41530/0 (socket says 172.21.15.64:41530) 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff596d9d640 1 -- 172.21.15.64:0/1940218289 learned_addr learned my addr 172.21.15.64:0/1940218289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff59659c640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff598100a80 0x7ff59810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.287+0000 7ff595d9b640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff598101420 0x7ff59810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff596d9d640 1 -- 172.21.15.64:0/1940218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff598101420 msgr2=0x7ff59810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:05.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff596d9d640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff598101420 0x7ff59810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff596d9d640 1 -- 172.21.15.64:0/1940218289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff598100a80 msgr2=0x7ff59810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff596d9d640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff598100a80 0x7ff59810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff596d9d640 1 -- 172.21.15.64:0/1940218289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5981a69f0 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff596d9d640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff59810ee60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff59000b970 tx=0x7ff59000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590019070 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5981a6ce0 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff590014440 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5900075d0 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.288+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5981a7220 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.289+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff598100f60 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.291+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7ff59001c070 con 0x7ff598106a80 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.291+0000 7ff5877fe640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5800a3430 0x7ff5800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.292+0000 7ff59659c640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5800a3430 0x7ff5800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.292+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff5900cf0f0 con 0x7ff598106a80 2024-09-19T05:17:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.292+0000 7ff59659c640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5800a3430 0x7ff5800a58f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff588006010 tx=0x7ff588000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.296+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff59009b500 con 0x7ff598106a80 2024-09-19T05:17:05.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.508+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff59802d7c0 con 0x7ff598106a80 2024-09-19T05:17:05.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.509+0000 7ff5877fe640 1 -- 172.21.15.64:0/1940218289 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7ff59009f800 con 0x7ff598106a80 2024-09-19T05:17:05.511 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:17:05.511 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-19T05:05:52:697534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/726782789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":726782789},{"type":"v1","addr":"172.21.15.64:6835","nonce":726782789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2636818153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2636818153},{"type":"v1","addr":"172.21.15.64:6837","nonce":2636818153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4284831800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4284831800},{"type":"v1","addr":"172.21.15.64:6839","nonce":4284831800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3250181965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3250181965},{"type":"v1","addr":"172.21.15.64:6841","nonce":3250181965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/550094745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":550094745},{"type":"v1","addr":"172.21.15.103:6833","nonce":550094745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/252917517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":252917517},{"type":"v1","addr":"172.21.15.174:6833","nonce":252917517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:04:08.534666+0000","modified":"2024-09-19T05:05:52.647352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24466,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/642500144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":642500144},{"type":"v1","addr":"172.21.15.103:6835","nonce":642500144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/3232384876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3232384876},{"type":"v1","addr":"172.21.15.103:6837","nonce":3232384876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2004435266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2004435266},{"type":"v1","addr":"172.21.15.174:6835","nonce":2004435266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1599126810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1599126810},{"type":"v1","addr":"172.21.15.174:6837","nonce":1599126810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3328031427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3328031427},{"type":"v1","addr":"172.21.15.174:6839","nonce":3328031427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6839/100524955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":100524955},{"type":"v1","addr":"172.21.15.103:6839","nonce":100524955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24466,24448]},"id":1}]} 2024-09-19T05:17:05.512 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5800a3430 msgr2=0x7ff5800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5800a3430 0x7ff5800a58f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff588006010 tx=0x7ff588000bc0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 msgr2=0x7ff59810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff59810ee60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff59000b970 tx=0x7ff59000bd70 comp rx=0 tx=0).stop 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 shutdown_connections 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7ff5800a3430 0x7ff5800a58f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff598106a80 0x7ff59810ee60 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff598101420 0x7ff59810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 --2- 172.21.15.64:0/1940218289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff598100a80 0x7ff59810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:05.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 >> 172.21.15.64:0/1940218289 conn(0x7ff5980fc7a0 msgr2=0x7ff5980fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:05.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 shutdown_connections 2024-09-19T05:17:05.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:05.511+0000 7ff59cd51640 1 -- 172.21.15.64:0/1940218289 wait complete. 2024-09-19T05:17:05.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:05 smithi174 ceph-mon[34488]: pgmap v346: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:17:05.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:05 smithi064 ceph-mon[30960]: pgmap v346: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:17:05.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:05 smithi103 ceph-mon[34446]: pgmap v346: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T05:17:06.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:17:06.394 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:06.747 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:06 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1940218289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:17:06.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:06 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1940218289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:17:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.827+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/2291260382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 msgr2=0x7fa5dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.827+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/2291260382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1067e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009900 tx=0x7fa5d002a410 comp rx=0 tx=0).stop 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/2291260382 shutdown_connections 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/2291260382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5dc106f10 0x7fa5dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/2291260382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1067e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/2291260382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc101de0 0x7fa5dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/2291260382 >> 172.21.15.64:0/2291260382 conn(0x7fa5dc0fd8c0 msgr2=0x7fa5dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/2291260382 shutdown_connections 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/2291260382 wait complete. 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 Processor -- start 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.828+0000 7fa5e2a4b640 1 -- start start 2024-09-19T05:17:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5dc101de0 0x7fa5dc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc106f10 0x7fa5dc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc111e20 con 0x7fa5dc102790 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc111ca0 con 0x7fa5dc101de0 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc111fa0 con 0x7fa5dc106f10 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41540/0 (socket says 172.21.15.64:41540) 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 -- 172.21.15.64:0/1278505262 learned_addr learned my addr 172.21.15.64:0/1278505262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5dbfff640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5dc101de0 0x7fa5dc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e0fe8640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc106f10 0x7fa5dc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 -- 172.21.15.64:0/1278505262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc106f10 msgr2=0x7fa5dc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc106f10 0x7fa5dc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 -- 172.21.15.64:0/1278505262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5dc101de0 msgr2=0x7fa5dc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5dc101de0 0x7fa5dc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 -- 172.21.15.64:0/1278505262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc19d350 con 0x7fa5dc102790 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e0fe8640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc106f10 0x7fa5dc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5db7fe640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1988e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0005940 tx=0x7fa5d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d003f030 con 0x7fa5dc102790 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5d0003aa0 con 0x7fa5dc102790 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc19d5e0 con 0x7fa5dc102790 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.829+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0035070 con 0x7fa5dc102790 2024-09-19T05:17:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.830+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc1a4f20 con 0x7fa5dc102790 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.831+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc1032c0 con 0x7fa5dc102790 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.832+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fa5d0042030 con 0x7fa5dc102790 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.833+0000 7fa5d97fa640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa5c40a33b0 0x7fa5c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.833+0000 7fa5dbfff640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa5c40a33b0 0x7fa5c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.833+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fa5d00f0870 con 0x7fa5dc102790 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.833+0000 7fa5dbfff640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa5c40a33b0 0x7fa5c40a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0058c0 tx=0x7fa5cc00a4c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:06.836+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5d00bd1c0 con 0x7fa5dc102790 2024-09-19T05:17:06.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:06 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1940218289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:17:07.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.014+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5dc10a150 con 0x7fa5dc102790 2024-09-19T05:17:07.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.016+0000 7fa5d97fa640 1 -- 172.21.15.64:0/1278505262 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23288 (secure 0 0 0) 0x7fa5d00bcac0 con 0x7fa5dc102790 2024-09-19T05:17:07.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-19T05:17:07.018 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"5ccce474-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:56:23.797274+0000","modified":"2024-09-19T05:16:17.102921+0000","last_up_change":"2024-09-19T05:03:18.395054+0000","last_in_change":"2024-09-19T05:02:55.514054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:59:34.802774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:04:04.880203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:04:06.925227+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52ee9cdd-278a-45e4-8b61-5fa0991e9841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6803","nonce":198612144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6805","nonce":198612144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6809","nonce":198612144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":198612144},{"type":"v1","addr":"172.21.15.64:6807","nonce":198612144}]},"public_addr":"172.21.15.64:6803/198612144","cluster_addr":"172.21.15.64:6805/198612144","heartbeat_back_addr":"172.21.15.64:6809/198612144","heartbeat_front_addr":"172.21.15.64:6807/198612144","state":["exists","up"]},{"osd":1,"uuid":"f802a0ea-cb17-4a5e-85c1-ab9764fd2e94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6801","nonce":1592456147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6803","nonce":1592456147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6807","nonce":1592456147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1592456147},{"type":"v1","addr":"172.21.15.103:6805","nonce":1592456147}]},"public_addr":"172.21.15.103:6801/1592456147","cluster_addr":"172.21.15.103:6803/1592456147","heartbeat_back_addr":"172.21.15.103:6807/1592456147","heartbeat_front_addr":"172.21.15.103:6805/1592456147","state":["exists","up"]},{"osd":2,"uuid":"0be6c1fd-a08a-4e73-904a-fee6cb38adee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6801","nonce":4215230653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6803","nonce":4215230653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6807","nonce":4215230653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4215230653},{"type":"v1","addr":"172.21.15.174:6805","nonce":4215230653}]},"public_addr":"172.21.15.174:6801/4215230653","cluster_addr":"172.21.15.174:6803/4215230653","heartbeat_back_addr":"172.21.15.174:6807/4215230653","heartbeat_front_addr":"172.21.15.174:6805/4215230653","state":["exists","up"]},{"osd":3,"uuid":"62258a68-87a3-4c2e-bc80-49a435c5df82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6811","nonce":973141608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6813","nonce":973141608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6817","nonce":973141608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":973141608},{"type":"v1","addr":"172.21.15.64:6815","nonce":973141608}]},"public_addr":"172.21.15.64:6811/973141608","cluster_addr":"172.21.15.64:6813/973141608","heartbeat_back_addr":"172.21.15.64:6817/973141608","heartbeat_front_addr":"172.21.15.64:6815/973141608","state":["exists","up"]},{"osd":4,"uuid":"9d7e9b07-2d48-4029-8396-fdbae9a82494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6809","nonce":2896874846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6811","nonce":2896874846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6815","nonce":2896874846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2896874846},{"type":"v1","addr":"172.21.15.103:6813","nonce":2896874846}]},"public_addr":"172.21.15.103:6809/2896874846","cluster_addr":"172.21.15.103:6811/2896874846","heartbeat_back_addr":"172.21.15.103:6815/2896874846","heartbeat_front_addr":"172.21.15.103:6813/2896874846","state":["exists","up"]},{"osd":5,"uuid":"93251354-cc4c-4dcf-9536-69770ad07286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6809","nonce":3607820315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6811","nonce":3607820315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6815","nonce":3607820315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3607820315},{"type":"v1","addr":"172.21.15.174:6813","nonce":3607820315}]},"public_addr":"172.21.15.174:6809/3607820315","cluster_addr":"172.21.15.174:6811/3607820315","heartbeat_back_addr":"172.21.15.174:6815/3607820315","heartbeat_front_addr":"172.21.15.174:6813/3607820315","state":["exists","up"]},{"osd":6,"uuid":"346e4af1-6d30-4b90-94c9-e660333548ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6819","nonce":3289767656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6821","nonce":3289767656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6825","nonce":3289767656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3289767656},{"type":"v1","addr":"172.21.15.64:6823","nonce":3289767656}]},"public_addr":"172.21.15.64:6819/3289767656","cluster_addr":"172.21.15.64:6821/3289767656","heartbeat_back_addr":"172.21.15.64:6825/3289767656","heartbeat_front_addr":"172.21.15.64:6823/3289767656","state":["exists","up"]},{"osd":7,"uuid":"29311352-ffd9-4b21-9876-4bb7c8a522b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6817","nonce":944387500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6819","nonce":944387500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6823","nonce":944387500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":944387500},{"type":"v1","addr":"172.21.15.103:6821","nonce":944387500}]},"public_addr":"172.21.15.103:6817/944387500","cluster_addr":"172.21.15.103:6819/944387500","heartbeat_back_addr":"172.21.15.103:6823/944387500","heartbeat_front_addr":"172.21.15.103:6821/944387500","state":["exists","up"]},{"osd":8,"uuid":"ae077168-4b5f-44a0-a269-03f13dba90b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6817","nonce":1095432672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6819","nonce":1095432672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6823","nonce":1095432672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1095432672},{"type":"v1","addr":"172.21.15.174:6821","nonce":1095432672}]},"public_addr":"172.21.15.174:6817/1095432672","cluster_addr":"172.21.15.174:6819/1095432672","heartbeat_back_addr":"172.21.15.174:6823/1095432672","heartbeat_front_addr":"172.21.15.174:6821/1095432672","state":["exists","up"]},{"osd":9,"uuid":"1767cc43-897a-408c-9e10-ecabe9b39731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6827","nonce":1960883919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6829","nonce":1960883919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6833","nonce":1960883919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":1960883919},{"type":"v1","addr":"172.21.15.64:6831","nonce":1960883919}]},"public_addr":"172.21.15.64:6827/1960883919","cluster_addr":"172.21.15.64:6829/1960883919","heartbeat_back_addr":"172.21.15.64:6833/1960883919","heartbeat_front_addr":"172.21.15.64:6831/1960883919","state":["exists","up"]},{"osd":10,"uuid":"b507cd6f-ed66-4b6e-9560-fca660e09350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6825","nonce":1109323153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6827","nonce":1109323153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6831","nonce":1109323153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1109323153},{"type":"v1","addr":"172.21.15.103:6829","nonce":1109323153}]},"public_addr":"172.21.15.103:6825/1109323153","cluster_addr":"172.21.15.103:6827/1109323153","heartbeat_back_addr":"172.21.15.103:6831/1109323153","heartbeat_front_addr":"172.21.15.103:6829/1109323153","state":["exists","up"]},{"osd":11,"uuid":"3169c4e9-d118-46eb-9b6d-b37e181ffa88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6825","nonce":3718770744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6827","nonce":3718770744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6831","nonce":3718770744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":3718770744},{"type":"v1","addr":"172.21.15.174:6829","nonce":3718770744}]},"public_addr":"172.21.15.174:6825/3718770744","cluster_addr":"172.21.15.174:6827/3718770744","heartbeat_back_addr":"172.21.15.174:6831/3718770744","heartbeat_front_addr":"172.21.15.174:6829/3718770744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:41.785832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:07.195790+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:32.504867+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:58.287598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:22.154646+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:46.529507+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:12.202507+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:37.546782+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:02.318073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:27.531447+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:02:52.212904+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:03:16.947178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:6800/3907259856":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/868126904":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/4083716942":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/2000236557":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/3045596814":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/4215744929":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/3048471422":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6800/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:6801/4164984314":"2024-09-20T04:56:54.761876+0000","172.21.15.64:0/1467175216":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/1065736541":"2024-09-20T04:57:18.299954+0000","172.21.15.64:6801/2584640926":"2024-09-20T04:57:18.299954+0000","172.21.15.64:0/2343862077":"2024-09-20T05:05:53.680120+0000","172.21.15.64:0/1076683694":"2024-09-20T05:05:53.680120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa5c40a33b0 msgr2=0x7fa5c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa5c40a33b0 0x7fa5c40a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0058c0 tx=0x7fa5cc00a4c0 comp rx=0 tx=0).stop 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 msgr2=0x7fa5dc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1988e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0005940 tx=0x7fa5d0004270 comp rx=0 tx=0).stop 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 shutdown_connections 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fa5c40a33b0 0x7fa5c40a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc106f10 0x7fa5dc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5dc102790 0x7fa5dc1988e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 --2- 172.21.15.64:0/1278505262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5dc101de0 0x7fa5dc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 >> 172.21.15.64:0/1278505262 conn(0x7fa5dc0fd8c0 msgr2=0x7fa5dc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 shutdown_connections 2024-09-19T05:17:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:07.018+0000 7fa5e2a4b640 1 -- 172.21.15.64:0/1278505262 wait complete. 2024-09-19T05:17:07.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T05:17:07.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:07 smithi174 ceph-mon[34488]: pgmap v347: 97 pgs: 97 active+clean; 19 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:17:07.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:07 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1278505262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:17:07.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:07 smithi064 ceph-mon[30960]: pgmap v347: 97 pgs: 97 active+clean; 19 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:17:07.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:07 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1278505262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:17:07.904 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:07.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:07 smithi103 ceph-mon[34446]: pgmap v347: 97 pgs: 97 active+clean; 19 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:17:07.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:07 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1278505262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:17:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 -- 172.21.15.64:0/1840761046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 msgr2=0x7f784c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 --2- 172.21.15.64:0/1840761046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c10e610 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7844009ca0 tx=0x7f784402a950 comp rx=0 tx=0).stop 2024-09-19T05:17:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 -- 172.21.15.64:0/1840761046 shutdown_connections 2024-09-19T05:17:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 --2- 172.21.15.64:0/1840761046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c10e610 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 --2- 172.21.15.64:0/1840761046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f784c103660 0x7f784c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 --2- 172.21.15.64:0/1840761046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 0x7f784c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 -- 172.21.15.64:0/1840761046 >> 172.21.15.64:0/1840761046 conn(0x7f784c0fc7a0 msgr2=0x7f784c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 -- 172.21.15.64:0/1840761046 shutdown_connections 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.387+0000 7f7851ea6640 1 -- 172.21.15.64:0/1840761046 wait complete. 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.388+0000 7f7851ea6640 1 Processor -- start 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.388+0000 7f7851ea6640 1 -- start start 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.388+0000 7f7851ea6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 0x7f784c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.388+0000 7f7851ea6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f784c103660 0x7f784c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.388+0000 7f7851ea6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.388+0000 7f7851ea6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f784c111e20 con 0x7f784c10c170 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f7851ea6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f784c111ca0 con 0x7f784c103660 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f7851ea6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f784c111fa0 con 0x7f784c102d40 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41548/0 (socket says 172.21.15.64:41548) 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 -- 172.21.15.64:0/3727356743 learned_addr learned my addr 172.21.15.64:0/3727356743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784affd640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f784c103660 0x7f784c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784b7fe640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 0x7f784c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 -- 172.21.15.64:0/3727356743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 msgr2=0x7f784c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 0x7f784c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 -- 172.21.15.64:0/3727356743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f784c103660 msgr2=0x7f784c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f784c103660 0x7f784c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 -- 172.21.15.64:0/3727356743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c19d270 con 0x7f784c10c170 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784b7fe640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 0x7f784c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f784bfff640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c19cb90 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f784402ae70 tx=0x7f784402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844034040 con 0x7f784c10c170 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78440308f0 con 0x7f784c10c170 2024-09-19T05:17:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c19d500 con 0x7f784c10c170 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.389+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440047f0 con 0x7f784c10c170 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.390+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1a4e40 con 0x7f784c10c170 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.390+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c06b070 con 0x7f784c10c170 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.392+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f7844037020 con 0x7f784c10c170 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.392+0000 7f7848ff9640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78340a3430 0x7f78340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.393+0000 7f784b7fe640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78340a3430 0x7f78340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.393+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f78440f0d60 con 0x7f784c10c170 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.393+0000 7f784b7fe640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78340a3430 0x7f78340a58f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f783c0058c0 tx=0x7f783c005850 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.396+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7844081210 con 0x7f784c10c170 2024-09-19T05:17:08.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.609+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f784c1a5070 con 0x7f784c10c170 2024-09-19T05:17:08.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.611+0000 7f7848ff9640 1 -- 172.21.15.64:0/3727356743 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f78440bd820 con 0x7f784c10c170 2024-09-19T05:17:08.613 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T05:17:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.614+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78340a3430 msgr2=0x7f78340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.614+0000 7f7851ea6640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78340a3430 0x7f78340a58f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f783c0058c0 tx=0x7f783c005850 comp rx=0 tx=0).stop 2024-09-19T05:17:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.614+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 msgr2=0x7f784c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:08.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.614+0000 7f7851ea6640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c19cb90 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f784402ae70 tx=0x7f784402f5d0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 shutdown_connections 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f78340a3430 0x7f78340a58f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f784c10c170 0x7f784c19cb90 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f784c103660 0x7f784c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 --2- 172.21.15.64:0/3727356743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f784c102d40 0x7f784c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 >> 172.21.15.64:0/3727356743 conn(0x7f784c0fc7a0 msgr2=0x7f784c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 shutdown_connections 2024-09-19T05:17:08.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:08.615+0000 7f7851ea6640 1 -- 172.21.15.64:0/3727356743 wait complete. 2024-09-19T05:17:08.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a[30956]: 2024-09-19T05:17:08.610+0000 7f46d048d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:17:08.969 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: ignoring --setuser ceph since I am not root 2024-09-19T05:17:08.969 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: ignoring --setgroup ceph since I am not root 2024-09-19T05:17:08.969 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-mds[64581]: -- 172.21.15.103:0/142962023 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ada905c20 con 0x558ada8e3000 2024-09-19T05:17:08.969 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: starting mds.b at 2024-09-19T05:17:08.970 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h[64272]: ignoring --setuser ceph since I am not root 2024-09-19T05:17:08.970 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h[64272]: ignoring --setgroup ceph since I am not root 2024-09-19T05:17:08.970 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-mds[64276]: -- 172.21.15.103:0/1694357627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa33cbe680 con 0x55fa33c93000 2024-09-19T05:17:08.970 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-mds[64276]: -- 172.21.15.103:0/1694357627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa33cb5860 con 0x55fa33c93000 2024-09-19T05:17:08.970 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h[64272]: starting mds.h at 2024-09-19T05:17:08.970 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e[63965]: ignoring --setuser ceph since I am not root 2024-09-19T05:17:08.971 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e[63965]: ignoring --setgroup ceph since I am not root 2024-09-19T05:17:08.971 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-mds[63969]: -- 172.21.15.103:0/2091375079 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e85a0f7c20 con 0x55e85a0d4800 2024-09-19T05:17:08.971 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:17:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e[63965]: starting mds.e at 2024-09-19T05:17:09.028 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c[64562]: ignoring --setuser ceph since I am not root 2024-09-19T05:17:09.028 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c[64562]: ignoring --setgroup ceph since I am not root 2024-09-19T05:17:09.028 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-mds[64566]: -- 172.21.15.174:0/2322857151 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644a584d860 con 0x5644a582a800 2024-09-19T05:17:09.028 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c[64562]: starting mds.c at 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f[64262]: ignoring --setuser ceph since I am not root 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f[64262]: ignoring --setgroup ceph since I am not root 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-mds[64266]: -- 172.21.15.174:0/3254027606 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ff0009c20 con 0x564feffe6800 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f[64262]: starting mds.f at 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i[64864]: ignoring --setuser ceph since I am not root 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i[64864]: ignoring --setgroup ceph since I am not root 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-mds[64868]: -- 172.21.15.174:0/1905751310 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba1f7e7860 con 0x55ba1f7c4800 2024-09-19T05:17:09.029 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:17:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i[64864]: starting mds.i at 2024-09-19T05:17:09.246 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T05:17:09.507 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: pgmap v348: 97 pgs: 97 active+clean; 19 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-19T05:17:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3727356743' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:17:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:17:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:17:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/3727356743' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:17:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:17:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:09 smithi064 ceph-mon[30960]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T05:17:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: pgmap v348: 97 pgs: 97 active+clean; 19 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-19T05:17:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3727356743' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:17:09.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:17:09.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:17:09.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/3727356743' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:17:09.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:17:09.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:09 smithi174 ceph-mon[34488]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: pgmap v348: 97 pgs: 97 active+clean; 19 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3727356743' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/3727356743' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:17:09.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:09 smithi103 ceph-mon[34446]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T05:17:09.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 -- 172.21.15.64:0/1102165763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97181082a0 msgr2=0x7f971810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 --2- 172.21.15.64:0/1102165763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97181082a0 0x7f971810ec40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9710009cd0 tx=0x7f971002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 -- 172.21.15.64:0/1102165763 shutdown_connections 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 --2- 172.21.15.64:0/1102165763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97181082a0 0x7f971810ec40 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 --2- 172.21.15.64:0/1102165763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97181036e0 0x7f9718107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 --2- 172.21.15.64:0/1102165763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9718102d30 0x7f9718103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 -- 172.21.15.64:0/1102165763 >> 172.21.15.64:0/1102165763 conn(0x7f97180fc7d0 msgr2=0x7f97180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 -- 172.21.15.64:0/1102165763 shutdown_connections 2024-09-19T05:17:09.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.988+0000 7f971fab3640 1 -- 172.21.15.64:0/1102165763 wait complete. 2024-09-19T05:17:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 Processor -- start 2024-09-19T05:17:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 -- start start 2024-09-19T05:17:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 0x7f9718100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97181036e0 0x7f9718100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97181082a0 0x7f9718101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:09.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9718111f50 con 0x7f9718102d30 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9718111dd0 con 0x7f97181082a0 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971fab3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97181120d0 con 0x7f97181036e0 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971d84f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 0x7f9718100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971d84f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 0x7f9718100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41554/0 (socket says 172.21.15.64:41554) 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971d84f640 1 -- 172.21.15.64:0/1997842055 learned_addr learned my addr 172.21.15.64:0/1997842055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971d04e640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97181036e0 0x7f9718100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.989+0000 7f971e050640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97181082a0 0x7f9718101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971d84f640 1 -- 172.21.15.64:0/1997842055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97181036e0 msgr2=0x7f9718100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971d84f640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97181036e0 0x7f9718100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971d84f640 1 -- 172.21.15.64:0/1997842055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97181082a0 msgr2=0x7f9718101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971d84f640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97181082a0 0x7f9718101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:09.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971d84f640 1 -- 172.21.15.64:0/1997842055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97181a6960 con 0x7f9718102d30 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971e050640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97181082a0 0x7f9718101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971d84f640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 0x7f9718100990 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f971400d950 tx=0x7f971400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714017070 con 0x7f9718102d30 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9714009c30 con 0x7f9718102d30 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400f620 con 0x7f9718102d30 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97181a6c50 con 0x7f9718102d30 2024-09-19T05:17:09.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.990+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a7190 con 0x7f9718102d30 2024-09-19T05:17:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.991+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718103110 con 0x7f9718102d30 2024-09-19T05:17:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.992+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f97140109a0 con 0x7f9718102d30 2024-09-19T05:17:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.993+0000 7f970effd640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f97040a3430 0x7f97040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.993+0000 7f971d04e640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f97040a3430 0x7f97040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.993+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8873+0+0 (secure 0 0 0) 0x7f97140cf9f0 con 0x7f9718102d30 2024-09-19T05:17:09.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.994+0000 7f971d04e640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f97040a3430 0x7f97040a58f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9718102420 tx=0x7f9708005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:09.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:09.996+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9714009de0 con 0x7f9718102d30 2024-09-19T05:17:10.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.211+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f971810a2b0 con 0x7f9718102d30 2024-09-19T05:17:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.618+0000 7f970effd640 1 -- 172.21.15.64:0/1997842055 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f971409c2f0 con 0x7f9718102d30 2024-09-19T05:17:10.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f97040a3430 msgr2=0x7f97040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:10.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f97040a3430 0x7f97040a58f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9718102420 tx=0x7f9708005f90 comp rx=0 tx=0).stop 2024-09-19T05:17:10.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 msgr2=0x7f9718100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:10.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 0x7f9718100990 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f971400d950 tx=0x7f971400de10 comp rx=0 tx=0).stop 2024-09-19T05:17:10.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 shutdown_connections 2024-09-19T05:17:10.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f97040a3430 0x7f97040a58f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97181082a0 0x7f9718101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97181036e0 0x7f9718100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 --2- 172.21.15.64:0/1997842055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9718102d30 0x7f9718100990 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 >> 172.21.15.64:0/1997842055 conn(0x7f97180fc7d0 msgr2=0x7f9718105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 shutdown_connections 2024-09-19T05:17:10.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:10.621+0000 7f971fab3640 1 -- 172.21.15.64:0/1997842055 wait complete. 2024-09-19T05:17:10.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6834/2897687059,v1:172.21.15.174:6835/2897687059] up:boot 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6835/4047835573,v1:172.21.15.103:6838/4047835573] up:boot 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6834/3977057206,v1:172.21.15.103:6837/3977057206] up:boot 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.103:6836/1905349349,v1:172.21.15.103:6839/1905349349] up:boot 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6836/3040659557,v1:172.21.15.174:6837/3040659557] up:boot 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: mds.? [v2:172.21.15.174:6838/2405280649,v1:172.21.15.174:6839/2405280649] up:boot 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:17:10.901 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:17:10.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:17:10.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:17:10.902 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:10 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1997842055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:17:10.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6834/2897687059,v1:172.21.15.174:6835/2897687059] up:boot 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6835/4047835573,v1:172.21.15.103:6838/4047835573] up:boot 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6834/3977057206,v1:172.21.15.103:6837/3977057206] up:boot 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.103:6836/1905349349,v1:172.21.15.103:6839/1905349349] up:boot 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6836/3040659557,v1:172.21.15.174:6837/3040659557] up:boot 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: mds.? [v2:172.21.15.174:6838/2405280649,v1:172.21.15.174:6839/2405280649] up:boot 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:17:10.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:17:10.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:17:10.970 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:10 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1997842055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:17:11.027 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6834/2897687059,v1:172.21.15.174:6835/2897687059] up:boot 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6835/4047835573,v1:172.21.15.103:6838/4047835573] up:boot 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6834/3977057206,v1:172.21.15.103:6837/3977057206] up:boot 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.103:6836/1905349349,v1:172.21.15.103:6839/1905349349] up:boot 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6836/3040659557,v1:172.21.15.174:6837/3040659557] up:boot 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: mds.? [v2:172.21.15.174:6838/2405280649,v1:172.21.15.174:6839/2405280649] up:boot 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:17:11.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:17:11.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:10 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1997842055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:17:11.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T05:17:11.609 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:11.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: pgmap v350: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:17:11.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:17:11.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: Cluster is now healthy 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/1997842055' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: fsmap 12 up:standby 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:17:11.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:11 smithi064 ceph-mon[30960]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: pgmap v350: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: Cluster is now healthy 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/1997842055' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: fsmap 12 up:standby 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:17:11.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:11 smithi103 ceph-mon[34446]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: pgmap v350: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: Cluster is now healthy 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/1997842055' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: fsmap 12 up:standby 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:17:12.028 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:17:12.029 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:11 smithi174 ceph-mon[34488]: from='mgr.15126 172.21.15.64:0/2466030823' entity='mgr.x' 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 -- 172.21.15.64:0/1353583909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04100a80 msgr2=0x7f6a04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 --2- 172.21.15.64:0/1353583909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04100a80 0x7f6a04100ee0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009930 tx=0x7f69f802a420 comp rx=0 tx=0).stop 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 -- 172.21.15.64:0/1353583909 shutdown_connections 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 --2- 172.21.15.64:0/1353583909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a04101420 0x7f6a0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 --2- 172.21.15.64:0/1353583909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04100a80 0x7f6a04100ee0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 --2- 172.21.15.64:0/1353583909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04106a80 0x7f6a04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:12.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 -- 172.21.15.64:0/1353583909 >> 172.21.15.64:0/1353583909 conn(0x7f6a040fc7a0 msgr2=0x7f6a040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:12.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 -- 172.21.15.64:0/1353583909 shutdown_connections 2024-09-19T05:17:12.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.088+0000 7f6a09748640 1 -- 172.21.15.64:0/1353583909 wait complete. 2024-09-19T05:17:12.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 Processor -- start 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 -- start start 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04100a80 0x7f6a040fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 0x7f6a040fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a04106a80 0x7f6a040fd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04110d30 con 0x7f6a04101420 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04110bb0 con 0x7f6a04100a80 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a09748640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04110eb0 con 0x7f6a04106a80 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 0x7f6a040fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a027fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 0x7f6a040fd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41580/0 (socket says 172.21.15.64:41580) 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a027fc640 1 -- 172.21.15.64:0/738471754 learned_addr learned my addr 172.21.15.64:0/738471754 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.089+0000 7f6a02ffd640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04100a80 0x7f6a040fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:12.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a037fe640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a04106a80 0x7f6a040fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a027fc640 1 -- 172.21.15.64:0/738471754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a04106a80 msgr2=0x7f6a040fd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a027fc640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a04106a80 0x7f6a040fd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a027fc640 1 -- 172.21.15.64:0/738471754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04100a80 msgr2=0x7f6a040fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a027fc640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04100a80 0x7f6a040fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a027fc640 1 -- 172.21.15.64:0/738471754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041a6860 con 0x7f6a04101420 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a027fc640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 0x7f6a040fd3e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005940 tx=0x7f69f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f803f030 con 0x7f6a04101420 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041a6af0 con 0x7f6a04101420 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69f80318a0 con 0x7f6a04101420 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035070 con 0x7f6a04101420 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.090+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041a7020 con 0x7f6a04101420 2024-09-19T05:17:12.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.091+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a04100ee0 con 0x7f6a04101420 2024-09-19T05:17:12.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.092+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7f69f8042030 con 0x7f6a04101420 2024-09-19T05:17:12.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.093+0000 7f69ebfff640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f69ec0a33b0 0x7f69ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:12.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.093+0000 7f6a02ffd640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f69ec0a33b0 0x7f69ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:12.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.093+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8873+0+0 (secure 0 0 0) 0x7f69f80f0210 con 0x7f6a04101420 2024-09-19T05:17:12.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.094+0000 7f6a02ffd640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f69ec0a33b0 0x7f69ec0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009b50 tx=0x7f69f4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:12.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.097+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69f80bca90 con 0x7f6a04101420 2024-09-19T05:17:12.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:12.276+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a0410a000 con 0x7f6a04101420 2024-09-19T05:17:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.120+0000 7f69ebfff640 1 -- 172.21.15.64:0/738471754 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f69f80bc390 con 0x7f6a04101420 2024-09-19T05:17:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-09-19T05:17:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f69ec0a33b0 msgr2=0x7f69ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f69ec0a33b0 0x7f69ec0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009b50 tx=0x7f69f4005f90 comp rx=0 tx=0).stop 2024-09-19T05:17:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 msgr2=0x7f6a040fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 0x7f6a040fd3e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005940 tx=0x7f69f80042f0 comp rx=0 tx=0).stop 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 shutdown_connections 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7f69ec0a33b0 0x7f69ec0a5870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a04106a80 0x7f6a040fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a04101420 0x7f6a040fd3e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 --2- 172.21.15.64:0/738471754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a04100a80 0x7f6a040fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.123+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 >> 172.21.15.64:0/738471754 conn(0x7f6a040fc7a0 msgr2=0x7f6a0406ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.124+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 shutdown_connections 2024-09-19T05:17:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:13.124+0000 7f6a09748640 1 -- 172.21.15.64:0/738471754 wait complete. 2024-09-19T05:17:13.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:13 smithi064 ceph-mon[30960]: pgmap v351: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:17:13.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:13 smithi064 ceph-mon[30960]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:17:13.150 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:13 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/738471754' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:17:13.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:13 smithi174 ceph-mon[34488]: pgmap v351: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:17:13.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:13 smithi174 ceph-mon[34488]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:17:13.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:13 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/738471754' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:17:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:13 smithi103 ceph-mon[34446]: pgmap v351: 97 pgs: 97 active+clean; 19 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-19T05:17:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:13 smithi103 ceph-mon[34446]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:17:13.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:13 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/738471754' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:17:13.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 5ccce474-7643-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T05:17:13.956 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:17:14.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:14 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/738471754' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:17:14.389 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:14 smithi064 ceph-mon[30960]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:17:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 -- 172.21.15.64:0/1231154650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc103660 msgr2=0x7fc7fc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 --2- 172.21.15.64:0/1231154650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc103660 0x7fc7fc10bb50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009900 tx=0x7fc7f002a480 comp rx=0 tx=0).stop 2024-09-19T05:17:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 -- 172.21.15.64:0/1231154650 shutdown_connections 2024-09-19T05:17:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 --2- 172.21.15.64:0/1231154650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 0x7fc7fc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 --2- 172.21.15.64:0/1231154650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc103660 0x7fc7fc10bb50 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 --2- 172.21.15.64:0/1231154650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7fc102d40 0x7fc7fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 -- 172.21.15.64:0/1231154650 >> 172.21.15.64:0/1231154650 conn(0x7fc7fc0fc7a0 msgr2=0x7fc7fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 -- 172.21.15.64:0/1231154650 shutdown_connections 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.387+0000 7fc80296b640 1 -- 172.21.15.64:0/1231154650 wait complete. 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.388+0000 7fc80296b640 1 Processor -- start 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.388+0000 7fc80296b640 1 -- start start 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.388+0000 7fc80296b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.388+0000 7fc80296b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7fc103660 0x7fc7fc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.388+0000 7fc80296b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 0x7fc7fc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:14.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc80296b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc111e60 con 0x7fc7fc102d40 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc80296b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc111ce0 con 0x7fc7fc103660 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc80296b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc111fe0 con 0x7fc7fc10c170 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57680/0 (socket says 172.21.15.64:57680) 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 -- 172.21.15.64:0/969656380 learned_addr learned my addr 172.21.15.64:0/969656380 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fb7fe640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7fc103660 0x7fc7fc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc800f08640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 0x7fc7fc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 -- 172.21.15.64:0/969656380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 msgr2=0x7fc7fc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 0x7fc7fc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 -- 172.21.15.64:0/969656380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7fc103660 msgr2=0x7fc7fc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7fc103660 0x7fc7fc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:14.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 -- 172.21.15.64:0/969656380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc1a68b0 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc800f08640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 0x7fc7fc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7fbfff640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1009a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c890 tx=0x7fc7ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec004220 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7ec0043c0 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc1a6ba0 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec004a00 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.389+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc1a70b0 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.390+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7fc103120 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.392+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138486+0+0 (secure 0 0 0) 0x7fc7ec004ba0 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.392+0000 7fc7f97fa640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc7e40a33b0 0x7fc7e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.393+0000 7fc7fb7fe640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc7e40a33b0 0x7fc7e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.393+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8231+0+0 (secure 0 0 0) 0x7fc7ec0cfcd0 con 0x7fc7fc102d40 2024-09-19T05:17:14.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.393+0000 7fc7fb7fe640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc7e40a33b0 0x7fc7e40a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0031440 tx=0x7fc7f0005e80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:17:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.396+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc7ec014030 con 0x7fc7fc102d40 2024-09-19T05:17:14.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:14 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/738471754' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:17:14.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:14 smithi103 ceph-mon[34446]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:17:14.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:14 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/738471754' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:17:14.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:14 smithi174 ceph-mon[34488]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:17:14.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:14.573+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc7fc101ed0 con 0x7fc7fc102d40 2024-09-19T05:17:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.140+0000 7fc7f97fa640 1 -- 172.21.15.64:0/969656380 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fc7ec09c7d0 con 0x7fc7fc102d40 2024-09-19T05:17:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-09-19T05:17:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc7e40a33b0 msgr2=0x7fc7e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc7e40a33b0 0x7fc7e40a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0031440 tx=0x7fc7f0005e80 comp rx=0 tx=0).stop 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 msgr2=0x7fc7fc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1009a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c890 tx=0x7fc7ec00cd50 comp rx=0 tx=0).stop 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 shutdown_connections 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:6800/2647680663,v1:172.21.15.64:6801/2647680663] conn(0x7fc7e40a33b0 0x7fc7e40a5870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7fc10c170 0x7fc7fc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7fc103660 0x7fc7fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 --2- 172.21.15.64:0/969656380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7fc102d40 0x7fc7fc1009a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.145+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 >> 172.21.15.64:0/969656380 conn(0x7fc7fc0fc7a0 msgr2=0x7fc7fc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.146+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 shutdown_connections 2024-09-19T05:17:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-19T05:17:15.146+0000 7fc80296b640 1 -- 172.21.15.64:0/969656380 wait complete. 2024-09-19T05:17:15.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:15 smithi064 ceph-mon[30960]: pgmap v354: 65 pgs: 65 active+clean; 577 KiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:17:15.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:15 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/969656380' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:17:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:15 smithi103 ceph-mon[34446]: pgmap v354: 65 pgs: 65 active+clean; 577 KiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:17:15.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:15 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/969656380' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:17:15.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:15 smithi174 ceph-mon[34488]: pgmap v354: 65 pgs: 65 active+clean; 577 KiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:17:15.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:15 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/969656380' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:17:15.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:17:15.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:17:15.876 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:17:15.910 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T05:17:15.910 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:17:15.927 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:17:15.944 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:17:15.967 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T05:17:15.967 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T05:17:15.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a 2024-09-19T05:17:16.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 systemd[1]: Stopping Ceph mon.a for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:16.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:16 smithi174 ceph-mon[34488]: from='client.? 172.21.15.64:0/969656380' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:17:16.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:16 smithi174 ceph-mon[34488]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:17:16.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 ceph-mon[30960]: from='client.? 172.21.15.64:0/969656380' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:17:16.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 ceph-mon[30960]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:17:16.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a[30956]: 2024-09-19T05:17:16.309+0000 7f46d5c98640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:16.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a[30956]: 2024-09-19T05:17:16.309+0000 7f46d5c98640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T05:17:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:16 smithi103 ceph-mon[34446]: from='client.? 172.21.15.64:0/969656380' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:17:16.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:16 smithi103 ceph-mon[34446]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:17:16.786 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 podman[108005]: 2024-09-19 05:17:16.513903616 +0000 UTC m=+0.347341820 container died 32af6c18b4dd9417c414d715f79d9f990376f0ac82e0c17d63fccf098299ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T05:17:17.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 podman[108005]: 2024-09-19 05:17:16.867851209 +0000 UTC m=+0.701289413 container cleanup 32af6c18b4dd9417c414d715f79d9f990376f0ac82e0c17d63fccf098299ebb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T05:17:17.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 19 05:17:16 smithi064 bash[108005]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-a 2024-09-19T05:17:17.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T05:17:17.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:17.314 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T05:17:17.314 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T05:17:17.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.b 2024-09-19T05:17:17.653 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:17 smithi103 systemd[1]: Stopping Ceph mon.b for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:17.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-b[34442]: 2024-09-19T05:17:17.651+0000 7f3d1e01b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:17.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-b[34442]: 2024-09-19T05:17:17.651+0000 7f3d1e01b640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T05:17:17.969 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:17 smithi103 podman[69881]: 2024-09-19 05:17:17.783838749 +0000 UTC m=+0.273968693 container died 42021eb22ec0df7197a9f30329c7b902643072ad153e6e54a97cae111083685e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T05:17:18.419 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:18 smithi103 podman[69881]: 2024-09-19 05:17:18.175717743 +0000 UTC m=+0.665847687 container cleanup 42021eb22ec0df7197a9f30329c7b902643072ad153e6e54a97cae111083685e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:18.419 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 19 05:17:18 smithi103 bash[69881]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-b 2024-09-19T05:17:18.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T05:17:18.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:18.615 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T05:17:18.615 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T05:17:18.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.c 2024-09-19T05:17:18.927 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:18 smithi174 systemd[1]: Stopping Ceph mon.c for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:19.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c[34484]: 2024-09-19T05:17:18.926+0000 7fc6002da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c[34484]: 2024-09-19T05:17:18.926+0000 7fc6002da640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T05:17:19.278 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:19 smithi174 podman[70353]: 2024-09-19 05:17:19.089397663 +0000 UTC m=+0.275614087 container died fdede484ad090cb2c23c21a25730cdb4492852dd2dca79529d0307ca55ef489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T05:17:19.620 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:19 smithi174 podman[70353]: 2024-09-19 05:17:19.3590767 +0000 UTC m=+0.545293124 container cleanup fdede484ad090cb2c23c21a25730cdb4492852dd2dca79529d0307ca55ef489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T05:17:19.620 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 05:17:19 smithi174 bash[70353]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mon-c 2024-09-19T05:17:19.782 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T05:17:19.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:19.842 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T05:17:19.842 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T05:17:19.843 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x 2024-09-19T05:17:20.150 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:19 smithi064 systemd[1]: Stopping Ceph mgr.x for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:20.500 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:17:20.180+0000 7fdc9d09f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:20.500 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x[31190]: 2024-09-19T05:17:20.180+0000 7fdc9d09f640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-19T05:17:20.500 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:20 smithi064 podman[108126]: 2024-09-19 05:17:20.211398062 +0000 UTC m=+0.166797373 container died da6dc243a6d36ede172c0a929aae5e93e272c42f527f7322440ff71103d03a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:17:20.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:20 smithi064 podman[108126]: 2024-09-19 05:17:20.606066247 +0000 UTC m=+0.561465548 container cleanup da6dc243a6d36ede172c0a929aae5e93e272c42f527f7322440ff71103d03a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T05:17:20.902 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:20 smithi064 bash[108126]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x 2024-09-19T05:17:21.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T05:17:21.223 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:20 smithi064 podman[108138]: 2024-09-19 05:17:20.966116395 +0000 UTC m=+0.747791095 container remove da6dc243a6d36ede172c0a929aae5e93e272c42f527f7322440ff71103d03a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:17:21.223 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:21 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-19T05:17:21.223 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:21 smithi064 systemd[1]: Stopped Ceph mgr.x for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:17:21.223 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 19 05:17:21 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.x.service: Consumed 56.623s CPU time. 2024-09-19T05:17:21.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:21.236 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T05:17:21.236 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T05:17:21.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.y 2024-09-19T05:17:21.593 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:17:21 smithi103 systemd[1]: Stopping Ceph mgr.y for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:21.881 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:17:21 smithi103 podman[70001]: 2024-09-19 05:17:21.592678445 +0000 UTC m=+0.157299950 container died 445aed7eeb37a82c0726fa5453b00ec472562c6b16d4cac1b1cde825ef45612f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-19T05:17:22.219 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:17:21 smithi103 podman[70001]: 2024-09-19 05:17:21.963477763 +0000 UTC m=+0.528099266 container cleanup 445aed7eeb37a82c0726fa5453b00ec472562c6b16d4cac1b1cde825ef45612f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T05:17:22.219 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 19 05:17:21 smithi103 bash[70001]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-y 2024-09-19T05:17:22.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T05:17:22.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:22.432 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T05:17:22.432 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T05:17:22.432 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.z 2024-09-19T05:17:22.777 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:17:22 smithi174 systemd[1]: Stopping Ceph mgr.z for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:23.049 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:17:22 smithi174 podman[70473]: 2024-09-19 05:17:22.778444393 +0000 UTC m=+0.146941051 container died a3c9f68e21d3f7ec2302928f88f5677015a76863fab310941c740aa35dd78a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T05:17:23.416 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:17:23 smithi174 podman[70473]: 2024-09-19 05:17:23.155262796 +0000 UTC m=+0.523759453 container cleanup a3c9f68e21d3f7ec2302928f88f5677015a76863fab310941c740aa35dd78a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:17:23.416 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 05:17:23 smithi174 bash[70473]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mgr-z 2024-09-19T05:17:23.585 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T05:17:23.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:23.621 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T05:17:23.621 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T05:17:23.621 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.0 2024-09-19T05:17:23.900 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:23 smithi064 systemd[1]: Stopping Ceph osd.0 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:24.401 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0[42571]: 2024-09-19T05:17:23.943+0000 7f12277f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:24.401 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0[42571]: 2024-09-19T05:17:23.943+0000 7f12277f9640 -1 osd.0 92 *** Got signal Terminated *** 2024-09-19T05:17:24.401 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0[42571]: 2024-09-19T05:17:23.943+0000 7f12277f9640 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:29.651 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:29 smithi064 podman[108247]: 2024-09-19 05:17:29.354104897 +0000 UTC m=+5.529672826 container died c06d67b8a2f0421202ea9e5c13d55c2d08ad7ee9258ce4a61144a6a82d64d15d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T05:17:30.150 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:29 smithi064 podman[108247]: 2024-09-19 05:17:29.852449016 +0000 UTC m=+6.028016944 container cleanup c06d67b8a2f0421202ea9e5c13d55c2d08ad7ee9258ce4a61144a6a82d64d15d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T05:17:30.150 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:29 smithi064 bash[108247]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0 2024-09-19T05:17:30.151 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108263]: 2024-09-19 05:17:30.087436252 +0000 UTC m=+0.730199040 container remove c06d67b8a2f0421202ea9e5c13d55c2d08ad7ee9258ce4a61144a6a82d64d15d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T05:17:30.733 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108352]: 2024-09-19 05:17:30.309704114 +0000 UTC m=+0.016733673 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:30.734 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108352]: 2024-09-19 05:17:30.421528423 +0000 UTC m=+0.128558008 container create 0f8dcb77a1bf021022a33e16654604cb7b10c4867f19ed1434e74fb40514ee3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T05:17:30.734 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108352]: 2024-09-19 05:17:30.631985254 +0000 UTC m=+0.339014811 container init 0f8dcb77a1bf021022a33e16654604cb7b10c4867f19ed1434e74fb40514ee3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T05:17:30.734 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108352]: 2024-09-19 05:17:30.635707775 +0000 UTC m=+0.342737318 container start 0f8dcb77a1bf021022a33e16654604cb7b10c4867f19ed1434e74fb40514ee3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T05:17:30.734 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108352]: 2024-09-19 05:17:30.731991902 +0000 UTC m=+0.439021451 container attach 0f8dcb77a1bf021022a33e16654604cb7b10c4867f19ed1434e74fb40514ee3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:17:31.150 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 19 05:17:30 smithi064 podman[108352]: 2024-09-19 05:17:30.830236841 +0000 UTC m=+0.537266399 container died 0f8dcb77a1bf021022a33e16654604cb7b10c4867f19ed1434e74fb40514ee3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T05:17:31.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T05:17:31.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:31.585 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T05:17:31.585 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T05:17:31.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.1 2024-09-19T05:17:31.894 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:31 smithi103 systemd[1]: Stopping Ceph osd.1 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:32.218 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1[38910]: 2024-09-19T05:17:31.891+0000 7f97b3bc2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:32.219 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1[38910]: 2024-09-19T05:17:31.891+0000 7f97b3bc2640 -1 osd.1 92 *** Got signal Terminated *** 2024-09-19T05:17:32.219 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1[38910]: 2024-09-19T05:17:31.891+0000 7f97b3bc2640 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:37.713 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:37 smithi103 podman[70127]: 2024-09-19 05:17:37.404362509 +0000 UTC m=+5.618861605 container died bcdd96ab11c299eb4e95f54383961d3d2ea08d93438bcd51c71be72d312b1f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T05:17:37.969 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:37 smithi103 podman[70127]: 2024-09-19 05:17:37.818124798 +0000 UTC m=+6.032623893 container cleanup bcdd96ab11c299eb4e95f54383961d3d2ea08d93438bcd51c71be72d312b1f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T05:17:37.969 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:37 smithi103 bash[70127]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1 2024-09-19T05:17:38.469 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70142]: 2024-09-19 05:17:38.069843105 +0000 UTC m=+0.664762427 container remove bcdd96ab11c299eb4e95f54383961d3d2ea08d93438bcd51c71be72d312b1f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T05:17:38.807 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70235]: 2024-09-19 05:17:38.392145828 +0000 UTC m=+0.016217205 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:38.807 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70235]: 2024-09-19 05:17:38.512194888 +0000 UTC m=+0.136266263 container create 2d49db96b0cdc0026fafb41c57f99ede59ff6c198f47467f26b888044c2cfc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T05:17:38.807 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70235]: 2024-09-19 05:17:38.731195944 +0000 UTC m=+0.355267336 container init 2d49db96b0cdc0026fafb41c57f99ede59ff6c198f47467f26b888044c2cfc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:38.807 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70235]: 2024-09-19 05:17:38.734565856 +0000 UTC m=+0.358637241 container start 2d49db96b0cdc0026fafb41c57f99ede59ff6c198f47467f26b888044c2cfc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T05:17:38.807 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70235]: 2024-09-19 05:17:38.80609551 +0000 UTC m=+0.430166902 container attach 2d49db96b0cdc0026fafb41c57f99ede59ff6c198f47467f26b888044c2cfc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T05:17:39.210 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 19 05:17:38 smithi103 podman[70235]: 2024-09-19 05:17:38.920899943 +0000 UTC m=+0.544971317 container died 2d49db96b0cdc0026fafb41c57f99ede59ff6c198f47467f26b888044c2cfc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:17:39.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T05:17:39.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:39.553 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T05:17:39.553 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T05:17:39.553 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.2 2024-09-19T05:17:40.011 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:39 smithi174 systemd[1]: Stopping Ceph osd.2 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:40.277 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2[39445]: 2024-09-19T05:17:40.009+0000 7f4c0f554640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:40.278 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2[39445]: 2024-09-19T05:17:40.009+0000 7f4c0f554640 -1 osd.2 92 *** Got signal Terminated *** 2024-09-19T05:17:40.278 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2[39445]: 2024-09-19T05:17:40.009+0000 7f4c0f554640 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:45.603 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:45 smithi174 podman[70601]: 2024-09-19 05:17:45.331053036 +0000 UTC m=+5.588058410 container died b9d498faffd627259b038821705e7013c6618c2bcff0e8b22171baa2758fdf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T05:17:45.970 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:45 smithi174 podman[70601]: 2024-09-19 05:17:45.692315493 +0000 UTC m=+5.949320858 container cleanup b9d498faffd627259b038821705e7013c6618c2bcff0e8b22171baa2758fdf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T05:17:45.970 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:45 smithi174 bash[70601]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2 2024-09-19T05:17:46.277 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:45 smithi174 podman[70630]: 2024-09-19 05:17:45.968588471 +0000 UTC m=+0.637969572 container remove b9d498faffd627259b038821705e7013c6618c2bcff0e8b22171baa2758fdf41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:46.572 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:46 smithi174 podman[70717]: 2024-09-19 05:17:46.220080538 +0000 UTC m=+0.016137435 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:46.572 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:46 smithi174 podman[70717]: 2024-09-19 05:17:46.352772261 +0000 UTC m=+0.148829200 container create 1bbd6fea5fffa760ba0bd1eb375e3118bc8b8d15d3b213d511b76a529e2703b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T05:17:47.028 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:46 smithi174 podman[70717]: 2024-09-19 05:17:46.571623218 +0000 UTC m=+0.367680113 container init 1bbd6fea5fffa760ba0bd1eb375e3118bc8b8d15d3b213d511b76a529e2703b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:17:47.028 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:46 smithi174 podman[70717]: 2024-09-19 05:17:46.575035901 +0000 UTC m=+0.371092798 container start 1bbd6fea5fffa760ba0bd1eb375e3118bc8b8d15d3b213d511b76a529e2703b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:17:47.028 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:46 smithi174 podman[70717]: 2024-09-19 05:17:46.679903899 +0000 UTC m=+0.475960804 container attach 1bbd6fea5fffa760ba0bd1eb375e3118bc8b8d15d3b213d511b76a529e2703b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T05:17:47.028 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:46 smithi174 podman[70717]: 2024-09-19 05:17:46.770739539 +0000 UTC m=+0.566796440 container died 1bbd6fea5fffa760ba0bd1eb375e3118bc8b8d15d3b213d511b76a529e2703b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T05:17:47.320 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 05:17:47 smithi174 podman[70740]: 2024-09-19 05:17:47.316606675 +0000 UTC m=+0.539849941 container remove 1bbd6fea5fffa760ba0bd1eb375e3118bc8b8d15d3b213d511b76a529e2703b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:47.335 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T05:17:47.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:47.375 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T05:17:47.375 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T05:17:47.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.3 2024-09-19T05:17:47.673 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:47 smithi064 systemd[1]: Stopping Ceph osd.3 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:48.150 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:47 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T05:17:47.670+0000 7f1536538640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:48.150 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:47 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T05:17:47.670+0000 7f1536538640 -1 osd.3 92 *** Got signal Terminated *** 2024-09-19T05:17:48.151 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:47 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T05:17:47.670+0000 7f1536538640 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:50.150 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:49 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T05:17:49.735+0000 7f152eb35640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.295714+0000 front 2024-09-19T05:17:27.295852+0000 (oldest deadline 2024-09-19T05:17:49.595901+0000) 2024-09-19T05:17:50.777 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:50.513+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:51.082 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:50 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T05:17:50.778+0000 7f152eb35640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.295714+0000 front 2024-09-19T05:17:27.295852+0000 (oldest deadline 2024-09-19T05:17:49.595901+0000) 2024-09-19T05:17:51.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:51.080+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:52.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:51.559+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:52.119 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:51 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3[49331]: 2024-09-19T05:17:51.772+0000 7f152eb35640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.295714+0000 front 2024-09-19T05:17:27.295852+0000 (oldest deadline 2024-09-19T05:17:49.595901+0000) 2024-09-19T05:17:52.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:51 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:51.860+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:52.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:52 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:52.117+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:52.130+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:53.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:52.604+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:53.218 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:52 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:52.886+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:53.332 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:53 smithi064 podman[108454]: 2024-09-19 05:17:53.065617188 +0000 UTC m=+5.482238865 container died 75fc0eb4b007afbe22f6f5868ad870ba5838a660a9a22e1c1eeb7bbd6d8eef0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T05:17:53.332 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:53.031+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:53.332 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:53 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:53.102+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:53.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:53.173+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:53.650 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:53 smithi064 podman[108454]: 2024-09-19 05:17:53.496428225 +0000 UTC m=+5.913049941 container cleanup 75fc0eb4b007afbe22f6f5868ad870ba5838a660a9a22e1c1eeb7bbd6d8eef0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T05:17:53.650 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:53 smithi064 bash[108454]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3 2024-09-19T05:17:54.012 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:53 smithi064 podman[108472]: 2024-09-19 05:17:53.697636485 +0000 UTC m=+0.628131240 container remove 75fc0eb4b007afbe22f6f5868ad870ba5838a660a9a22e1c1eeb7bbd6d8eef0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:17:54.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:53.622+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:54.218 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:53 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:53.933+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:54.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:54.009+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:54.263 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:54 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:54.097+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:54.263 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:54 smithi064 podman[108559]: 2024-09-19 05:17:54.025033852 +0000 UTC m=+0.016479381 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:54.263 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:54 smithi064 podman[108559]: 2024-09-19 05:17:54.150934231 +0000 UTC m=+0.142379751 container create a33379c6a8b943e3fc1c8a4b6c5e4c270908173679e46cd0f46f0d6b73bd9a0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T05:17:54.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:54.184+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:54.576 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:54 smithi064 podman[108559]: 2024-09-19 05:17:54.367914093 +0000 UTC m=+0.359359619 container init a33379c6a8b943e3fc1c8a4b6c5e4c270908173679e46cd0f46f0d6b73bd9a0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T05:17:54.576 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:54 smithi064 podman[108559]: 2024-09-19 05:17:54.371432997 +0000 UTC m=+0.362878517 container start a33379c6a8b943e3fc1c8a4b6c5e4c270908173679e46cd0f46f0d6b73bd9a0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-19T05:17:54.576 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:54 smithi064 podman[108559]: 2024-09-19 05:17:54.425958742 +0000 UTC m=+0.417404275 container attach a33379c6a8b943e3fc1c8a4b6c5e4c270908173679e46cd0f46f0d6b73bd9a0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:17:54.863 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 19 05:17:54 smithi064 podman[108559]: 2024-09-19 05:17:54.575125028 +0000 UTC m=+0.566570547 container died a33379c6a8b943e3fc1c8a4b6c5e4c270908173679e46cd0f46f0d6b73bd9a0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T05:17:55.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:54.581+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:55.129 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:55 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:55.057+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:55.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T05:17:55.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:55.198 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T05:17:55.198 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T05:17:55.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.4 2024-09-19T05:17:55.218 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:54 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:54.956+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:55.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:55 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:55.128+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:55.497 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:17:55 smithi103 systemd[1]: Stopping Ceph osd.4 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:55.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:55 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:55.159+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:55.939 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:17:55 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:17:55.495+0000 7fd36b428640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:55.940 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:17:55 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:17:55.495+0000 7fd36b428640 -1 osd.4 92 *** Got signal Terminated *** 2024-09-19T05:17:55.940 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:17:55 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:17:55.495+0000 7fd36b428640 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:56.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:55 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:55.548+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:56.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:55 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:55.938+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:56.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:56.072+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:56.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:56 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:56.147+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:56.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:56 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:56.170+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:57.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:56 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:56.574+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:57.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:57 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:57.116+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:57.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:57 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:57.043+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:57.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:56 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:56.987+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:57.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:57 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:57.170+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:58.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:57 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:57.619+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:58.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:57 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:57.619+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:17:58.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:58.156+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:58.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:58.051+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:58.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:58 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:58.051+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:17:58.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:58 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:58.031+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:58.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:58 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:58.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:58.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:58 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:58.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:17:59.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:58 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:58.638+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:17:59.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:58 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:58.638+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:17:59.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:58 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:58.990+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:17:59.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:17:58 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:17:58.898+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:17:59.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:59.077+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:17:59.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:17:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:17:59.077+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:17:59.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:59.201+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:17:59.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:17:59 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:17:59.201+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:17:59.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:59 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:59.228+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:17:59.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:17:59 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:17:59.228+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:17:59.719 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:17:59 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:17:59.231+0000 7fd363a25640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.013049+0000 front 2024-09-19T05:17:35.013076+0000 (oldest deadline 2024-09-19T05:17:59.113206+0000) 2024-09-19T05:18:00.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:59 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:59.588+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:00.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:17:59 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:17:59.588+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:00.218 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:17:59 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:17:59.866+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:00.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:17:59 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:17:59.984+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:00.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:00.069+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:00.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:00.069+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:00.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:00.231+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:00.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:00 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:00.231+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:00.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:00 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:00.228+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:00.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:00 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:00.228+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:00.718 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:00 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4[45698]: 2024-09-19T05:18:00.250+0000 7fd363a25640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.013049+0000 front 2024-09-19T05:17:35.013076+0000 (oldest deadline 2024-09-19T05:17:59.113206+0000) 2024-09-19T05:18:01.027 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:00 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:00.605+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:01.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:00 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:00.605+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:01.164 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:00 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:00.830+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:01.165 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:00 smithi103 podman[70349]: 2024-09-19 05:18:00.857953973 +0000 UTC m=+5.462353808 container died 74e1822cc9bf87f75b54095beddf228f252299dbaed5a89127e1d1474069cbee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T05:18:01.165 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:01 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:01.018+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:01.165 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:01 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:01.018+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:01.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:01 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:01.042+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:01.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:01 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:01.042+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:01.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:01 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:01.250+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:01.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:01 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:01.250+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:01.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:01 smithi103 podman[70349]: 2024-09-19 05:18:01.279043437 +0000 UTC m=+5.883443271 container cleanup 74e1822cc9bf87f75b54095beddf228f252299dbaed5a89127e1d1474069cbee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T05:18:01.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:01 smithi103 bash[70349]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4 2024-09-19T05:18:01.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:01 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:01.180+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:01.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:01 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:01.180+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:01.805 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:01 smithi103 podman[70365]: 2024-09-19 05:18:01.496774432 +0000 UTC m=+0.634472963 container remove 74e1822cc9bf87f75b54095beddf228f252299dbaed5a89127e1d1474069cbee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T05:18:02.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:01 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:01.597+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:02.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:01 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:01.597+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:02.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:01 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:01.569+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:02.143 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:01 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:01.997+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:02.143 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:01 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:01.997+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:02.143 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:01 smithi103 podman[70455]: 2024-09-19 05:18:01.801900069 +0000 UTC m=+0.016278165 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:02.143 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:01 smithi103 podman[70455]: 2024-09-19 05:18:01.931198566 +0000 UTC m=+0.145576647 container create 368c074a3b8341f370914806d25b5efe72d6e57dbf8cbabc40344ce36788e9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T05:18:02.143 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:01 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:01.804+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:02.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:02 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:02.080+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:02.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:02 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:02.080+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:02.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:02 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:02.247+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:02.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:02 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:02.247+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:02.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:02 smithi103 podman[70455]: 2024-09-19 05:18:02.141984131 +0000 UTC m=+0.356362223 container init 368c074a3b8341f370914806d25b5efe72d6e57dbf8cbabc40344ce36788e9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T05:18:02.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:02 smithi103 podman[70455]: 2024-09-19 05:18:02.145262756 +0000 UTC m=+0.359640843 container start 368c074a3b8341f370914806d25b5efe72d6e57dbf8cbabc40344ce36788e9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-19T05:18:02.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:02 smithi103 podman[70455]: 2024-09-19 05:18:02.225178004 +0000 UTC m=+0.439556104 container attach 368c074a3b8341f370914806d25b5efe72d6e57dbf8cbabc40344ce36788e9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T05:18:02.469 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 19 05:18:02 smithi103 podman[70455]: 2024-09-19 05:18:02.329631357 +0000 UTC m=+0.544009441 container died 368c074a3b8341f370914806d25b5efe72d6e57dbf8cbabc40344ce36788e9b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:18:02.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:02 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:02.226+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:02.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:02 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:02.226+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:02.969 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:02 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:02.848+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:02.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T05:18:03.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:02 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:02.550+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:03.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:02 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:02.568+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:03.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:02 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:02.568+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:03.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:03.079 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T05:18:03.079 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T05:18:03.079 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.5 2024-09-19T05:18:03.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:03 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:03.063+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:03.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:03 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:03.063+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:03.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:03 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:03.244+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:03.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:03 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:03.244+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:03.403 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:03.250+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:03.403 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:03.250+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:03.404 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:03 smithi174 systemd[1]: Stopping Ceph osd.5 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:03.404 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:03.401+0000 7f72b4cbc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:03.404 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:03.401+0000 7f72b4cbc640 -1 osd.5 92 *** Got signal Terminated *** 2024-09-19T05:18:03.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:02 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:02.980+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:03.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:02 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:02.980+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:03.777 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:03.536+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:03.778 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:03.401+0000 7f72b4cbc640 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:03.778 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:03.581+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:03.778 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:03 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:03.581+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:04.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:03 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:03.870+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:04.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:03 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:03.979+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:04.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:03 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:03.979+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:04.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:04 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:04.245+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:04.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:04 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:04.245+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:04.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:04 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:04.052+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:04.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:04 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:04.052+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:04.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:04 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:04.214+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:04.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:04 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:04.214+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:05.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:04 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:04.528+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:05.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:04 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:04.542+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:05.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:04 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:04.542+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:05.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:04 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:04.892+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:05.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:04 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:04.978+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:05.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:04 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:04.978+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:05.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:05 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:05.007+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:05.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:05 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:05.007+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:05.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:05 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:05.253+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:05.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:05 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:05.253+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:05.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:05 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:05.175+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:05.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:05 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:05.175+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:06.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:05 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:05.575+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:06.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:05 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:05.591+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:06.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:05 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:05.591+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:06.218 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:05 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:05.878+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:06.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:05 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:05.878+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:06.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:05 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:05.951+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:06.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:05 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:05.951+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:06.292 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:06.038+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:06.292 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:06.038+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:06.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:06.221+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:06.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:06.221+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:06.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:06.290+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:06.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:06.290+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:06.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:06.290+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:07.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:06.544+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:07.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:06.544+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:07.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:06.636+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:07.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:06.636+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:07.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:06 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:06.636+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.495782+0000 front 2024-09-19T05:17:41.495805+0000 (oldest deadline 2024-09-19T05:18:05.596012+0000) 2024-09-19T05:18:07.120 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:06 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:06.860+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:07.120 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:06 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:06.860+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:07.121 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:06 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:06.918+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:07.121 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:06 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:06.918+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:07.121 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:06 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:06.918+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:07.276 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:07.015+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:07.277 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:07.015+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:07.277 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:07.015+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:07.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:07.259+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:07.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:07.259+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:07.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:07.275+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:07.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:07.275+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:07.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:07.275+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:08.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:07.540+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:08.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:07.540+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:08.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:07.645+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:27.593390+0000 front 2024-09-19T05:17:27.593414+0000 (oldest deadline 2024-09-19T05:17:49.893492+0000) 2024-09-19T05:18:08.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:07.646+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.194511+0000 front 2024-09-19T05:17:35.194471+0000 (oldest deadline 2024-09-19T05:17:57.494529+0000) 2024-09-19T05:18:08.028 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:07 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5[46067]: 2024-09-19T05:18:07.646+0000 7f72ad2b9640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.495782+0000 front 2024-09-19T05:17:41.495805+0000 (oldest deadline 2024-09-19T05:18:05.596012+0000) 2024-09-19T05:18:08.131 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:07 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:07.844+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:08.131 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:07 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:07.844+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:08.132 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:07 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:07.961+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:08.132 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:07 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:07.961+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:08.132 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:07 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:07.961+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:08.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:07.968+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:08.288 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:07.968+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:08.289 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:07 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:07.968+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:08.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:08.234+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:08.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:08.234+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:08.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:08.234+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:08.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:08.286+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:08.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:08.286+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:08.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:08.286+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:08.853 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:08.535+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:08.853 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:08 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:08.535+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:09.112 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:08 smithi174 podman[70818]: 2024-09-19 05:18:08.85256979 +0000 UTC m=+5.571303482 container died 6978c7c41eb02d4d8a70b98e380c7677b6cee2dfb88480a757ec34a5c077ea0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:18:09.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:08.835+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:09.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:08.835+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:09.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:08.973+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:09.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:08.973+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:09.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:08 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:08.973+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:09.311 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:08.959+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:09.311 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:08.959+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:09.311 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:08 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:08.959+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:09.490 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:09 smithi174 podman[70818]: 2024-09-19 05:18:09.209862606 +0000 UTC m=+5.928596294 container cleanup 6978c7c41eb02d4d8a70b98e380c7677b6cee2dfb88480a757ec34a5c077ea0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T05:18:09.491 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:09 smithi174 bash[70818]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5 2024-09-19T05:18:09.491 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:09 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:09.263+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:09.491 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:09 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:09.263+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:09.491 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:09 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:09.263+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:09.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:09.310+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:09.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:09.310+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:09.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:09.310+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:09.777 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:09 smithi174 podman[70835]: 2024-09-19 05:18:09.506341311 +0000 UTC m=+0.646052294 container remove 6978c7c41eb02d4d8a70b98e380c7677b6cee2dfb88480a757ec34a5c077ea0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:18:09.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:09 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:09.489+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:09.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:09 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:09.489+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:10.131 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:09 smithi174 podman[70925]: 2024-09-19 05:18:09.737600691 +0000 UTC m=+0.015632913 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:10.132 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:09 smithi174 podman[70925]: 2024-09-19 05:18:09.845412018 +0000 UTC m=+0.123444231 container create a43b1f47b1206da20f3bfbd85c33650c902c900ca6ac32489d967c7fc88915fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T05:18:10.132 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:10 smithi174 podman[70925]: 2024-09-19 05:18:10.055772326 +0000 UTC m=+0.333804546 container init a43b1f47b1206da20f3bfbd85c33650c902c900ca6ac32489d967c7fc88915fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T05:18:10.132 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:10 smithi174 podman[70925]: 2024-09-19 05:18:10.059231327 +0000 UTC m=+0.337263544 container start a43b1f47b1206da20f3bfbd85c33650c902c900ca6ac32489d967c7fc88915fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:10.132 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:10 smithi174 podman[70925]: 2024-09-19 05:18:10.130762511 +0000 UTC m=+0.408794735 container attach a43b1f47b1206da20f3bfbd85c33650c902c900ca6ac32489d967c7fc88915fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T05:18:10.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:09 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:09.829+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:10.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:09 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:09.829+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:10.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:09 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:09.931+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:10.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:09 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:09.931+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:10.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:09 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:09.931+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:10.295 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:09.986+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:10.295 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:09.986+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:10.295 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:09.986+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:10.492 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 05:18:10 smithi174 podman[70925]: 2024-09-19 05:18:10.248399487 +0000 UTC m=+0.526431702 container died a43b1f47b1206da20f3bfbd85c33650c902c900ca6ac32489d967c7fc88915fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T05:18:10.492 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:10 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:10.291+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:10.493 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:10 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:10.291+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:10.493 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:10 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:10.291+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:10.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:10 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:10.293+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:10.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:10 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:10.293+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:10.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:10 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:10.293+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:10.768 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:10 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:10.505+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:10.768 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:10 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:10.505+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:10.784 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T05:18:10.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:10.822 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T05:18:10.822 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T05:18:10.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.6 2024-09-19T05:18:11.125 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:10 smithi064 systemd[1]: Stopping Ceph osd.6 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:11.125 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:10 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:10.961+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:11.125 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:10 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:10.961+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:11.125 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:10 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:10.961+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:11.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:10 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:10.844+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:11.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:10 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:10.844+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:11.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:10 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:10.956+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:11.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:10 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:10.956+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:11.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:10 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:10.956+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:11.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:11.123+0000 7f6d774d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:11.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:11.123+0000 7f6d774d9640 -1 osd.6 92 *** Got signal Terminated *** 2024-09-19T05:18:11.401 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:11.123+0000 7f6d774d9640 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:11.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:11.297+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:11.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:11.297+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:11.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:11.297+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:11.520 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:11 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:11.252+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:11.521 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:11 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:11.252+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:11.521 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:11 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:11.252+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:11.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:11 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:11.519+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:11.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:11 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:11.519+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:12.218 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:11 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:11.828+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:12.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:11 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:11.828+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:12.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:11 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:11.982+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:12.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:11 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:11.982+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:12.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:11 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:11.982+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:12.324 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:11.917+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:12.325 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:11.917+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:12.325 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:11 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:11.917+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:12.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:12 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:12.271+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:12.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:12 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:12.271+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:12.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:12 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:12.271+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:12.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:12 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:12.322+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:12.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:12 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:12.322+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:12.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:12 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:12.322+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:13.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:12 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:12.539+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:13.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:12 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:12.539+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:13.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:12 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:12.886+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:13.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:12 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:12.886+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:13.151 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:12 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:12.886+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:13.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:12 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:12.810+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:13.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:12 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:12.810+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:13.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:12 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:12.974+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:13.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:12 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:12.974+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:13.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:12 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:12.974+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:13.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:13 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:13.256+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:13.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:13 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:13.256+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:13.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:13 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:13.256+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:13.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:13.275+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:13.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:13.275+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:13.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:13.275+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:14.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:13 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:13.573+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:14.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:13 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:13.573+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:14.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:13 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:13.769+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:14.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:13 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:13.769+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:14.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:13 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:13.989+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:14.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:13 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:13.989+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:14.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:13 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:13.989+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:14.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:13 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:13.989+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:14.307 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:13.922+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:14.307 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:13.922+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:14.307 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:13 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:13.922+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:14.548 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:14 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:14.295+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:14.548 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:14 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:14.295+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:14.548 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:14 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:14.295+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:14.652 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:14.306+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:14.652 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:14.306+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:14.653 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:14.306+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:15.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:14 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:14.546+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:15.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:14 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:14.546+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:15.038 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:14 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:14.776+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:15.038 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:14 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:14.776+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:15.356 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:14.927+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:15.356 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:14.927+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:15.356 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:14 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:14.927+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:15.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:15 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:15.036+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:15.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:15 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:15.036+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:15.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:15 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:15.036+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:15.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:15 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:15.036+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:15.527 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:15.517+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:15.528 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:15.517+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:15.528 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:15.517+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:15.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:15.270+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:15.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:15.270+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:15.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:15.270+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:15.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:15 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:15.270+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:15.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:15.354+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:15.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:15.354+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:15.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:15.354+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:16.083 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:15 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:15.770+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:16.084 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:15 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:15.770+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:16.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:15.881+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.732804+0000 front 2024-09-19T05:17:28.732850+0000 (oldest deadline 2024-09-19T05:17:52.232977+0000) 2024-09-19T05:18:16.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:15.881+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.734499+0000 front 2024-09-19T05:17:32.734374+0000 (oldest deadline 2024-09-19T05:17:58.034237+0000) 2024-09-19T05:18:16.151 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:15 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6[56324]: 2024-09-19T05:18:15.881+0000 7f6d73ade640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.535394+0000 front 2024-09-19T05:17:41.535281+0000 (oldest deadline 2024-09-19T05:18:06.235256+0000) 2024-09-19T05:18:16.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:16.081+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:16.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:16.081+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:16.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:16.081+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:16.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:16.081+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:16.565 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:16.307+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:16.565 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:16.307+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:16.565 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:16.307+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:16.565 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:16.307+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:16.650 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:16 smithi064 podman[108659]: 2024-09-19 05:18:16.462527838 +0000 UTC m=+5.443409414 container died b122aa8d0e1e5838ce03e5e76eeb787cdf2588352e605199b1d77eb74babad40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T05:18:16.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:16.342+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:16.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:16.342+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:16.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:16.342+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:16.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:16 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:16.342+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:17.027 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:16.563+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:17.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:16.563+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:17.028 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:16 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:16.563+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:17.132 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:16.794+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:17.133 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:16.794+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:17.133 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:16 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:16.794+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:17.142 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:16 smithi064 podman[108659]: 2024-09-19 05:18:16.898316915 +0000 UTC m=+5.879198480 container cleanup b122aa8d0e1e5838ce03e5e76eeb787cdf2588352e605199b1d77eb74babad40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:18:17.142 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:16 smithi064 bash[108659]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6 2024-09-19T05:18:17.400 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108675]: 2024-09-19 05:18:17.141129231 +0000 UTC m=+0.671151191 container remove b122aa8d0e1e5838ce03e5e76eeb787cdf2588352e605199b1d77eb74babad40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T05:18:17.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:17.347+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:17.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:17.347+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:17.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:17.347+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:17.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:17 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:17.347+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:17.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:17.131+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:17.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:17.131+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:17.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:17.131+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:17.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:17.131+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:17.745 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108766]: 2024-09-19 05:18:17.380988366 +0000 UTC m=+0.016442701 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:17.745 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108766]: 2024-09-19 05:18:17.517133267 +0000 UTC m=+0.152587589 container create 8d5178c95dd12aa318064c1a8bbbec406018199fcb68843bba118293d0a3b16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T05:18:17.745 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108766]: 2024-09-19 05:18:17.744539738 +0000 UTC m=+0.379994069 container init 8d5178c95dd12aa318064c1a8bbbec406018199fcb68843bba118293d0a3b16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:17.566+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:17.566+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:17.566+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:17.323+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:17.323+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:17.323+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:17.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:17 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:17.323+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:18.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108766]: 2024-09-19 05:18:17.74810285 +0000 UTC m=+0.383557178 container start 8d5178c95dd12aa318064c1a8bbbec406018199fcb68843bba118293d0a3b16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T05:18:18.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108766]: 2024-09-19 05:18:17.819424058 +0000 UTC m=+0.454878390 container attach 8d5178c95dd12aa318064c1a8bbbec406018199fcb68843bba118293d0a3b16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:18.150 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:17 smithi064 podman[108766]: 2024-09-19 05:18:17.942298695 +0000 UTC m=+0.577753029 container died 8d5178c95dd12aa318064c1a8bbbec406018199fcb68843bba118293d0a3b16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:18:18.177 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:17.774+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:18.177 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:17.774+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:18.177 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:17.774+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:18.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T05:18:18.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:18.175+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:18.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:18.175+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:18.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:18.175+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:18.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:18.175+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:18 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:18.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:18 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:18.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:18 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:18.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:18 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:18.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:18 smithi064 podman[108789]: 2024-09-19 05:18:18.439454248 +0000 UTC m=+0.486846808 container remove 8d5178c95dd12aa318064c1a8bbbec406018199fcb68843bba118293d0a3b16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:18 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-19T05:18:18.485 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:18 smithi064 systemd[1]: Stopped Ceph osd.6 for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:18:18.486 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 19 05:18:18 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.6.service: Consumed 9.393s CPU time. 2024-09-19T05:18:18.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:18.501 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T05:18:18.501 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T05:18:18.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.7 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:18.550+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:18.550+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:18.550+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:18.334+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:18.334+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:18.334+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:18.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:18 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:18.334+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:18.783 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 systemd[1]: Stopping Ceph osd.7 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:19.156 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:18.781+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:19.157 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:18.781+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:19.157 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:18.781+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:19.157 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:18.833+0000 7fb81df0e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:19.157 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:18.833+0000 7fb81df0e640 -1 osd.7 92 *** Got signal Terminated *** 2024-09-19T05:18:19.157 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:18 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:18.833+0000 7fb81df0e640 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:19.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:19.153+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:19.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:19.153+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:19.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:19.153+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:19.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:19.153+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:19.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:19 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:19.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:19.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:19 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:19.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:19.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:19 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:19.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:19.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:19 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:19.373+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:19.507+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:19.507+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:19.507+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:19.358+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:19.358+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:19.358+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:19.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:19 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:19.358+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:20.149 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:19.771+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:20.149 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:19.771+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:20.149 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:19 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:19.771+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:20.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:20.147+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:20.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:20.147+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:20.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:20.147+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:20.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:20.147+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:20.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:20.387+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:20.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:20.387+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:20.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:20.387+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:20.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:20 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:20.387+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:20.777 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:20.505+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:20.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:20.505+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:20.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:20.505+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:20.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:20.407+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:20.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:20.407+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:20.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:20.407+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:20.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:20 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:20.407+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:21.170 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:20.813+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:21.170 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:20.813+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:21.170 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:20 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:20.813+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:21.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:21.168+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:21.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:21.168+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:21.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:21.168+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:21.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:21.168+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:21.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:21.390+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:21.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:21.390+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:21.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:21.390+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:21.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:21 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:21.390+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:21.456+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:21.456+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:21.456+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:21.449+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:21.449+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:21.449+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:21.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:21 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:21.449+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:22.167 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:21.829+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:22.167 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:21.829+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:22.167 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:21.829+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:22.468 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:22.165+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:22.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:22.165+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:22.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:22.165+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:22.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:22.165+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:22.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:22 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:22.361+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:22.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:22 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:22.361+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:22.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:22 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:22.361+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:22.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:22 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:22.361+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:22.777 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:22.492+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:22.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:22.492+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:22.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:22.492+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:22.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:22.406+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:22.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:22.406+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:22.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:22.406+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:22.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:22 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:22.406+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:23.208 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:22.865+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:23.208 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:22.865+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:23.208 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:22.865+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:23.208 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:22 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:22.865+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.429020+0000 front 2024-09-19T05:17:58.428960+0000 (oldest deadline 2024-09-19T05:18:21.929070+0000) 2024-09-19T05:18:23.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:23.206+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:23.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:23.206+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:23.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:23.206+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:23.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:23.206+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:23.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:23.388+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:23.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:23.388+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:23.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:23.388+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:23.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:23.388+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:23.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:23 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:23.388+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:23.508+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:23.508+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:23.508+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:23.508+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:23.427+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:23.427+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:23.427+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:23.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:23 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:23.427+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:24.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:23.819+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:32.725384+0000 front 2024-09-19T05:17:32.725413+0000 (oldest deadline 2024-09-19T05:17:58.025599+0000) 2024-09-19T05:18:24.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:23.819+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.126747+0000 front 2024-09-19T05:17:42.126559+0000 (oldest deadline 2024-09-19T05:18:05.626787+0000) 2024-09-19T05:18:24.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:23.819+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.228312+0000 front 2024-09-19T05:17:50.228359+0000 (oldest deadline 2024-09-19T05:18:16.128186+0000) 2024-09-19T05:18:24.219 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:23 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7[52675]: 2024-09-19T05:18:23.819+0000 7fb81a513640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.429020+0000 front 2024-09-19T05:17:58.428960+0000 (oldest deadline 2024-09-19T05:18:21.929070+0000) 2024-09-19T05:18:24.575 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:24 smithi103 podman[70572]: 2024-09-19 05:18:24.319258713 +0000 UTC m=+5.614556314 container died 1c5b8b16dbda40490cf5417fd85f49ddc2106411823c843806aa5f8094de703f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T05:18:24.575 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:24 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:24.242+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:24.575 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:24 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:24.242+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:24.575 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:24 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:24.242+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:24.576 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:24 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:24.242+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:24.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:24 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:24.367+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:24.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:24 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:24.367+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:24.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:24 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:24.367+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:24.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:24 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:24.367+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:24.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:24 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:24.367+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:24.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:24.485+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:24.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:24.485+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:24.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:24.485+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:24.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:24.485+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:24.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:24.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:24.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:24.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:24.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:24.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:24.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:24 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:24.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:24.950 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:24 smithi103 podman[70572]: 2024-09-19 05:18:24.673087092 +0000 UTC m=+5.968384689 container cleanup 1c5b8b16dbda40490cf5417fd85f49ddc2106411823c843806aa5f8094de703f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:18:24.950 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:24 smithi103 bash[70572]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7 2024-09-19T05:18:25.214 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:24 smithi103 podman[70588]: 2024-09-19 05:18:24.949356306 +0000 UTC m=+0.630285590 container remove 1c5b8b16dbda40490cf5417fd85f49ddc2106411823c843806aa5f8094de703f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T05:18:25.469 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:25 smithi103 podman[70675]: 2024-09-19 05:18:25.186262487 +0000 UTC m=+0.015733475 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:25.469 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:25 smithi103 podman[70675]: 2024-09-19 05:18:25.308649596 +0000 UTC m=+0.138120629 container create f0bbb45341fa11553b42eba54897123fcb7e988d64fbcfea5ab8d04809187854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T05:18:25.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:25 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:25.211+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:25.470 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:25 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:25.211+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:25.470 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:25 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:25.211+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:25.470 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:25 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:25.211+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:25.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:25 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:25.336+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:25.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:25 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:25.336+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:25.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:25 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:25.336+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:25.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:25 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:25.336+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:25.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:25 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:25.336+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:25.522+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:25.522+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:25.522+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:25.522+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:25.413+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:25.413+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:25.413+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:25.413+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:25.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:25 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:25.413+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:25.969 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:25 smithi103 podman[70675]: 2024-09-19 05:18:25.519279994 +0000 UTC m=+0.348750982 container init f0bbb45341fa11553b42eba54897123fcb7e988d64fbcfea5ab8d04809187854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-19T05:18:25.969 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:25 smithi103 podman[70675]: 2024-09-19 05:18:25.522921408 +0000 UTC m=+0.352392399 container start f0bbb45341fa11553b42eba54897123fcb7e988d64fbcfea5ab8d04809187854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T05:18:25.969 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:25 smithi103 podman[70675]: 2024-09-19 05:18:25.60255063 +0000 UTC m=+0.432021634 container attach f0bbb45341fa11553b42eba54897123fcb7e988d64fbcfea5ab8d04809187854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T05:18:25.969 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 19 05:18:25 smithi103 podman[70675]: 2024-09-19 05:18:25.712750273 +0000 UTC m=+0.542221262 container died f0bbb45341fa11553b42eba54897123fcb7e988d64fbcfea5ab8d04809187854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T05:18:26.266 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:26 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:26.209+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:26.266 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:26 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:26.209+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:26.266 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:26 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:26.209+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:26.266 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:26 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:26.209+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:26.266 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:26 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:26.209+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:26.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T05:18:26.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:26.326 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T05:18:26.326 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T05:18:26.326 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.8 2024-09-19T05:18:26.629 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:26.388+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:26.629 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:26.388+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:26.629 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:26.388+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:26.629 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:26.388+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:26.388+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 systemd[1]: Stopping Ceph osd.8 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.539+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.539+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.539+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.539+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.628+0000 7fc64e7a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.628+0000 7fc64e7a3640 -1 osd.8 92 *** Got signal Terminated *** 2024-09-19T05:18:26.630 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:26 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:26.628+0000 7fc64e7a3640 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:26.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:26 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:26.301+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:26.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:26 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:26.301+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:26.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:26 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:26.301+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:26.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:26 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:26.301+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:26.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:26 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:26.301+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:27.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:27 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:27.201+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:27.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:27 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:27.201+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:27.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:27 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:27.201+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:27.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:27 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:27.201+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:27.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:27 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:27.201+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:27.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:27 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:27.264+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:27.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:27 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:27.264+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:27.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:27 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:27.264+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:27.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:27 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:27.264+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:27.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:27 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:27.264+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:27.520+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:27.520+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:27.520+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:27.520+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:27.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:27.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:27.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:27.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:27.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:27.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:27.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:27 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:27.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:28.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:28 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:28.189+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:28.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:28 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:28.189+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:28.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:28 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:28.189+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:28.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:28 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:28.189+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:28.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:28 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:28.189+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:28.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:28 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:28.217+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:28.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:28 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:28.217+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:28.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:28 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:28.217+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:28.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:28 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:28.217+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:28.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:28 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:28.217+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:28.548+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:28.548+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:28.548+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:28.548+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:28.359+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:28.359+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:28.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:28.359+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:28.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:28.359+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:28.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:28 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:28.359+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:29.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:29 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:29.197+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:29.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:29 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:29.197+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:29.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:29 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:29.197+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:29.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:29 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:29.197+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:29.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:29 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:29.197+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:29.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:29.180+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:29.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:29.180+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:29.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:29.180+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:29.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:29.180+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:29.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:29 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:29.180+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:29.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:29.521+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:29.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:29.521+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:29.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:29.521+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:29.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:29.521+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:29.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:29.521+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:03.843974+0000 front 2024-09-19T05:18:03.843994+0000 (oldest deadline 2024-09-19T05:18:29.144151+0000) 2024-09-19T05:18:29.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:29.344+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:29.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:29.344+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:29.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:29.344+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:29.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:29.344+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:29.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:29 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:29.344+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:30.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:30 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:30.166+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:30.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:30 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:30.166+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:30.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:30 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:30.166+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:30.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:30 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:30.166+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:30.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:30 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:30.166+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:30.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:30.245+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:30.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:30.245+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:30.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:30.245+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:30.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:30.245+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:30.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:30.245+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:30.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:30 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:30.245+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:30.493+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:30.493+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:30.493+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:30.493+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:30.493+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:03.843974+0000 front 2024-09-19T05:18:03.843994+0000 (oldest deadline 2024-09-19T05:18:29.144151+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:30.392+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:30.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:30.392+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:30.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:30.392+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:30.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:30.392+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:30.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:30.392+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:30.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:30 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:30.392+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:31.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:31 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:31.147+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:31.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:31 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:31.147+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:31.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:31 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:31.147+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:31.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:31 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:31.147+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:31.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:31 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:31.147+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:31.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:31.224+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:31.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:31.224+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:31.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:31.224+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:31.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:31.224+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:31.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:31.224+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:31.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:31 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:31.224+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:31.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:31.460+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:35.339805+0000 front 2024-09-19T05:17:35.339919+0000 (oldest deadline 2024-09-19T05:18:01.239639+0000) 2024-09-19T05:18:31.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:31.460+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:41.239957+0000 front 2024-09-19T05:17:41.240031+0000 (oldest deadline 2024-09-19T05:18:05.940152+0000) 2024-09-19T05:18:31.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:31.460+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.541807+0000 front 2024-09-19T05:17:50.541794+0000 (oldest deadline 2024-09-19T05:18:14.641638+0000) 2024-09-19T05:18:31.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:31.460+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.442993+0000 front 2024-09-19T05:18:00.442974+0000 (oldest deadline 2024-09-19T05:18:22.743008+0000) 2024-09-19T05:18:31.778 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8[53003]: 2024-09-19T05:18:31.460+0000 7fc646da0640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:03.843974+0000 front 2024-09-19T05:18:03.843994+0000 (oldest deadline 2024-09-19T05:18:29.144151+0000) 2024-09-19T05:18:31.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:31.370+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:31.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:31.370+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:31.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:31.370+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:31.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:31.370+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:31.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:31.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:31.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:31.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:32.277 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:31 smithi174 podman[71040]: 2024-09-19 05:18:31.993916052 +0000 UTC m=+5.467710312 container died 4d688e126865897748fee0841f15ea0c62a750040264e419e66ec43236b6e75c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:18:32.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:32 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:32.119+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:32.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:32 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:32.119+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:32.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:32 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:32.119+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:32.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:32 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:32.119+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:32.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:32 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:32.119+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:32.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:32 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:32.119+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:32.654 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:32 smithi174 podman[71040]: 2024-09-19 05:18:32.369314595 +0000 UTC m=+5.843108896 container cleanup 4d688e126865897748fee0841f15ea0c62a750040264e419e66ec43236b6e75c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:32 smithi174 bash[71040]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:32 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:32.363+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:32 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:32.363+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:32 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:32.363+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:32 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:32.363+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:32 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:32.363+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:32.655 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:32 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:32.363+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:32.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:32 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:32.234+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:32.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:32 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:32.234+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:32.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:32 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:32.234+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:32.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:32 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:32.234+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:32.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:32 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:32.234+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:32.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:32 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:32.234+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:32.982 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:32 smithi174 podman[71055]: 2024-09-19 05:18:32.65375631 +0000 UTC m=+0.654102688 container remove 4d688e126865897748fee0841f15ea0c62a750040264e419e66ec43236b6e75c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:33.274 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:32 smithi174 podman[71145]: 2024-09-19 05:18:32.884068124 +0000 UTC m=+0.015756964 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:33.274 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:32 smithi174 podman[71145]: 2024-09-19 05:18:32.987876649 +0000 UTC m=+0.119565477 container create 4fadaba26864b7d7e90ee653e04b60719a47f69d7192b8f5c208689da26ff33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T05:18:33.274 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 podman[71145]: 2024-09-19 05:18:33.190161025 +0000 UTC m=+0.321849853 container init 4fadaba26864b7d7e90ee653e04b60719a47f69d7192b8f5c208689da26ff33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T05:18:33.274 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 podman[71145]: 2024-09-19 05:18:33.193370239 +0000 UTC m=+0.325059066 container start 4fadaba26864b7d7e90ee653e04b60719a47f69d7192b8f5c208689da26ff33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:18:33.274 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 podman[71145]: 2024-09-19 05:18:33.273264607 +0000 UTC m=+0.404953440 container attach 4fadaba26864b7d7e90ee653e04b60719a47f69d7192b8f5c208689da26ff33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:33.528 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 podman[71145]: 2024-09-19 05:18:33.381710509 +0000 UTC m=+0.513399346 container died 4fadaba26864b7d7e90ee653e04b60719a47f69d7192b8f5c208689da26ff33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T05:18:33.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:33.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:33.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:33.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:33.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:33.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:33.529 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:33.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:33.529 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:33.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:33.529 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:33.373+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:33.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:33 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:33.152+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:33.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:33 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:33.152+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:33.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:33 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:33.152+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:33.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:33 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:33.152+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:33.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:33 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:33.152+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:33.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:33 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:33.152+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:33.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:33 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:33.250+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:33.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:33 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:33.250+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:33.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:33 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:33.250+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:33.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:33 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:33.250+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:33.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:33 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:33.250+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:33.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:33 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:33.250+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:33.921 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T05:18:33.945 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 podman[71168]: 2024-09-19 05:18:33.901585845 +0000 UTC m=+0.511420275 container remove 4fadaba26864b7d7e90ee653e04b60719a47f69d7192b8f5c208689da26ff33b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T05:18:33.945 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.8.service: Deactivated successfully. 2024-09-19T05:18:33.945 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 systemd[1]: Stopped Ceph osd.8 for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:18:33.945 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 05:18:33 smithi174 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.8.service: Consumed 11.210s CPU time. 2024-09-19T05:18:33.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:33.961 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T05:18:33.961 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T05:18:33.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.9 2024-09-19T05:18:34.276 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 systemd[1]: Stopping Ceph osd.9 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.186+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.186+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.186+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.186+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.186+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.186+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.274+0000 7fdb24b94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.274+0000 7fdb24b94640 -1 osd.9 92 *** Got signal Terminated *** 2024-09-19T05:18:34.277 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:34 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:34.274+0000 7fdb24b94640 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:34.527 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:34 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:34.398+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:34.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:34 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:34.398+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:34.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:34 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:34.398+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:34.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:34 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:34.398+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:34.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:34 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:34.398+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:34.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:34 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:34.398+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:34.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:34 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:34.218+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:34.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:34 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:34.218+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:34.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:34 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:34.218+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:34.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:34 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:34.218+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:34.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:34 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:34.218+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:34.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:34 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:34.218+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:35.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:35 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:35.215+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:35.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:35 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:35.215+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:35.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:35 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:35.215+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:35.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:35 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:35.215+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:35.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:35 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:35.215+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:35.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:35 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:35.215+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:35.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:35 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:35.391+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:35.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:35 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:35.391+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:35.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:35 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:35.391+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:35.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:35 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:35.391+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:35.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:35 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:35.391+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:35.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:35 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:35.391+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:35.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:35 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:35.205+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:35.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:35 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:35.205+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:35.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:35 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:35.205+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:35.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:35 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:35.205+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:35.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:35 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:35.205+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:35.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:35 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:35.205+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:36.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:36.212+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:36.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:36.212+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:36.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:36.212+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:36.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:36.212+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:36.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:36.212+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:36.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:36 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:36.212+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:36.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:36 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:36.188+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:36.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:36 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:36.188+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:36.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:36 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:36.188+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:36.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:36 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:36.188+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:36.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:36 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:36.188+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:36.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:36 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:36.188+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:36.777 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:36 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:36.415+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:36.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:36 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:36.415+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:36.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:36 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:36.415+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:36.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:36 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:36.415+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:36.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:36 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:36.415+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:36.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:36 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:36.415+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:37.263 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:37.263 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:37.264 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:37.264 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:37.264 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:37.264 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:37.264 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:37 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:37.261+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:37.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:37.185+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:37.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:37.185+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:37.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:37.185+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:37.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:37.185+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:37.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:37.185+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:37.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:37 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:37.185+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:37.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:37.437+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:37.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:37.437+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:37.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:37.437+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:37.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:37.437+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:37.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:37.437+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:37.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:37.437+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:38.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:38.153+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:38.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:38.153+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:38.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:38.153+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:38.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:38.153+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:38.652 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:38.153+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:38.652 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:38 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:38.153+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:38.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:38 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:38.259+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:38.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:38 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:38.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:38.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:38 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:38.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:38.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:38 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:38.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:38.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:38 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:38.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:38.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:38 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:38.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:38.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:38 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:38.405+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:39.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:39.178+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.690085+0000 front 2024-09-19T05:17:26.689952+0000 (oldest deadline 2024-09-19T05:17:50.190070+0000) 2024-09-19T05:18:39.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:39.178+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.091469+0000 front 2024-09-19T05:17:36.091273+0000 (oldest deadline 2024-09-19T05:17:58.391377+0000) 2024-09-19T05:18:39.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:39.178+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.293671+0000 front 2024-09-19T05:17:42.293783+0000 (oldest deadline 2024-09-19T05:18:05.793650+0000) 2024-09-19T05:18:39.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:39.178+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.395802+0000 front 2024-09-19T05:17:51.395864+0000 (oldest deadline 2024-09-19T05:18:15.495954+0000) 2024-09-19T05:18:39.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:39.178+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:17:58.397114+0000 front 2024-09-19T05:17:58.397128+0000 (oldest deadline 2024-09-19T05:18:22.496915+0000) 2024-09-19T05:18:39.651 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9[63327]: 2024-09-19T05:18:39.178+0000 7fdb1d191640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.098335+0000 front 2024-09-19T05:18:07.098398+0000 (oldest deadline 2024-09-19T05:18:31.198142+0000) 2024-09-19T05:18:39.679 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:39 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:39.374+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:39.679 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:39 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:39.374+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:39.679 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:39 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:39.374+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:39.679 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:39 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:39.374+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:39.679 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:39 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:39.374+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:39.679 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:39 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:39.374+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:39.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:39 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:39.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:40.001 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:39 smithi064 podman[108869]: 2024-09-19 05:18:39.728680931 +0000 UTC m=+5.565514572 container died 05b35c0193dab6800d5d21b59a8f38fd3bc1099c8ae973b562c2435b06e0448b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T05:18:40.252 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 podman[108869]: 2024-09-19 05:18:40.074375374 +0000 UTC m=+5.911209001 container cleanup 05b35c0193dab6800d5d21b59a8f38fd3bc1099c8ae973b562c2435b06e0448b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T05:18:40.252 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 bash[108869]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9 2024-09-19T05:18:40.650 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 podman[108885]: 2024-09-19 05:18:40.25051354 +0000 UTC m=+0.516790918 container remove 05b35c0193dab6800d5d21b59a8f38fd3bc1099c8ae973b562c2435b06e0448b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:40.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:40 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:40.303+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:40.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:40.332+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:40.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:40.332+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:40.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:40.332+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:40.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:40.332+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:40.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:40.332+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:40.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:40 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:40.332+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:40.954 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 podman[108971]: 2024-09-19 05:18:40.603392852 +0000 UTC m=+0.016539239 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:40.954 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 podman[108971]: 2024-09-19 05:18:40.717752955 +0000 UTC m=+0.130899329 container create c6c2bdea79d50346f262a8485dd1336aaf734870ab8c606439d8b3f3bdb8d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:18:40.955 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 podman[108971]: 2024-09-19 05:18:40.953782368 +0000 UTC m=+0.366928741 container init c6c2bdea79d50346f262a8485dd1336aaf734870ab8c606439d8b3f3bdb8d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:41.400 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:40 smithi064 podman[108971]: 2024-09-19 05:18:40.957285936 +0000 UTC m=+0.370432311 container start c6c2bdea79d50346f262a8485dd1336aaf734870ab8c606439d8b3f3bdb8d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T05:18:41.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:41 smithi064 podman[108971]: 2024-09-19 05:18:41.04531835 +0000 UTC m=+0.458464725 container attach c6c2bdea79d50346f262a8485dd1336aaf734870ab8c606439d8b3f3bdb8d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T05:18:41.401 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 19 05:18:41 smithi064 podman[108971]: 2024-09-19 05:18:41.152185578 +0000 UTC m=+0.565331961 container died c6c2bdea79d50346f262a8485dd1336aaf734870ab8c606439d8b3f3bdb8d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:41.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:41.295+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:41.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T05:18:41.777 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:41.368+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:41.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:41.368+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:41.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:41.368+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:41.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:41.368+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:41.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:41.368+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:41.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:41.368+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:41.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:41.780 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T05:18:41.780 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T05:18:41.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.10 2024-09-19T05:18:42.077 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:41 smithi103 systemd[1]: Stopping Ceph osd.10 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.075+0000 7f9b08b37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.075+0000 7f9b08b37640 -1 osd.10 92 *** Got signal Terminated *** 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.075+0000 7f9b08b37640 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:42.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:42.470 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:42.470 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:42 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:42.289+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:42.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:42.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:42.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:42.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:42.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:42.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:42.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:42.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:42.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:42.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:42.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:42 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:42.371+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:43.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:43 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:43.288+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:43.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:43 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:43.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:43.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:43 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:43.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:43.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:43 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:43.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:43.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:43 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:43.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:43.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:43 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:43.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:43.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:43 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:43.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:44.722 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:44 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:44.293+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:44.737 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:44.302+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:44.737 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:44.302+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:44.737 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:44.302+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:44.737 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:44.302+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:44.738 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:44.302+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:44.738 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:44 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:44.302+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:45.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:45 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:45.286+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:45.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:45 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:45.335+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:28.587644+0000 front 2024-09-19T05:17:28.587733+0000 (oldest deadline 2024-09-19T05:17:51.487660+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:31.487925+0000 front 2024-09-19T05:17:31.488074+0000 (oldest deadline 2024-09-19T05:18:00.288482+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:44.890142+0000 front 2024-09-19T05:17:44.890041+0000 (oldest deadline 2024-09-19T05:18:06.589924+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:51.692071+0000 front 2024-09-19T05:17:51.692146+0000 (oldest deadline 2024-09-19T05:18:13.391826+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.393125+0000 front 2024-09-19T05:18:00.393187+0000 (oldest deadline 2024-09-19T05:18:25.693169+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:07.394203+0000 front 2024-09-19T05:18:07.394311+0000 (oldest deadline 2024-09-19T05:18:29.694137+0000) 2024-09-19T05:18:46.719 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:46 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10[59386]: 2024-09-19T05:18:46.323+0000 7f9b0513c640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-19T05:18:14.995072+0000 front 2024-09-19T05:18:14.995022+0000 (oldest deadline 2024-09-19T05:18:36.694902+0000) 2024-09-19T05:18:46.777 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:46.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:46.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:46.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:46.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:46.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:46.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:46 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:46.341+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:47.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:47 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:47.331+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:47.835 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:47 smithi103 podman[70802]: 2024-09-19 05:18:47.539879276 +0000 UTC m=+5.564364324 container died b3bfce76667efe68ded467deff08dc4a31376e4e7b81c7fd4cea103b6ebb29a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T05:18:48.177 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:47 smithi103 podman[70802]: 2024-09-19 05:18:47.90865344 +0000 UTC m=+5.933138487 container cleanup b3bfce76667efe68ded467deff08dc4a31376e4e7b81c7fd4cea103b6ebb29a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True) 2024-09-19T05:18:48.178 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:47 smithi103 bash[70802]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10 2024-09-19T05:18:48.469 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70819]: 2024-09-19 05:18:48.176798692 +0000 UTC m=+0.639520186 container remove b3bfce76667efe68ded467deff08dc4a31376e4e7b81c7fd4cea103b6ebb29a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T05:18:48.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:48.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:48.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:48.778 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:48.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:48.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:48.779 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:48 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:48.337+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:48.790 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70909]: 2024-09-19 05:18:48.427208406 +0000 UTC m=+0.015851918 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:48.790 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70909]: 2024-09-19 05:18:48.552558527 +0000 UTC m=+0.141202030 container create b13129f74796423f8ba5f51c04290a829edebeb954d8aa9bd663a55a8d49fdc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T05:18:48.790 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70909]: 2024-09-19 05:18:48.788270253 +0000 UTC m=+0.376913760 container init b13129f74796423f8ba5f51c04290a829edebeb954d8aa9bd663a55a8d49fdc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T05:18:49.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70909]: 2024-09-19 05:18:48.792271564 +0000 UTC m=+0.380915070 container start b13129f74796423f8ba5f51c04290a829edebeb954d8aa9bd663a55a8d49fdc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:18:49.219 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70909]: 2024-09-19 05:18:48.905047725 +0000 UTC m=+0.493691236 container attach b13129f74796423f8ba5f51c04290a829edebeb954d8aa9bd663a55a8d49fdc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T05:18:49.220 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 19 05:18:48 smithi103 podman[70909]: 2024-09-19 05:18:48.989891128 +0000 UTC m=+0.578534638 container died b13129f74796423f8ba5f51c04290a829edebeb954d8aa9bd663a55a8d49fdc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:18:49.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T05:18:49.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:49.624 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T05:18:49.624 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T05:18:49.624 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.11 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:49.654 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.312+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:49.913 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 systemd[1]: Stopping Ceph osd.11 for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:50.269 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.911+0000 7fb60d4e8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:50.269 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.911+0000 7fb60d4e8640 -1 osd.11 92 *** Got signal Terminated *** 2024-09-19T05:18:50.269 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:49 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:49.911+0000 7fb60d4e8640 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:18:50.269 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:50.269 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:50.269 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:50.270 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:50.270 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:50.270 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:50.270 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:50 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:50.267+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:51.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:51 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:51.239+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:52.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:52 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:52.216+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:53.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:53.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:53.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:53.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:53.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:53.529 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:53.529 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:53 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:53.251+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-19T05:17:26.202807+0000 front 2024-09-19T05:17:26.202884+0000 (oldest deadline 2024-09-19T05:17:51.502751+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-19T05:17:36.605073+0000 front 2024-09-19T05:17:36.605140+0000 (oldest deadline 2024-09-19T05:17:58.205421+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T05:17:42.306202+0000 front 2024-09-19T05:17:42.306243+0000 (oldest deadline 2024-09-19T05:18:07.506836+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-19T05:17:50.407917+0000 front 2024-09-19T05:17:50.407977+0000 (oldest deadline 2024-09-19T05:18:14.507831+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-19T05:18:00.408880+0000 front 2024-09-19T05:18:00.408814+0000 (oldest deadline 2024-09-19T05:18:24.508769+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T05:18:08.009647+0000 front 2024-09-19T05:18:08.009689+0000 (oldest deadline 2024-09-19T05:18:29.709730+0000) 2024-09-19T05:18:54.528 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:54 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11[59737]: 2024-09-19T05:18:54.210+0000 7fb609aed640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-19T05:18:22.512195+0000 front 2024-09-19T05:18:22.512318+0000 (oldest deadline 2024-09-19T05:18:44.812142+0000) 2024-09-19T05:18:55.638 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:55 smithi174 podman[71262]: 2024-09-19 05:18:55.369615319 +0000 UTC m=+5.549816538 container died dbfe523371516cf45916e93e9bcdd76660894a334201eff8fb7768c2cc76d424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T05:18:56.005 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:55 smithi174 podman[71262]: 2024-09-19 05:18:55.719313648 +0000 UTC m=+5.899514864 container cleanup dbfe523371516cf45916e93e9bcdd76660894a334201eff8fb7768c2cc76d424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T05:18:56.005 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:55 smithi174 bash[71262]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11 2024-09-19T05:18:56.277 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71278]: 2024-09-19 05:18:56.00385924 +0000 UTC m=+0.627649802 container remove dbfe523371516cf45916e93e9bcdd76660894a334201eff8fb7768c2cc76d424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T05:18:56.608 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71367]: 2024-09-19 05:18:56.329810268 +0000 UTC m=+0.111227413 container create 464ced723399e492933ac3e24f8be3d0721e7e69acd6421f1e50d284936d2ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True) 2024-09-19T05:18:56.608 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71367]: 2024-09-19 05:18:56.234220746 +0000 UTC m=+0.015637901 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:18:56.608 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71367]: 2024-09-19 05:18:56.53214818 +0000 UTC m=+0.313565335 container init 464ced723399e492933ac3e24f8be3d0721e7e69acd6421f1e50d284936d2ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T05:18:56.608 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71367]: 2024-09-19 05:18:56.535324889 +0000 UTC m=+0.316742035 container start 464ced723399e492933ac3e24f8be3d0721e7e69acd6421f1e50d284936d2ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:18:56.608 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71367]: 2024-09-19 05:18:56.607026233 +0000 UTC m=+0.388443383 container attach 464ced723399e492933ac3e24f8be3d0721e7e69acd6421f1e50d284936d2ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:56.936 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 05:18:56 smithi174 podman[71367]: 2024-09-19 05:18:56.72642589 +0000 UTC m=+0.507843049 container died 464ced723399e492933ac3e24f8be3d0721e7e69acd6421f1e50d284936d2ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T05:18:57.248 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T05:18:57.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:57.328 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T05:18:57.328 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T05:18:57.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.a 2024-09-19T05:18:57.638 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:18:57 smithi064 systemd[1]: Stopping Ceph mds.a for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:57.900 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:18:57 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a[70382]: 2024-09-19T05:18:57.636+0000 7f95018f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:57.900 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:18:57 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a[70382]: 2024-09-19T05:18:57.636+0000 7f95018f0640 -1 mds.a *** got signal Terminated *** 2024-09-19T05:19:00.185 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:18:59 smithi064 podman[109073]: 2024-09-19 05:18:59.921115122 +0000 UTC m=+2.391501238 container died 4e6b160f101019f8dd2ba41854e0f29b7eba3a6ba7bae0c6820a68639af14b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T05:19:00.635 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:19:00 smithi064 podman[109073]: 2024-09-19 05:19:00.383263279 +0000 UTC m=+2.853649428 container cleanup 4e6b160f101019f8dd2ba41854e0f29b7eba3a6ba7bae0c6820a68639af14b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T05:19:00.635 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 19 05:19:00 smithi064 bash[109073]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-a 2024-09-19T05:19:00.771 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T05:19:00.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:00.805 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T05:19:00.805 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T05:19:00.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.d 2024-09-19T05:19:00.900 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:00 smithi064 systemd[1]: Stopping Ceph mds.d for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:01.400 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:01 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d[69200]: 2024-09-19T05:19:01.129+0000 7f59fb96a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:01.401 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:01 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d[69200]: 2024-09-19T05:19:01.129+0000 7f59fb96a640 -1 mds.d *** got signal Terminated *** 2024-09-19T05:19:05.400 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 podman[109194]: 2024-09-19 05:19:05.141416438 +0000 UTC m=+4.114869980 container died 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T05:19:05.672 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 podman[109194]: 2024-09-19 05:19:05.478649058 +0000 UTC m=+4.452102599 container cleanup 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T05:19:05.672 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 bash[109194]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d 2024-09-19T05:19:05.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T05:19:05.943 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 podman[109206]: 2024-09-19 05:19:05.671366261 +0000 UTC m=+0.528376476 container remove 27295cfa4a3b0f1d02bc3fd51c26babc681358010ac905768b1da129d456f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T05:19:05.944 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-19T05:19:05.944 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 systemd[1]: Stopped Ceph mds.d for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:19:05.944 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 19 05:19:05 smithi064 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.d.service: Consumed 1.065s CPU time. 2024-09-19T05:19:05.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:05.955 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T05:19:05.955 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T05:19:05.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.g 2024-09-19T05:19:06.291 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:19:06 smithi064 systemd[1]: Stopping Ceph mds.g for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:06.650 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:19:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g[69966]: 2024-09-19T05:19:06.290+0000 7f1bbfcd6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:06.650 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:19:06 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g[69966]: 2024-09-19T05:19:06.290+0000 7f1bbfcd6640 -1 mds.g *** got signal Terminated *** 2024-09-19T05:19:08.542 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:19:08 smithi064 podman[109317]: 2024-09-19 05:19:08.25572195 +0000 UTC m=+2.072432755 container died 648d649d97a1249acc8fc758fb37d763b6084c992b8935db230c209587c64872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:19:08.850 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:19:08 smithi064 podman[109317]: 2024-09-19 05:19:08.623323172 +0000 UTC m=+2.440033957 container cleanup 648d649d97a1249acc8fc758fb37d763b6084c992b8935db230c209587c64872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:19:08.851 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 19 05:19:08 smithi064 bash[109317]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-g 2024-09-19T05:19:08.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T05:19:09.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:09.018 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T05:19:09.018 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T05:19:09.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.j 2024-09-19T05:19:09.400 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 19 05:19:09 smithi064 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-j[68789]: 2024-09-19T05:19:09.343+0000 7f0fba966640 -1 mds.j *** got signal Terminated *** 2024-09-19T05:19:13.620 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 19 05:19:13 smithi064 podman[109440]: 2024-09-19 05:19:13.357739817 +0000 UTC m=+4.111562966 container died 34fcded5e19320b1a29dd5269f01770fb9e8b20a3ccff70ce9e387543d03c867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T05:19:13.900 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 19 05:19:13 smithi064 podman[109440]: 2024-09-19 05:19:13.701881341 +0000 UTC m=+4.455704483 container cleanup 34fcded5e19320b1a29dd5269f01770fb9e8b20a3ccff70ce9e387543d03c867 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T05:19:13.900 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 19 05:19:13 smithi064 bash[109440]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-j 2024-09-19T05:19:14.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T05:19:14.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:14.182 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T05:19:14.182 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T05:19:14.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.b 2024-09-19T05:19:14.469 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:19:14 smithi103 systemd[1]: Stopping Ceph mds.b for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:14.793 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:19:14 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: 2024-09-19T05:19:14.533+0000 7f35f0bde640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:14.793 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:19:14 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b[64577]: 2024-09-19T05:19:14.533+0000 7f35f0bde640 -1 mds.b *** got signal Terminated *** 2024-09-19T05:19:16.961 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:19:16 smithi103 podman[71016]: 2024-09-19 05:19:16.683763947 +0000 UTC m=+2.286490703 container died bad5c262f7406920a5cb4986fd8f73f0057a867f7b8eead3443655b443e46679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T05:19:17.218 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:19:17 smithi103 podman[71016]: 2024-09-19 05:19:17.04291332 +0000 UTC m=+2.645640076 container cleanup bad5c262f7406920a5cb4986fd8f73f0057a867f7b8eead3443655b443e46679 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:19:17.219 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 19 05:19:17 smithi103 bash[71016]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-b 2024-09-19T05:19:17.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T05:19:17.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:17.525 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T05:19:17.525 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T05:19:17.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.e 2024-09-19T05:19:17.848 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:19:17 smithi103 systemd[1]: Stopping Ceph mds.e for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:18.219 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:19:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e[63965]: 2024-09-19T05:19:17.845+0000 7f32b1563640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:18.588 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:19:17 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e[63965]: 2024-09-19T05:19:17.845+0000 7f32b1563640 -1 mds.e *** got signal Terminated *** 2024-09-19T05:19:20.939 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:19:20 smithi103 podman[71148]: 2024-09-19 05:19:20.67786624 +0000 UTC m=+2.936965139 container died 24163245ae70915e17c429c710054084b1fb711bfde6f5a5b35c9062964f76ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T05:19:21.219 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:19:21 smithi103 podman[71148]: 2024-09-19 05:19:21.021485404 +0000 UTC m=+3.280584276 container cleanup 24163245ae70915e17c429c710054084b1fb711bfde6f5a5b35c9062964f76ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T05:19:21.219 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 19 05:19:21 smithi103 bash[71148]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-e 2024-09-19T05:19:21.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T05:19:21.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:21.481 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T05:19:21.481 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T05:19:21.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.h 2024-09-19T05:19:21.793 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:21 smithi103 systemd[1]: Stopping Ceph mds.h for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:22.232 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h[64272]: 2024-09-19T05:19:21.792+0000 7fb73bdb6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:22.233 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:21 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h[64272]: 2024-09-19T05:19:21.792+0000 7fb73bdb6640 -1 mds.h *** got signal Terminated *** 2024-09-19T05:19:24.944 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:24 smithi103 podman[71268]: 2024-09-19 05:19:24.68286198 +0000 UTC m=+2.986445890 container died d9d7bc7fde38f5d621649f0c25fa7dab81efd16d5e0926d5332175226c88c54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-19T05:19:25.221 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:25 smithi103 podman[71268]: 2024-09-19 05:19:25.016897994 +0000 UTC m=+3.320481902 container cleanup d9d7bc7fde38f5d621649f0c25fa7dab81efd16d5e0926d5332175226c88c54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T05:19:25.221 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:25 smithi103 bash[71268]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h 2024-09-19T05:19:25.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T05:19:25.516 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:25 smithi103 podman[71281]: 2024-09-19 05:19:25.24316928 +0000 UTC m=+0.561135034 container remove d9d7bc7fde38f5d621649f0c25fa7dab81efd16d5e0926d5332175226c88c54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T05:19:25.516 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:25 smithi103 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-19T05:19:25.516 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:25 smithi103 systemd[1]: Stopped Ceph mds.h for 5ccce474-7643-11ef-bceb-c7b262605968. 2024-09-19T05:19:25.516 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 19 05:19:25 smithi103 systemd[1]: ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.h.service: Consumed 2.067s CPU time. 2024-09-19T05:19:25.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:25.527 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T05:19:25.527 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T05:19:25.527 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.k 2024-09-19T05:19:25.830 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:19:25 smithi103 systemd[1]: Stopping Ceph mds.k for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:26.218 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:19:25 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k[63651]: 2024-09-19T05:19:25.829+0000 7f6462556640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:26.219 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:19:25 smithi103 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k[63651]: 2024-09-19T05:19:25.829+0000 7f6462556640 -1 mds.k *** got signal Terminated *** 2024-09-19T05:19:30.219 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:19:29 smithi103 podman[71389]: 2024-09-19 05:19:29.837839768 +0000 UTC m=+4.086341822 container died dd2d8a6ada26d080841763691dce9679eb602a7f74408dcf2575f9b3957e13a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T05:19:30.564 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:19:30 smithi103 podman[71389]: 2024-09-19 05:19:30.378638208 +0000 UTC m=+4.627140257 container cleanup dd2d8a6ada26d080841763691dce9679eb602a7f74408dcf2575f9b3957e13a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T05:19:30.564 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 19 05:19:30 smithi103 bash[71389]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-k 2024-09-19T05:19:30.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T05:19:30.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:30.817 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T05:19:30.817 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T05:19:30.817 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.c 2024-09-19T05:19:31.119 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:19:30 smithi174 systemd[1]: Stopping Ceph mds.c for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:31.528 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:19:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c[64562]: 2024-09-19T05:19:31.117+0000 7f0a6a22f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:31.528 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:19:31 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c[64562]: 2024-09-19T05:19:31.117+0000 7f0a6a22f640 -1 mds.c *** got signal Terminated *** 2024-09-19T05:19:32.952 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:19:32 smithi174 podman[71471]: 2024-09-19 05:19:32.688063259 +0000 UTC m=+1.666306773 container died 932430102e2c7cd8c59b4b5ead678d49e61bf554947af871a735856b615494e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T05:19:33.277 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:19:33 smithi174 podman[71471]: 2024-09-19 05:19:33.04243298 +0000 UTC m=+2.020676480 container cleanup 932430102e2c7cd8c59b4b5ead678d49e61bf554947af871a735856b615494e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:19:33.278 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 05:19:33 smithi174 bash[71471]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-c 2024-09-19T05:19:33.476 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T05:19:33.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:33.512 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T05:19:33.512 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T05:19:33.512 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.f 2024-09-19T05:19:34.028 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:19:33 smithi174 systemd[1]: Stopping Ceph mds.f for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:34.028 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:19:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f[64262]: 2024-09-19T05:19:33.821+0000 7fea5ea55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:34.028 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:19:33 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f[64262]: 2024-09-19T05:19:33.821+0000 7fea5ea55640 -1 mds.f *** got signal Terminated *** 2024-09-19T05:19:36.972 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:19:36 smithi174 podman[71604]: 2024-09-19 05:19:36.686699577 +0000 UTC m=+2.959827146 container died 15f2a2c6a0bd01100edcedded4e154344234c5c1d0a73b7d665dce0929eb6900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T05:19:37.278 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:19:37 smithi174 podman[71604]: 2024-09-19 05:19:37.053983658 +0000 UTC m=+3.327111216 container cleanup 15f2a2c6a0bd01100edcedded4e154344234c5c1d0a73b7d665dce0929eb6900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-19T05:19:37.278 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 05:19:37 smithi174 bash[71604]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-f 2024-09-19T05:19:37.493 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T05:19:37.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:37.527 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T05:19:37.527 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T05:19:37.527 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.i 2024-09-19T05:19:37.850 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:37 smithi174 systemd[1]: Stopping Ceph mds.i for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:38.277 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i[64864]: 2024-09-19T05:19:37.849+0000 7fab23e62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:38.278 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:37 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i[64864]: 2024-09-19T05:19:37.849+0000 7fab23e62640 -1 mds.i *** got signal Terminated *** 2024-09-19T05:19:41.008 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:40 smithi174 podman[71725]: 2024-09-19 05:19:40.67590039 +0000 UTC m=+2.929723763 container died 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:19:41.008 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:41 smithi174 podman[71725]: 2024-09-19 05:19:41.007301925 +0000 UTC m=+3.261125297 container cleanup 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:19:41.277 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:41 smithi174 bash[71725]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i 2024-09-19T05:19:41.278 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 05:19:41 smithi174 podman[71737]: 2024-09-19 05:19:41.158430272 +0000 UTC m=+0.479228197 container remove 7b1dee635a8b582a227dc51486a4bf4de08c19b644ca3f065e398e43df886ecc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-i, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:19:41.389 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T05:19:41.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:41.423 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T05:19:41.423 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T05:19:41.423 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.l 2024-09-19T05:19:41.728 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:19:41 smithi174 systemd[1]: Stopping Ceph mds.l for 5ccce474-7643-11ef-bceb-c7b262605968... 2024-09-19T05:19:42.028 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:19:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l[63964]: 2024-09-19T05:19:41.726+0000 7f2cd224b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:19:42.028 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:19:41 smithi174 ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l[63964]: 2024-09-19T05:19:41.726+0000 7f2cd224b640 -1 mds.l *** got signal Terminated *** 2024-09-19T05:19:44.420 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:19:44 smithi174 podman[71846]: 2024-09-19 05:19:44.15743631 +0000 UTC m=+2.519104204 container died 2ebaa3649d6d5a0f0dd526ba6e792b51c98d442abae5adb94bf7172e46d4e263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T05:19:44.420 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:19:44 smithi174 podman[71846]: 2024-09-19 05:19:44.418852708 +0000 UTC m=+2.780520594 container cleanup 2ebaa3649d6d5a0f0dd526ba6e792b51c98d442abae5adb94bf7172e46d4e263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T05:19:44.670 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 05:19:44 smithi174 bash[71846]: ceph-5ccce474-7643-11ef-bceb-c7b262605968-mds-l 2024-09-19T05:19:44.834 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5ccce474-7643-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T05:19:44.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:19:44.878 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T05:19:44.878 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:19:45.102 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:19:47.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:19:47.770 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:19:50.321 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:19:50.521 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:19:52.886 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:19:52.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:19:52.952 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:19:52.978 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T05:19:52.980 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi064/crash 2024-09-19T05:19:52.981 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:19:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:19:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:19:53.024 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi103/crash 2024-09-19T05:19:53.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:19:53.066 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:19:53.066 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:19:53.068 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi174/crash 2024-09-19T05:19:53.069 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:19:53.107 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/5ccce474-7643-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:19:53.108 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:19:53.109 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T05:19:53.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T05:19:53.151 INFO:tasks.cephadm:Compressing logs... 2024-09-19T05:19:53.151 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:19:53.194 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:19:53.197 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:19:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-19T05:19:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:: No such file or directory 2024-09-19T05:19:53.219 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T05:19:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T05:19:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:19:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:19:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T05:19:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:19:53.222 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T05:19:53.222 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-19T05:19:53.222 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T05:19:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T05:19:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log: 84.4% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:19:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T05:19:53.223 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:19:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T05:19:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:19:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T05:19:53.224 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T05:19:53.230 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T05:19:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T05:19:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:19:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T05:19:53.238 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:19:53.238 INFO:teuthology.orchestra.run.smithi174.stderr:: No such file or directory 2024-09-19T05:19:53.238 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:19:53.238 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T05:19:53.238 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.11.log: 81.6%gzip -5 -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log.gz --verbose 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr: -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.5.log: /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T05:19:53.239 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.l.log: 87.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:19:53.240 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T05:19:53.240 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T05:19:53.240 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T05:19:53.240 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.z.log: 91.0% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:19:53.244 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:19:53.244 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T05:19:53.244 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T05:19:53.244 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.d.log: 87.5% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:19:53.244 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T05:19:53.244 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T05:19:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T05:19:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:19:53.245 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr: 94.5% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log: 81.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log: 87.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.h.log: 91.0% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr: 91.9% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T05:19:53.246 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T05:19:53.247 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:19:53.248 INFO:teuthology.orchestra.run.smithi174.stderr: 92.0% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T05:19:53.249 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T05:19:53.249 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:19:53.250 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T05:19:53.253 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T05:19:53.261 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:19:53.262 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:19:53.271 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:19:53.290 INFO:teuthology.orchestra.run.smithi174.stderr: 92.8% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T05:19:53.298 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:19:53.305 INFO:teuthology.orchestra.run.smithi103.stderr: 94.5% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T05:19:53.305 INFO:teuthology.orchestra.run.smithi174.stderr: 92.9% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T05:19:53.308 INFO:teuthology.orchestra.run.smithi103.stderr: 94.4% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T05:19:53.542 INFO:teuthology.orchestra.run.smithi174.stderr: 91.7% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T05:19:53.627 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T05:19:53.665 INFO:teuthology.orchestra.run.smithi174.stderr: 95.8% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T05:19:53.696 INFO:teuthology.orchestra.run.smithi103.stderr: 91.8% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T05:19:53.963 INFO:teuthology.orchestra.run.smithi174.stderr: 93.0% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T05:19:53.971 INFO:teuthology.orchestra.run.smithi174.stderr: 93.3% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T05:19:54.056 INFO:teuthology.orchestra.run.smithi174.stderr: 93.2% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T05:19:54.110 INFO:teuthology.orchestra.run.smithi103.stderr: 93.1% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T05:19:54.129 INFO:teuthology.orchestra.run.smithi103.stderr: 92.7% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T05:19:54.156 INFO:teuthology.orchestra.run.smithi103.stderr: 93.2% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T05:19:54.239 INFO:teuthology.orchestra.run.smithi103.stderr: 93.1% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T05:19:54.255 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T05:19:54.257 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T05:19:54.288 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T05:19:54.339 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T05:19:54.369 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T05:19:55.356 INFO:teuthology.orchestra.run.smithi174.stderr: 96.0% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T05:19:55.358 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-09-19T05:19:55.359 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m2.144s 2024-09-19T05:19:55.359 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m5.155s 2024-09-19T05:19:55.359 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m0.202s 2024-09-19T05:19:56.998 INFO:teuthology.orchestra.run.smithi103.stderr: 93.6% -- replaced with /var/log/ceph/5ccce474-7643-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T05:19:56.999 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-19T05:19:56.999 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m3.789s 2024-09-19T05:19:57.000 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m7.877s 2024-09-19T05:19:57.000 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.281s 2024-09-19T05:20:28.518 INFO:teuthology.orchestra.run.smithi064.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-19T05:20:28.520 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-09-19T05:20:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m35.313s 2024-09-19T05:20:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m39.573s 2024-09-19T05:20:28.520 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m1.433s 2024-09-19T05:20:28.520 INFO:tasks.cephadm:Archiving logs... 2024-09-19T05:20:28.521 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi064/log 2024-09-19T05:20:28.522 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:20:34.631 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi103/log 2024-09-19T05:20:34.632 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:20:35.761 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi174/log 2024-09-19T05:20:35.762 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:20:36.418 INFO:tasks.cephadm:Removing cluster... 2024-09-19T05:20:36.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --force 2024-09-19T05:20:36.631 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:20:36.902 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --force 2024-09-19T05:20:37.104 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:20:37.367 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5ccce474-7643-11ef-bceb-c7b262605968 --force 2024-09-19T05:20:37.569 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 5ccce474-7643-11ef-bceb-c7b262605968 2024-09-19T05:20:37.832 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T05:20:37.832 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:20:37.851 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:20:37.867 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:20:37.883 INFO:tasks.cephadm:Teardown complete 2024-09-19T05:20:37.884 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T05:20:37.895 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T05:20:37.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:20:37.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:20:37.910 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:20:38.013 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T05:20:38.025 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T05:20:38.026 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:20:38.029 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:20:38.031 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:20:38.043 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:20:38.043 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:20:38.046 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:20:38.048 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:20:38.048 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-19T05:20:38.048 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 -76us[ -74us] +/- 48ms 2024-09-19T05:20:38.048 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 +392us[ +392us] +/- 77ms 2024-09-19T05:20:38.048 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -233us[ -233us] +/- 70ms 2024-09-19T05:20:38.048 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:20:38.049 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:20:38.049 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-19T05:20:38.049 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 -61us[ -49us] +/- 48ms 2024-09-19T05:20:38.049 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +392us[ +392us] +/- 77ms 2024-09-19T05:20:38.050 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -217us[ -217us] +/- 70ms 2024-09-19T05:20:38.050 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:20:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:20:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-19T05:20:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 -73us[ -63us] +/- 48ms 2024-09-19T05:20:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +391us[ +401us] +/- 77ms 2024-09-19T05:20:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -218us[ -208us] +/- 70ms 2024-09-19T05:20:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:20:38.051 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T05:20:38.090 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T05:20:38.091 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T05:20:38.136 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:20:38.223 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:20:38.291 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:20:38.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:20:38.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:38.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:20:38.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:38.420 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:20:38.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:38.449 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T05:20:38.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T05:20:38.470 INFO:teuthology.task.internal:Duration was 1849.434907 seconds 2024-09-19T05:20:38.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T05:20:38.480 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T05:20:38.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:20:38.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:20:38.485 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:20:38.521 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:20:38.522 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:20:38.529 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:20:39.008 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T05:20:39.009 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-09-19T05:20:39.009 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:20:39.031 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T05:20:39.031 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:20:39.076 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-09-19T05:20:39.076 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:20:39.097 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T05:20:39.098 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:20:39.100 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:20:39.120 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:20:39.164 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T05:20:39.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:20:39.168 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:20:39.187 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:20:40.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T05:20:40.344 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T05:20:40.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:20:40.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:20:40.401 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:20:40.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T05:20:40.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:20:40.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:20:40.447 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:20:40.466 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-09-19T05:20:40.472 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-19T05:20:40.497 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-09-19T05:20:40.525 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:20:40.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:40.540 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:20:40.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:40.581 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:20:40.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:40.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T05:20:40.609 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T05:20:40.609 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi064 2024-09-19T05:20:40.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:20:40.670 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi103 2024-09-19T05:20:40.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:20:40.710 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911652/remote/smithi174 2024-09-19T05:20:40.710 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:20:40.748 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T05:20:40.749 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:20:40.751 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:20:40.754 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:20:40.798 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T05:20:40.811 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T05:20:40.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T05:20:40.823 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T05:20:40.823 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:20:40.826 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:20:40.829 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:20:40.839 INFO:teuthology.orchestra.run.smithi064.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:20 /home/ubuntu/cephtest 2024-09-19T05:20:40.842 INFO:teuthology.orchestra.run.smithi103.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:20 /home/ubuntu/cephtest 2024-09-19T05:20:40.854 INFO:teuthology.orchestra.run.smithi174.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:20 /home/ubuntu/cephtest 2024-09-19T05:20:40.855 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T05:20:40.869 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T05:20:40.893 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} duration: 1849.4349069595337 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T05:20:40.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:20:40.976 INFO:teuthology.run:pass